Caroline Maria Böhm

• Discussions of the replication crisis in psychology and related disciplines have focused on statistical power, analytic flexibility, publication bias, and questionable research practices as contributors to low replication rates. In contrast, the role of measurement practices (how constructs are operationalized, how scales are modified, and how measurement decisions are reported) has received comparatively little systematic attention. This dissertation investigates the prevalence of measurement practices in replication research and how they impact the replicability of empirical findings across two projects. Project 1 is a systematic review of measurement practices in 100 direct replication studies from psychology, economics, and political science. Two independent raters coded measurement and context characteristics for each study. Discrepancies were resolved by a third coder. Results show that measurement modifications were widespread. The majority of replication studies modified at least one scale property, most commonly through translation, response format changes, and item wording changes. In addition, over three quarters of all coded instruments were ad hoc measures without clear links to validated instruments. Reporting transparency was often insufficient for replication researchers to reconstruct measurement procedures. Consistent with this, inter-rater reliability was notably lower for variables requiring direct comparison between original and replication studies, which we interpret as reflecting the greater coding effort these variables demand when reporting is incomplete. Project 2 quantifies the consequences of measurement modification using two large panel datasets (LISS from the Netherlands, MIDUS from the United States), and is organized into two studies. Study 1 examines the computational reproducibility of 130 published effects drawn from these datasets. Reproducibility rates varied substantially with code availability: effects accompanied by executable code reproduced more accurately than those requiring re-implementation from the methods section alone. Study 2 draws on the reproduced effects as baselines and applies two common modifications post hoc while holding all other factors constant: item dropping (at two levels of reduction, using three selection strategies) and collapsing of response categories. Both modifications reduced replication success and increased effect size heterogeneity. Roughly one in five previously significant effects lost significance or reversed direction, corresponding to a 15–25% reduction in frequentist replication success. Bayesian signal retention ratios indicated larger losses of approximately 30%. Frequentist and Bayesian indicators diverged systematically, suggesting that whether measurement modification is detected depends on the success criterion applied. No item selection strategy reliably outperformed the others. Together, the two projects show that the types of modifications commonly found in replication research have measurable consequences for replication outcomes. These findings identify measurement practices as a neglected but empirically consequential source of variability in replication success, and they point to transparent measurement reporting as a precondition for meaningful interpretation of replication results.