A large fraction (some claim > 1/2) of published research in top journals in applied sciences such as medicine and psychology is irreproduceable. In light of this ‘replicability crisis’, standard p-value based hypothesis testing has come under intense scrutiny. One of its many problems is the following: if our test result is promising but nonconclusive…

Repeated measures Anova or Manova are frequently used for analyzing event-related brain potentials. They are typically performed on averaged repeated stimulus trials as a way of increasing the reliability of the electroencephalogram signal. Averaging, however, leads to information loss concerning the covariance matrix of random individual differences of participant treatment and time effects, which could…

The social sciences are amidst a paradigm shift towards open science. In part, this transition has been fueled by cases of scientific fraud and increasing awareness of questionable research practices. However, Open science is not merely a cure (or punishment) for this crisis – it is also an opportunity. Technological advances enable researchers to more…

Bayesian methods have repeatedly shown to be advantageous for small-sample situations. To benefit from these advantages, researchers are required to quantify existing background information in informative prior distributions, which are currently used only scarcely. A prominent reason for this may be the distinct heterogeneity of studies in psychological and educational research. Studies are being conducted…

Scientific surveys based on random probability samples are ubiquitously used in the social sciences to study and describe large populations. They provide a critical source of quantifiable information used by governments and policy-makers to make informed decisions. However, probability-based surveys are increasingly expensive to carry out and declining response rates observed over recent decades have…

Scholars are confronted with ever-larger amounts of textual data. All this data present new and unique opportunities to scholars, while simultaneously confronting them with unprecedented challenges. How to select relevant text effectively and efficiently from an almost unlimited amount of data? Conducting a systematic review on this data is often a very time consuming and…

Sequential estimation is a well-recognized approach to inference in statistical theory. In sequential estimation, the sample size for a study is not prespecified before the study begins, rather the study itself informs when researchers should stop sampling, namely when a stopping rule has been satisfied. In particular, data in sequential estimation procedures are collected in…

Multilevel models (i.e., mixed-effects models) are used to predict outcomes with one or more sources of dependency, such as in clustered observations or repeated measures. In frequentist settings, the dominant estimation method for multilevel models with normally distributed residuals at each level (i.e., general linear mixed-effects models) is residual maximum likelihood (REML), which provides unbiased…

Differential item functioning (DIF) is a pernicious statistical issue that can mask true group differences on a target latent construct. A considerable amount of research has focused on evaluating methods for testing DIF, such as using likelihood ratio tests in item response theory (IRT). Most of this research has focused on the asymptotic properties of…

We often find multiple studies investigating the same intervention on different groups of patients. Combining data from these trials allows us to share information about the effect of the intervention in a.) one of the specific patient groups; and/or b.) the general patient population. The aim of information sharing between trials is often twofold: A…