Common Statistical Mistakes in PhD Research

1. Choosing the Statistical Test Before Defining the Research Question

Start with the research question, not with a preferred statistical test. The objective and study design should guide the analysis.

2. Using the Wrong Statistical Test

Two independent groups may require a t-test or appropriate alternative; three or more groups may require ANOVA; binary outcomes may require logistic regression; time-to-event outcomes may require survival analysis; repeated measurements may require mixed-effects models.

3. Ignoring Study Design

Cross-sectional, cohort, case-control, randomized and longitudinal studies have different analytical requirements. Statistical planning should begin during study design.

4. Performing Multiple Tests Without Considering Multiplicity

Running many tests and reporting only significant findings can increase false-positive risk. Appropriate methods for accounting for multiplicity may be required.

5. Reporting Only P-Values

A p-value alone does not tell you how large or meaningful an effect is. Consider effect estimates, confidence intervals, p-values and relevant descriptive statistics.

6. Confusing Statistical Significance With Practical Significance

A statistically significant result is not necessarily an important result. Consider the magnitude and practical relevance of an effect.

7. Ignoring Missing Data

Deleting participants with missing values may introduce bias and reduce power. Investigate the amount, pattern and likely mechanism of missingness before choosing an approach.

8. Treating Repeated Measurements as Independent

Measurements from the same participant are correlated. Mixed-effects models or MMRM may be appropriate depending on the research design.

9. Ignoring Model Assumptions

Depending on the method, assumptions may include independence, linearity, normality of residuals, homogeneity of variance or proportional hazards. Assess relevant assumptions.

10. Changing the Analysis After Seeing the Results

Trying different analyses until a significant result appears can increase false-positive findings. For confirmatory research, specify the primary analysis before examining results when feasible.

11. Using Correlation to Claim Causation

A significant correlation does not prove that one variable causes another. Causal conclusions require an appropriate design and assumptions.

12. Overfitting the Statistical Model

Too many predictors relative to the available information can produce unstable models. Model complexity should be justified by objectives, sample size, observations and scientific knowledge.

13. Ignoring Confounding

An observed association may be partly explained by another variable. Potential confounders should be considered during design and analysis.

14. Reporting Results Without Explaining Them

Statistical output is not the same as scientific interpretation. A good thesis explains what was found, how large and precise the effect was, what it means scientifically, and what the limitations are.