Fundamental statistical methods

Fundamental statistical methods for IPD meta-analysis

Statistical methods for IPD meta-analysis are needed for a quantitative synthesis of the IPD.

These use either a two-stage or a one-stage approach to produce summary results (e.g. about treatment effect).
Recommendations for choosing between the approaches are available.

Below we provide key guidance, and for further details see Chapters 5 to 8 of our book and various references.

Two-stage approach

The first stage typically involves a standard regression analysis in each trial separately to produce aggregate data, such as treatment effect estimates and their variances.

The second stage uses well-known (e.g. inverse variance weighted) meta-analysis methods to combine this aggregate data and produce summary results and forest plots. Either a common-effect or a random-effects model is assumed.

A common-effect model assumes that the true treatment effect is the same in every trial.
A random-effects model allows for between-trial heterogeneity in the true treatment effect, and is more plausible because included trials often differ in their characteristics.

A frequentist or Bayesian estimation framework can be used.
Bayesian estimation is appealing, to produce direct probabilistic statements that account for all parameter uncertainty, and to include prior distributions for the between-trial variance.
In a frequentist framework, restricted maximum likelihood (REML) estimation is recommended for fitting the random-effects model in the second stage, with confidence intervals derived using the approach of Hartung-Knapp-Sidik-Jonkman.

Heterogeneity can be summarised by the estimate of between-trial variance of true treatment effects, and a 95% prediction interval for the potential true treatment effect in a new trial.
A meta-regression extends the random-effects model by including trial-level covariates (that define subgroups of trials) that may explain between-trial heterogeneity. However, meta-regression usually has low power and should be interpreted cautiously.

If appropriate, aggregate data from non-IPD trials can be included in the second stage of the two-stage meta-analysis, alongside the aggregate data derived directly from IPD trials.
This helps explore whether the main IPD meta-analysis results (based on IPD trials only) are robust to the inclusion of non-IPD trials. However, suitable aggregate data from non-IPD trials may be limited.

A two-stage IPD meta-analysis can be implemented using the ipdmetan package in Stata, for a wide variety of outcome types.

Two-stage

One-stage approach

A one-stage approach analyses the IPD from all trials altogether in a single statistical analysis.
This typically requires a generalised linear mixed model (GLMM) or a hierarchical survival (frailty) model, which extend standard models (such as linear, logistic, Poisson and Cox) used in a single trial setting.

A one-stage IPD meta-analysis utilises a more exact statistical likelihood than a two-stage meta-analysis approach, which is advantageous when included trials have few participants or outcome events.

One-stage IPD meta-analysis models usually include multiple parameters and these are estimated simultaneously. For each parameter (such as the intercept, treatment effect, residual variances).
The analyst must specify whether they are common (the same in each trial), stratified (different in each trial) or random (different in each trial and assumed drawn from a particular distribution).

Clustering of participants within trials must be accounted for (e.g. using a stratified trial intercept or random trial intercepts for GLMMs), as otherwise summary meta-analysis results may be biased or overly precise.
A stratified trial intercept is generally preferred, unless there are computational concerns.
The use of random trial intercepts allows information about baseline risk to be shared across trials, which may compromise randomisation within each trial (e.g. when the treatment:control allocation ratio differs across trials).

Restricted maximum likelihood (REML) estimation is recommended for one-stage models with continuous outcomes, with confidence intervals derived using the approach of Kenward-Roger or Satterthwaite.
For binary outcomes, unless most included trials have sparse numbers of events, REML estimation of a pseudo-likelihood is recommended.

Where REML or a pseudo-likelihood is not available or not appropriate, maximum likelihood estimation of a one-stage model with a stratified intercept can be improved by trial-specific centering of the treatment variable, and by deriving confidence intervals using the t-distribution.
Bayesian estimation of one-stage models is an appealing alternative to frequentist estimation methods, especially to enable direct probabilistic statements that account for all parameter uncertainty.

One-stage

Two-stage versus one-stage: differences and recommendations

Two-stage, or not two-stage, that is the question

Many articles have compared the one-stage and two-stage approaches via theory, simulation and empirical examples.
Crucially, differences in one-stage and two-stage summary results will usually be small when the same assumptions and estimation methods are used.

When notable differences do arise, generally they are caused by a change in modelling assumptions and/or estimation methods, and not due to using a one-stage or two-stage process per se.
Thus, the choice of model assumptions and estimation methods is usually more important than selection of a one-stage or two-stage approach.

An important exception is when most trials in the meta-analysis are small (in terms of numbers of participants or outcome events). In this case a one-stage approach is recommended as it uses a more exact statistical likelihood than that assumed in the second stage of the two-stage approach.

In other situations, researchers can feel free to choose either a one-stage or a two-stage approach, with due care given to their choice of modelling assumptions, parameter specification, and estimation methods.

Unless most trials in the IPD meta-analysis are small, generally the two-stage approach will suffice. It is also more accessible (especially for those familiar with conventional aggregate data meta-analysis approaches), and more easily enables visual summaries (e.g. forest plots). It is also more convenient for including trials that only provide remote access to their IPD (so cannot be merged with other IPD), and trials that only provide aggregate data.

It may be helpful to do both a one-stage and a two-stage analysis (and report both), to check whether conclusions are robust to the choice of approach.

1vs2