Assessment of the Performance of Imputation Techniques in Observational Studies with Two Measurements

Abstract

In observational studies with two measurements when the measured outcome pertains to a health related quality of life (HRQoL) variable, one motivation of the research may be to determine the potential predictors of the mean change of the outcome of interest. It is very common in such studies for data to be missing, which can bias the results. Different imputation techniques have been proposed to cope with missing data in outcome variables. We compared five analysis approaches (Complete Case, Available Case, K- Nearest Neighbour, Propensity Score, and a Markov Chain Monte Carlo algorithm) to assess their performance when handling missing data at different missingness rates and mechanisms (MCAR, MAR and MNAR). These strategies were applied to a pre-post study of patients with Chronic Obstructive Pulmonary Disease. We analyzed the relationship of the changes in subjects HRQoL over one year with clinical and socio-demographic characteristics. A simulation study was also performed to illustrate the performance of the imputation methods. Relative and standardized bias was assessed on each scenario. For all missingness mechanisms, not imputing and using MCMC method, both combined with mixed-model analysis, showed lowest standardized bias. Conversely, Propensity Score showed worst bias values. When missingness pattern is MCAR or MAR and rate small, we recommend using mixed models. Nevertheless, when missingness percentage is high, in order to gain sample size and statistical power, MCMC is preferred, although there are no bias differences compared with the mixed models without imputation. For a MNAR scenario, a further sensitivity analysis should be made.