The Current State of Validation of Administrative Healthcare Databases in Italy: A Systematic Review

Authors

  • Iosief Abraha Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy
  • Massimiliano Orso Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy
  • Piero Grilli Azienda Sanitaria Locale USL 2, Medicina Generale, Terni, Italy
  • Francesco Cozzolino Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy
  • Paolo Eusebi Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy
  • Paola Casucci Regional Health Authority of Umbria, Health ICT Service, Perugia, Italy
  • Mauro Marchesi Azienda Ospedaliero-Universitaria di Perugia, Servizio Immunoematologia e Trasfusionale, Perugia, Italy
  • Maria Laura Luchetta Azienda Sanitaria Locale USL 2, Medicina Generale, Terni, Italy
  • Luisa Fruttini Istituto Zooprofilattico Sperimentale dell'Umbria e delle Marche, Unità Operativa Editoria Biblioteca, Perugia, Italy
  • Raoul Ciappelloni Istituto Zooprofilattico Sperimentale dell'Umbria e delle Marche, Unità Operativa Editoria Biblioteca, Perugia, Italy
  • Rita De Florio Azienda Sanitaria Locale USL 2, Medicina Generale, Terni, Italy
  • Gianni Giovannini Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy
  • Alessandro Montedori Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

DOI:

https://doi.org/10.6000/1929-6029.2014.03.03.10

Keywords:

Healthcare databases, Sensitivity, Specificity, Predictive values, Health administrative data, Diagnostic accuracy, Misclassification bias, Health services research, Epidemiology.

Abstract

Background: Administrative healthcare databases are widely present in Italy. Our aim was to describe the current state of healthcare databases validity in terms of discharge diagnoses (according to the International Classification of Diseases, ICD-9 code) and their output in terms of research.

Methods: A systematic search of electronic databases including Medline and Embase (1995-2013) and of local sources was performed. Inclusion criteria were: healthcare databases in any Italian territory routinely and passively collecting data; medical investigations or procedures at patient level data; the use of a validation process. The quality of studies was evaluated using the STARD criteria. Citations of the included studies were explored using Scopus and Google Scholar.

Results: The search strategy allowed the identification of 16 studies of which 3 were in Italian. Thirteen studies used regional administrative databases from Lombardia, Piemonte, Lazio, Friuli-Venezia Giulia and Veneto. The ICD-9 codes of the following diseases were successfully validated: amyotrophic lateral sclerosis (3 studies in four different regional administrative databases), stroke (3 studies), gastrointestinal bleeding (1 study), thrombocytopenia (1 study), epilepsy (1 study), infection (1 study), chronic obstructive pulmonary disease (1 study), Guillain-Barre syndrome (1 study), and cancer diseases (4 studies). The quality of reporting was variable among the studies. Only 6 administrative databases produced further research related to the validated ICD-9 codes.

Conclusion: Administrative healthcare databases in Italy need an extensive process of validation for multiple diagnostic codes to perform high quality epidemiological and health services research.

Author Biographies

Iosief Abraha, Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

Regional Health Authority of Umbria

Massimiliano Orso, Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

Regional Health Authority of Umbria

Piero Grilli, Azienda Sanitaria Locale USL 2, Medicina Generale, Terni, Italy

Azienda Sanitaria Locale USL 2

Francesco Cozzolino, Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

Regional Health Authority of Umbria

Paolo Eusebi, Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

Regional Health Authority of Umbria

Paola Casucci, Regional Health Authority of Umbria, Health ICT Service, Perugia, Italy

Regional Health Authority of Umbria

Mauro Marchesi, Azienda Ospedaliero-Universitaria di Perugia, Servizio Immunoematologia e Trasfusionale, Perugia, Italy

Azienda Ospedaliero-Universitaria di Perugia

Maria Laura Luchetta, Azienda Sanitaria Locale USL 2, Medicina Generale, Terni, Italy

Azienda Sanitaria Locale USL 2

Luisa Fruttini, Istituto Zooprofilattico Sperimentale dell'Umbria e delle Marche, Unità Operativa Editoria Biblioteca, Perugia, Italy

Istituto Zooprofilattico Sperimentale dell'Umbria e delle Marche

Raoul Ciappelloni, Istituto Zooprofilattico Sperimentale dell'Umbria e delle Marche, Unità Operativa Editoria Biblioteca, Perugia, Italy

Istituto Zooprofilattico Sperimentale dell'Umbria e delle Marche

Rita De Florio, Azienda Sanitaria Locale USL 2, Medicina Generale, Terni, Italy

4Azienda Sanitaria Locale USL 2

Gianni Giovannini, Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

Regional Health Authority of Umbria

Alessandro Montedori, Regional Health Authority of Umbria, Health Planning Service, Perugia, Italy

Regional Health Authority of Umbria

References

Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol 2005; 58: 323-37. http://dx.doi.org/10.1016/j.jclinepi.2004.10.012 DOI: https://doi.org/10.1016/j.jclinepi.2004.10.012

Prins H, Hasman A. Appropriateness of ICD-coded diagnostic inpatient hospital discharge data for medical practice assessment. A systematic review. Methods of information in medicine 2013; 52: 3-17. http://dx.doi.org/10.3414/ME12-01-0022 DOI: https://doi.org/10.3414/ME12-01-0022

World Health Organization. International statistical classification of diseases and health related problems, 10th revision. Geneva: WHO 1992.

Iezzoni LI. Assessing quality using administrative data. Annals of internal medicine 1997; 127: 666-74. http://dx.doi.org/10.7326/0003-4819-127-8_Part_2-199710151-00048 DOI: https://doi.org/10.7326/0003-4819-127-8_Part_2-199710151-00048

West SL, Strom BL, Poole C. Validity of Pharmacoepidemiologic Drug and Diagnosis Data: John Wiley & Sons, Ltd; 2007. 709-65 p. DOI: https://doi.org/10.1002/9780470059876.ch45

Lalmohamed A, Vestergaard P, Cooper C, de Boer A, et al. Timing of stroke in patients undergoing total hip replacement and matched controls: a nationwide cohort study. Stroke; a journal of cerebral circulation 2012; 43: 3225-9. http://dx.doi.org/10.1161/STROKEAHA.112.668509 DOI: https://doi.org/10.1161/STROKEAHA.112.668509

Hottes TS, Skowronski DM, Hiebert B, Janjua NZ, et al. Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias. PLoS One 2011; 6: e22618. DOI: https://doi.org/10.1371/journal.pone.0022618

Traversa G, Bianchi C, Da Cas R, Abraha I, Menniti-Ippolito F, Venegoni M. Cohort study of hepatotoxicity associated with nimesulide and other non-steroidal anti-inflammatory drugs. BMJ 2003; 327: 18-22. http://dx.doi.org/10.1136/bmj.327.7405.18 DOI: https://doi.org/10.1136/bmj.327.7405.18

Prieto-Alhambra D, Javaid MK, Judge A, Maskell J, et al. Hormone replacement therapy and mid-term implant survival following knee or hip arthroplasty for osteoarthritis: a population-based cohort study. Annals of the rheumatic diseases 2014. DOI: https://doi.org/10.1136/annrheumdis-2013-204043

Fattore G, Torbica A. Inpatient reimbursement system in Italy: how do tariffs relate to costs? Health care management science 2006; 9: 251-8. http://dx.doi.org/10.1007/s10729-006-9092-2 DOI: https://doi.org/10.1007/s10729-006-9092-2

Abraha I, Montedori A, Stracci F, Rossi M, Romagnoli C. Statin compliance in the Umbrian population. European journal of clinical pharmacology 2003; 59: 659-61. http://dx.doi.org/10.1007/s00228-003-0675-2 DOI: https://doi.org/10.1007/s00228-003-0675-2

Gini R, Francesconi P, Mazzaglia G, Cricelli I, et al. Chronic disease prevalence from Italian administrative databases in the VALORE project: a validation through comparison of population estimates with general practice databases and national survey. BMC Public Health 2013; 13: 15. http://dx.doi.org/10.1186/1471-2458-13-15 DOI: https://doi.org/10.1186/1471-2458-13-15

Baviera M, Santalucia P, Cortesi L, Marzona I, et al. Sex differences in cardiovascular outcomes, pharmacological treatments and indicators of care in patients with newly diagnosed diabetes: Analyses on administrative database. European journal of internal medicine 2014. http://dx.doi.org/10.1016/j.ejim.2014.01.022 DOI: https://doi.org/10.1016/j.ejim.2014.01.022

Bottacchi E, Corso G, Tosi P, Morosini MV, et al. The cost of first-ever stroke in Valle d'Aosta, Italy: linking clinical registries and administrative data. BMC health services research 2012; 12: 372. http://dx.doi.org/10.1186/1472-6963-12-372 DOI: https://doi.org/10.1186/1472-6963-12-372

Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol 2011; 64: 821-9. http://dx.doi.org/10.1016/j.jclinepi.2010.10.006 DOI: https://doi.org/10.1016/j.jclinepi.2010.10.006

Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Annals of internal medicine 2003; 138: 40-4. http://dx.doi.org/10.7326/0003-4819-138-1-200301070-00010 DOI: https://doi.org/10.7326/0003-4819-138-1-200301070-00010

Carrara G, Scire CA, Cimmino MA, Zambon A, et al. Derivation and validation of a diagnostic algorithm to identify patients with rheumatoid arthritis in administrative health database. Annals of the rheumatic diseases 2013; 72. DOI: https://doi.org/10.1136/annrheumdis-2013-eular.1066

Faustini A, Canova C, Cascini S, Baldo V, et al. The reliability of hospital and pharmaceutical data to assess prevalent cases of chronic obstructive pulmonary disease. Copd. 9: 184-96. DOI: https://doi.org/10.3109/15412555.2011.654014

Valkhoff VE, Coloma PM, Lapi F, Gini R, et al. Positive predictive value for upper gastrointestinal bleeding in four health care databases using different coding systems in the EU-ADR project. Pharmacoepidemiology and Drug Safety. 21: 393.

Coloma PM, Valkhoff VE, Mazzaglia G, Nielsson MS, et al. Accuracy of coding-based algorithms in identification of acute myocardial infarction from multi-country electronic healthcare records (EHR) databases. Pharmacoepidemiology and Drug Safety 2012; 21: 395-6.

Moro ML, Morsillo F. Can hospital discharge diagnoses be used for surveillance of surgical-site infections? J Hosp Infect 2004; 56: 239-41. http://dx.doi.org/10.1016/j.jhin.2003.12.022 DOI: https://doi.org/10.1016/j.jhin.2003.12.022

Tinelli M, Mannino S, Lucchi S, Piatti A, et al. Healthcare-acquired infections in rehabilitation units of the Lombardy Region, Italy. Infection 2002; 39: 353-8. http://dx.doi.org/10.1007/s15010-011-0152-2 DOI: https://doi.org/10.1007/s15010-011-0152-2

Calzari MG, Vinceti M, Avanzini P, Rodolfi R, et al. [Sensitivity and accuracy of health databases in determining incidence of lymphoid malignancies in an Italian population]. Annali di Igiene 2006; 18: 127-36.

Palange S, Ascoli V, Carnovale-Scalzo C, Forastiere F, et al. Stime di incidenza del mesotelioma pleurico nel Lazio. Med Lav 2004; 95: 45-54.

Fano V, D'Ovidio M, del Zio K, Renzi D, et al. [The role of the quality of hospital discharge records on the comparative evaluation of outcomes: the example of chronic obstructive pulmonary disease (COPD)]. Epidemiol Prev 2011; 36: 172-9.

Baldi I, Vicari P, Di Cuonzo D, Zanetti R, et al. A high positive predictive value algorithm using hospital administrative data identified incident cancer cases. Journal of Clinical Epidemiology 2008; 61: 373-9. http://dx.doi.org/10.1016/j.jclinepi.2007.05.017 DOI: https://doi.org/10.1016/j.jclinepi.2007.05.017

Bogliun G, Beghi E. Validity of hospital discharge diagnoses for public health surveillance of the Guillain-Barré syndrome. Neurol Sci 2002; 23: 113-7. http://dx.doi.org/10.1007/s100720200036 DOI: https://doi.org/10.1007/s100720200036

Cattaruzzi C, Troncon MG, Agostinis L, Garcia Rodriguez LA. Positive predictive value of ICD-9th codes for upper gastrointestinal bleeding and perforation in the Sistema Informativo Sanitario Regionale database. Journal of Clinical Epidemiology 1999; 52: 499-502. http://dx.doi.org/10.1016/S0895-4356(99)00004-9 DOI: https://doi.org/10.1016/S0895-4356(99)00004-9

Chio A, Ciccone G, Calvo A, Vercellino M, et al. Validity of hospital morbidity records for amyotrophic lateral sclerosis. A population-based study. J Clin Epidemiol 2002; 55: 723-7. http://dx.doi.org/10.1016/S0895-4356(02)00409-2 DOI: https://doi.org/10.1016/S0895-4356(02)00409-2

Pisa FE, Verriello L, Deroma L, Drigo D, et al. The accuracy of discharge diagnosis coding for Amyotrophic Lateral Sclerosis in a large teaching hospital. European journal of epidemiology 2009; 24: 635-40. http://dx.doi.org/10.1007/s10654-009-9376-1 DOI: https://doi.org/10.1007/s10654-009-9376-1

Schifano P, Papini P, Agabiti N, Scarinci M, Borgia P, Perucci CA. Indicators of breast cancer severity and appropriateness of surgery based on hospital administrative data in the Lazio Region, Italy. BMC Public Health 2006; 6: 25. http://dx.doi.org/10.1186/1471-2458-6-25 DOI: https://doi.org/10.1186/1471-2458-6-25

Spolaore P, Brocco S, Fedeli U, Visentin C, et al. Measuring accuracy of discharge diagnoses for a region-wide surveillance of hospitalized strokes. Stroke; a journal of cerebral circulation 2005; 36: 1031-4. http://dx.doi.org/10.1161/01.STR.0000160755.94884.4a DOI: https://doi.org/10.1161/01.STR.0000160755.94884.4a

Beghi E, Logroscino G, Micheli A, Millul A, et al. Validity of hospital discharge diagnoses for the assessment of the prevalence and incidence of amyotrophic lateral sclerosis. Amyotrophic Lateral Sclerosis and Other Motor Neuron Disorders 2001; 2: 99-104. http://dx.doi.org/10.1080/146608201316949541 DOI: https://doi.org/10.1080/146608201316949541

Franchi C, Giussani G, Messina P, Montesano M, et al. Validation of healthcare administrative data for the diagnosis of epilepsy. Journal of epidemiology and community health 2013; 67: 1019-24. http://dx.doi.org/10.1136/jech-2013-202528 DOI: https://doi.org/10.1136/jech-2013-202528

Galdarossa M, Vianello F, Tezza F, Allemand E, et al. Epidemiology of primary and secondary thrombocytopenia: first analysis of an administrative database in a major Italian institution. Blood Coagul Fibrinolysis. 23: 271-7. http://dx.doi.org/10.1097/MBC.0b013e328351882d DOI: https://doi.org/10.1097/MBC.0b013e328351882d

Leone MA, Capponi A, Varrasi C, Tarletti R, Monaco F. Accuracy of the ICD-9 codes for identifying TIA and stroke in an Italian automated database. Neurol Sci 2004; 25: 281-8. http://dx.doi.org/10.1007/s10072-004-0355-8 DOI: https://doi.org/10.1007/s10072-004-0355-8

Rinaldi R, Vignatelli L, Galeotti M, Azzimondi G, De Carolis P. Accuracy of ICD-9 codes in identifying ischemic stroke in the General Hospital of Lugo di Romagna (Italy). Neurol Sci 2003; 24: 65-9. DOI: https://doi.org/10.1007/s100720300074

Castellsague J, Pisa F, Rosolen V, Drigo D, et al. Risk of upper gastrointestinal complications in a cohort of users of nimesulide and other nonsteroidal anti-inflammatory drugs in Friuli Venezia Giulia, Italy. Pharmacoepidemiol Drug Saf 2013; 22: 365-75. http://dx.doi.org/10.1002/pds.3385 DOI: https://doi.org/10.1002/pds.3385

Sturkenboom MC, Romano F, Simon G, Correa-Leite ML, et al. The iatrogenic costs of NSAID therapy: a population study. Arthritis and rheumatism 2002; 47: 132-40. http://dx.doi.org/10.1002/art.10268 DOI: https://doi.org/10.1002/art.10268

Li X, Hilsden R, Hossain S, Fleming J, Winget M. Validation of administrative data sources for endoscopy utilization in colorectal cancer diagnosis. BMC Health Serv Res 2012; 12: 358. http://dx.doi.org/10.1186/1472-6963-12-358 DOI: https://doi.org/10.1186/1472-6963-12-358

Myers R, Leung Y, Shaheen A, Li B. Validation of ICD-9-CM/ICD-10 coding algorithms for the identification of patients with acetaminophen overdose and hepatotoxicity using administrative data. BMC Health Serv Res 2007; 7: 159. http://dx.doi.org/10.1186/1472-6963-7-159 DOI: https://doi.org/10.1186/1472-6963-7-159

Harris S, Glazier R, Tompkins J, Wilton A, et al. Investigating concordance in diabetes diagnosis between primary care charts (electronic medical records) and health administrative data: a retrospective cohort study. BMC Health Serv Res 2010; 10: 347. http://dx.doi.org/10.1186/1472-6963-10-347 DOI: https://doi.org/10.1186/1472-6963-10-347

Beghi E, Millul A, Micheli A, Vitelli E, Logroscino G, Group S. Incidence of ALS in Lombardy, Italy. Neurology 2007; 68: 141-5. http://dx.doi.org/10.1212/01.wnl.0000250339.14392.bb DOI: https://doi.org/10.1212/01.wnl.0000250339.14392.bb

Logroscino G, Traynor BJ, Hardiman O, Chio A, et al. Incidence of amyotrophic lateral sclerosis in Europe. Journal of neurology, neurosurgery, and psychiatry 2010; 81: 385-90. http://dx.doi.org/10.1136/jnnp.2009.183525 DOI: https://doi.org/10.1136/jnnp.2009.183525

Chio A, Bottacchi E, Buffa C, Mutani R, Mora G, Parals. Positive effects of tertiary centres for amyotrophic lateral sclerosis on outcome and use of hospital facilities. Journal of neurology, neurosurgery, and psychiatry 2006; 77: 948-50. http://dx.doi.org/10.1136/jnnp.2005.083402 DOI: https://doi.org/10.1136/jnnp.2005.083402

Sacerdote C, Baldi I, Bertetto O, Dicuonzo D, et al. Hospital factors and patient characteristics in the treatment of colorectal cancer: a population based study. BMC Public Health 2012; 12: 775. http://dx.doi.org/10.1186/1471-2458-12-775 DOI: https://doi.org/10.1186/1471-2458-12-775

Pagano E, Filippini C, Di Cuonzo D, Ruffini E, et al. Factors affecting pattern of care and survival in a population-based cohort of non-small-cell lung cancer incident cases. Cancer epidemiology 2010; 34: 483-9. http://dx.doi.org/10.1016/j.canep.2010.04.002 DOI: https://doi.org/10.1016/j.canep.2010.04.002

Rosato R, Sacerdote C, Pagano E, Di Cuonzo D, et al. Appropriateness of early breast cancer management in relation to patient and hospital characteristics: a population based study in Northern Italy. Breast cancer research and treatment 2009; 117: 349-56. http://dx.doi.org/10.1007/s10549-008-0252-6 DOI: https://doi.org/10.1007/s10549-008-0252-6

Giorgi Rossi P, Chini F, Barca A, Baiocchi D, et al. Efficacy of disease management profiles. The mammographic screening program of Lazio. Tumori 2008; 94: 297-303. http://dx.doi.org/10.1258/096914106778440662 DOI: https://doi.org/10.1177/030089160809400302

Giorgi Rossi P, Federici A, Farchi S, Chini F, et al. The effect of screening programmes on the treatment of benign breast neoplasms: observations from current practice in Italy. Journal of medical screening 2006; 13: 123-8. DOI: https://doi.org/10.1258/096914106778440662

Van Walraven C, Bennett C, Forster AJ. Derivation and validation of a MEDLINE search strategy for research studies that use administrative data. Health services research 2010; 45: 1836-45. http://dx.doi.org/10.1111/j.1475-6773.2010.01159.x DOI: https://doi.org/10.1111/j.1475-6773.2010.01159.x

Tessari R, Migliore E, Balzi D, Barchielli A, et al. [Asthma prevalence estimated using a standard algorithm based on electronic health data in various areas of Italy]. Epidemiol Prev 2008; 32: 56-65.

Migliore E, Bugiani M, Piccioni P, Galassi C, et al. [Obstructive lung disease prevalence estimated using a standard algorithm based on electronic health data in various areas of Italy]. Epidemiol Prev 2008; 32: 66-77.

Trifiro G, Patadia V, Schuemie MJ, Coloma PM, et al. EU-ADR healthcare database network vs. spontaneous reporting system database: preliminary comparison of signal detection. Studies in health technology and informatics 2011; 166: 25-30.

Colais P, Pinnarelli L, Fusco D, Davoli M, Braga M, Perucci CA. The impact of a pay-for-performance system on timing to hip fracture surgery: experience from the Lazio Region (Italy). BMC health services research 2013; 13: 393. http://dx.doi.org/10.1186/1472-6963-13-393 DOI: https://doi.org/10.1186/1472-6963-13-393

OsMed Gdl. L'uso dei farmaci in Italia - Rapporto nazionale anno 2010. Il Pensiero Scientifico Editore 2011.

Published

2014-08-25

How to Cite

Abraha, I., Orso, M., Grilli, P., Cozzolino, F., Eusebi, P., Casucci, P., Marchesi, M., Luchetta, M. L., Fruttini, L., Ciappelloni, R., Florio, R. D., Giovannini, G., & Montedori, A. (2014). The Current State of Validation of Administrative Healthcare Databases in Italy: A Systematic Review. International Journal of Statistics in Medical Research, 3(3), 309–320. https://doi.org/10.6000/1929-6029.2014.03.03.10

Issue

Section

General Articles