SCRIPT X2B8 Dataset: per-day clinical features to model successful next-day extubation

Abstract

Criteria to identify patients who are ready to be liberated from mechanical ventilation are imprecise, often resulting in prolonged mechanical ventilation or reintubation, both of which are associated with adverse outcomes. We sought to determine whether machine learning applied to the electronic health record could predict extubation success. With the X2B8 dataset, we provide cleaned next-day extubation labels for an internal cohort of 696 patients and 9,828 ICU days, and an external test cohort of 333 patients and 2,835 ICU days. With an eye towards future deployment of a model that could be used during daily clinical rounds, we aggregated data from 37 clinical features from midnight to 8 AM. Data have been deidentified per Health Insurance Portability and Accountability Act (HIPAA) Safe Harbor rules. We use this dataset in a manuscript examining different machine learning models to predict successful extubation (see Usage section for details) and share it for others to work with.

Background

Mechanical ventilation is an important intervention in critical care, providing life support for patients with respiratory failure due to various conditions such as pneumonia, asthma, and heart failure. While mechanical ventilation can be lifesaving, there are risks and adverse outcomes associated with prolonged ventilation. Increased time under ventilation makes patients susceptible to complications such as ventilator associated pneumonia (VAP) [1], a type of pneumonia infection shown to have adverse outcomes. Additionally, too much time under ventilation can compromise the native lung function and make it more difficult to wean off, as well as increase healthcare burden.

A goal in critical care is to extubate patients successfully from mechanical ventilation at the earliest time to minimize these risks. Traditional criteria to assess for this would be clinical judgment by the critical care team, assessment from the respiratory therapist, and spontaneous breathing trials to assess the patient's ability to wean off the ventilator. However, this assessment lacks precision. Overly cautious delays can lead to exposure of mechanical ventilation risks such as VAP and compromised native lung function, greater duration of ICU stay, and greater healthcare burden. Failed extubation is when a patient does not successfully remain off the ventilator and can result in respiratory failure. We define failed extubation as reintubation within 48 hours of the extubation event. This is associated with increased morbidity, mortality, and longer hospital stays [2].

The intent of the X2B8 dataset is to enable data driven analysis and predictive modeling of intubation/extubation in the medical ICU. For example, this EHR data can be used for extubation prediction, modeling clinical state during an ICU stay, and response to different treatments such as hemodialysis and continuous renal replacement therapy (CRRT). We encourage users to use EHR data to identify complex patterns and predictors of ICU task outcomes that may not be apparent through traditional analysis or overlooked in fast-paced clinical assessment.

The motivations for sharing data include addressing critical clinical challenges, as data sharing helps in understanding and solving complex problems in clinical settings. Additionally, it supports the introduction of machine learning models in ICU scenarios, enabling the development and validation of tools that can improve patient outcomes. Another key motivation is ensuring the transparency and reproducibility of our manuscript, allowing others to verify findings and build upon the research. Lastly, data sharing promotes evidence-based practice, providing robust support for clinical decision-making and advancing the overall quality of care.

Methods

Patients were enrolled in the Successful Clinical Response In Pneumonia Therapy (SCRIPT) Systems Biology Center [3], a single-site prospective cohort study of patients requiring mechanical ventilation, who underwent bronchoalveolar lavage for known or suspected pneumonia (NU IRB # STU00204868) in a quaternary care hospital from 2018 to 2023. Patients or their legal authorized representative consented to participate in this study. External testing was done using EHR data collected from patients in a mixed medical surgical ICU from a community hospital, Central DuPage Hospital (CDH) from 2018 to 2022 (NU IRB # STU00216678); this is a retrospective data-only protocol and received a waiver of informed consent.

EHR data were extracted from the Northwestern University Enterprise Data Warehouse [4], and were manually reviewed and validated by ICU physicians who focused specifically on ventilation features and markers of intubation and extubation. Features included in the dataset span vital signs, ventilator parameters, laboratory values, mental status assessments, and organ failure assessments (see Data Dictionary). Extensive data cleaning and filtering was done to label next-day extubation status, with over two hundred patient charts manually reviewed by physicians. Details of cleaning are available in our manuscript [5] and code repository [6]. The data were de-identified according to the HIPAA Safe Harbor rules [7]. All dates have been removed and are presented relative to a patient's ICU stay. For example ICU stay 1, day 1 represents the first day of the patient's first ICU stay.

This dataset has the following unique features:

1. Focus on extubation prediction. Data selected and cleaned with input from physicians to focus on 37 features representative of variables routinely examined during daily ICU rounds.
2. Carefully curated next day extubation/intubation labels, and tracheostomy labels.
3. Internal training and validation cohort, and external test cohort, showing generalizability
4. Aggregation from 12AM-8AM, rather than the entire 24hr day as done in the original CarpeDiem dataset, with an eye towards potential future deployment during daily ICU rounds.

Inclusion: Patients in the medical ICU, requiring mechanical ventilation, with suspected pneumonia; patients with successful extubation are included in the training dataset

Exclusion from main training set, given lack of successful extubation, but whose data are provided in subset files:

• Patients on tracheostomy (in trach.csv)
• Patients with failed extubation outcomes (in fail.csv)

Variables (full list available in Data Dictionary):

• Demographics
• Vital signs
• Ventilator parameters
• Laboratory values
• Mental status assessments
• Organ failure assessments

This dataset is designed for predictive models to be trained on features from patient days to predict patient-level outcomes. We trained both sequential models (ex. RNN, LSTM) to learn trajectories within ICU stays, and boosting models to treat patient days independent of ICU stay. Please see our manuscript [5] for more details. Following our patient day organization scheme, models could be trained to predict any outcome, metric, or clinical intervention in the dataset. Additionally, unsupervised methods could be employed to identify patterns in the data that may be associated with features of interest.

Data Description

There were 712 enrollments in SCRIPT during the study period, with 940 separate ICU stays totaling 16,402 ICU days. After filtering days and stays for our next-day successful extubation task, we trained and evaluated using 696 unique patients, 781 ICU stays, and 9,828 ICU days. Our CDH dataset consisted of 459 unique patients, 518 ICU stays, and 5,814 ICU days. This was filtered down to 333 unique patients, 349 ICU stays, and 2,835 ICU days. The failed extubation rate, defined as requiring reintubation within two days, was 23.1% and 15.8% in the SCRIPT and CDH cohorts, respectively. In the SCRIPT cohort, the median [Q1,Q3] patient age was 63 [51, 72], and 44% of the patients were female. The percent of patients with unfavorable outcomes (death, discharge to hospice, or lung transplantation) for the SCRIPT cohort was 45%. Please see Tables 1-2 and Supplemental Table 2 from our manuscript [5] for detailed cohort descriptions.

The data is organized into two folders for the internal and external cohorts. The README describes a breakdown of the files within each folder. Each table is of the same structure, just containing different patients according to how they were either filtered or allocated to a particular train/val/test split. Each table is indexed by a patient day. The table is additionally grouped by ICU stay such that a given ICU stay will appear in sequential order in the table. Features are represented as a single value for each patient day, where continuous variables are represented as the average of all measurements taken between midnight and 8AM that day. Please see the data dictionary for more details.

Dataset Structure

The dataset is organized into CSV files representing patient-ICU-day observations, with each row containing values for clinical features and extubation outcomes. Multiple measurements of a given variable are aggregated from 12AM-8AM by mean. A data dictionary (x2b8_data_dictionary.csv) explains the columns in more detail.

Internal SCRIPT Cohort

• train.csv: Data for training machine learning models.
• val.csv: Data for model tuning and hyperparameter optimization.
• test.csv: Data for internal evaluation.
• trach.csv: Data from patients filtered due to tracheostomy.
• fail.csv: Data from patients filtered due to failed extubation.

External Test Cohort

• full.csv: Data for external evaluation.
• trach.csv: Data from patients filtered due to tracheostomy.
• fail.csv: Data from patients filtered due to failed extubation.

Data Dictionary

The x2b8_data_dictionary.csv file provides descriptions of all 37 clinical features, including vital signs, ventilator parameters, laboratory values, and support device usage.

Usage Notes

We use this dataset in a manuscript examining different machine learning models to predict successful extubation [5] and share it for others to work with. The code repository used in our paper is available at GitHub repository [6]. Please feel free to open issues on that platform with questions.

Limitations include the missing data inherent to EHR data, though we believe this is informative in and of itself, as physicians will not check labs when they are not felt to be helpful or clinically important. We chose 37 parameters of clinical interest as identified by our critical care physicians, but there are many other parameters available or measured in the ICU that are not included in our dataset.

The dataset was compiled for the purposes of predicting successful next-day extubation, but different predictions related to the decision to extubate could also be modeled with this dataset. For example, the data is well-suited to identify indicators of failed extubation. Usage of other support devices such as ECMO, tracheostomy, CRRT, and hemodialysis are labeled and could also be a predictive target. In addition, outcomes such as mortality, prolonged intubation, and successful discharge from the ICU may be of interest. Finally, unsupervised exploratory analysis of the dataset would be of value to find patterns associated with any of the above outcomes and statuses.

Release Notes

Version 1.0.0 initial release.

Ethics

This study was approved by the Northwestern University Institutional Review Board with study IDs STU00204868 and STU00216678.

Acknowledgements

The authors would like to thank Northwestern Memorial Hospital and the Northwestern University Feinberg School of Medicine for their support, as well as all the patients, providers, and SCRIPT team members. SCRIPT is funded by NIH NIAID U19AI135964. Work in the Division of Pulmonary and Critical Care is also supported by the Simpson Querrey Lung Institute for Translational Science (SQLIFTS).

Conflicts of Interest

The authors declare no conflicts of interest.

References

1. Melsen WG, Rovers MM, Groenwold RHH, Bergmans DC, Camus C, Bauer TT, et al. Attributable mortality of ventilator-associated pneumonia: a meta-analysis of individual patient data from randomised prevention studies. Lancet Infect Dis. 2013 Aug;13(8):665–71.
2. Epstein, S. K. & Ciubotaru, R. L. Independent effects of etiology of failure and time to reintubation on outcome for patients failing extubation. Am. J. Respir. Crit. Care Med. 158, 489–493 (1998).
3. NU SCRIPT. “Successful Clinical Response In Pneumonia Therapy (SCRIPT) Systems Biology Center.” https://script.northwestern.edu/
4. Starren JB, Winter AQ, Lloyd-Jones DM. Enabling a learning health system through a unified enterprise data warehouse: The experience of the northwestern university clinical and translational sciences (NUCATS) institute. Clin Transl Sci. 2015 Aug;8(4):269–71.
5. Fenske SW, Peltekian A, Kang M, Markov NS, Zhu M, Grudzinski K, et al. Developing and validating a machine learning model to predict successful next-day extubation in the ICU [Internet]. medRxiv. 2024. p. 2024.06.28.24309547. Available from: https://www.medrxiv.org/content/10.1101/2024.06.28.24309547v1.full-text
6. 2024_Fenske_Peltekian Code Repository [Internet]. [cited 2024 Aug 5]. Available from: https://github.com/NUPulmonary/2024_Fenske_Peltekian
7. Office for Civil Rights (OCR). Guidance regarding methods for DE-identification of protected health information in accordance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule. HHS.gov. 2012; published online Sept 7. https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html (accessed Sept 9, 2022).