MedDec: Medical Decisions for Discharge Summaries in the MIMIC-III Database

Abstract

Medical decisions directly impact individuals’ health and well-being. Extracting decision spans from clinical notes plays a crucial role in understanding medical decision-making processes. The MedDec dataset includes expert-annotated decisions of ten types, including defining problems, setting treatment goals, making drug-related decisions, performing therapeutic procedures, giving advice and precautions, and deferment decisions, spanning across eleven diverse patient phenotypes such as heart disease, lung disease, cancer, psychiatric disorders, and chronic pain, from 451 expert-annotated discharge summaries in the MIMIC-III database, where annotators identified decision spans within each discharge summary, noting character start and end positions alongside the decision type. The dataset enables the task of medical decision extraction, aiming to jointly extract and classify different types of medical decisions within clinical notes. This dataset also provides insights into medical decisions across diverse demographics, including gender, race, and English proficiency, sourced from the MIMIC-III database

Background

Clinical notes are a valuable source of information detailing medical decision-making processes, including patient conditions, medications, laboratory results, and follow-up information. However, extracting knowledge from these notes automatically poses significant challenges due to their imprecise and heterogeneous nature, and the necessity for data annotation. While comprehensive taxonomies for medical decisions exist [1-2], there has been no dataset specifically designed for extracting and classifying medical decisions in clinical narratives. MedDec addresses this gap by offering an expert-annotated dataset for medical decision extraction within discharge summaries. This dataset aims to facilitate the development of evidence-based decision-making guidelines, identify deviations from best practices, and inform health policy development.

Methods

MedDec was created using patient data sourced from the Medical Information Mart for Intensive Care (MIMIC-III), a publicly available dataset of de-identified clinical data from ICU patients [3]. The dataset includes annotations of ten types of medical decisions across eleven different phenotypes from 451 discharge summaries in the MIMIC-III database according to the Decision Identification and Classification Taxonomy for Use in Medicine (DICTUM) [4].

We sampled 192 discharge summaries from the dataset developed by Gehrmann et al. (2018) [5], representing a cross-section of clinical conditions, including phenotypes related to heart disease, lung disease, cancer, psychiatric disorders, and chronic pain. An additional 259 discharge summaries were included to ensure coverage of diverse phenotypes. The final dataset represents 42.6% female patients, 75.9% white patients, and 85.2% English-proficient patients."

All discharge summaries were manually annotated by two trained medical students, and disagreements were adjudicated by an MD. The annotation achieved a substantial inter-annotator agreement (Cohen’s Kappa = 0.74). No NLP or automatic methods were used in the annotation process.

Each discharge summary is identified using three unique identifiers: SUBJECT_ID (representing a unique patient), HADM_ID (representing a unique hospital admission), and ROW_ID (a row identifier unique to the table in MIMIC-III). These IDs are derived from the MIMIC-III dataset, and they enable joining MedDec with other tables in MIMIC-III.

Data Description

MedDec consists of annotated decisions in 451 discharge summaries, covering 419 unique patients, and over 54,000 sentences. The dataset includes diverse patient groups based on sex, race, and English proficiency. Medical decisions are categorized according to the DICTUM taxonomy, which includes ten decision categories. The dataset features 1.4 million tokens, with 879,000 forming part of a decision span and 37,000 overlapping between spans. This detailed annotation allows for comprehensive analysis and the development of BioNLP techniques focused on medical decision-making.

Of the 451 discharge summaries, 192 overlap with the phenotype annotations from the Phenotype Annotations for MIMIC-III database [6]. This overlap is based on the shared [SUBJECT_ID, HADM_ID, ROW_ID] identifiers from the MIMIC-III dataset. These identifiers can be used to cross-reference patient data across the MIMIC-III tables and other relevant datasets, allowing for analysis of the decision-making process within specific phenotypes.

The dataset includes medical decisions across 11 phenotypes: Substance Abuse, Advanced Lung Disease,  Alcohol Abuse, Psychiatric Disorders, Obesity, Advanced Heart Disease, Advanced Cancer, Chronic Neurological Dystrophies, Depression, Chronic Pain Fibromyalgia, and "None".

The discharge summary text is not included with this release, and should be obtained from the NOTEEVENTS table from the MIMIC-III database [3]. Patient language proficiency and ethnicity were sourced from the ADMISSIONS table in the MIMIC-III database, while gender was extracted from the PATIENTS table.

Category List and Descriptions

Table 1, adapted from DICTUM [4], provides descriptions and examples of different types of medical decisions in clinical notes.

Dataset File Format and Structure

The dataset is provided in JSON format. The dataset's IDs — SUBJECT_ID, HADM_ID, and ROW_ID — correspond to MIMIC-III [3] identifiers. SUBJECT_ID refers to a unique patient, HADM_ID to a specific admission, and ROW_ID to a row in the MIMIC-III NOTEEVENTS table.

The JSON files provided in this dataset contain only the medical decision annotations. Discharge summary text, phenotype annotations, and demographic information, are not included in the JSON files but can be obtained from the corresponding tables in the MIMIC-III database and the Phenotype Annotations for MIMIC-III dataset as needed. Users can cross-reference these datasets using the shared [SUBJECT_ID, HADM_ID, ROW_ID] identifiers.

Each JSON file contains the following fields:

• annotator_id: ID of annotator.
• discharge_summary_id: Unique identifier for each discharge summary.
  • Format: [SUBJECT_ID]_[HADM_ID]_[ROW_ID].
• annotations: List of annotated spans, each containing:
  • start_offset: The starting character index of the span.
  • end_offset: The ending character index of the span.
  • category: The category of the medical decision.
  • decision: The text of the annotated span.
  • annotation_id: Unique identifier for each annotation.

Data Splits

The dataset is divided into training, validation, and test sets as follows:

• Train: 350 notes
• Validation: 53 notes
• Test: 48 notes

Important Note About Test Set

The test set, comprising 48 notes, is not included with the initial release of the dataset. It will be made publicly available after the completion of the shared task. Details about the shared task, including guidelines for participation and evaluation metrics, will be announced soon. The test set follows the same structure and format as the training and validation sets, with each note linked to [SUBJECT_ID, HADM_ID, ROW_ID] identifiers to maintain consistency with MIMIC-III.

Usage Notes

MedDec has been used to 1) develop models for medical decision extraction and classification using NLP span-detection techniques, 2) analyze statistics on medical decisions across protected variables and patient phenotypes, and 3) create metrics for evaluating the complexity of different types of medical decisions [7].

Additionally, MedDec has been been employed to identify disparities in medical decision-making across different patient demographics, including sex, race, and language proficiency [8].

Ethics

This project adheres to ethical considerations and safeguards to ensure the responsible and ethical handling of medical data and its implications. We have made every effort to protect human subject information and minimize the potential risk of loss of patient privacy and confidentiality (all authors with access to the data have successfully completed a training program in the protection of human subjects and privacy protection). In addition, our work is transformational in nature, and its broader impacts are first and foremost the potential to improve the well-being of individual patients in society and support clinicians in their medical decision-making efforts.

Conflicts of Interest

The authors declare no conflicts of interest.

References

1. Braddock, C. H., 3rd, Fihn, S. D., Levinson, W., Jonsen, A. R., & Pearlman, R. A. (1997). How doctors and patients discuss routine clinical decisions. Informed decision making in the outpatient setting. Journal of general internal medicine, 12(6), 339–345. https://doi.org/10.1046/j.1525-1497.1997.00057.x
2. Ofstad, E. H., Frich, J. C., Schei, E., Frankel, R. M., Benth, J. Š., & Gulbrandsen, P. (2018). Clinical decisions presented to patients in hospital encounters: a cross-sectional study using a novel taxonomy. BMJ open, 8(1), e018042.
3. Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Celi, L. A., & Mark, R. G. (2016). MIMIC-III, a freely accessible critical care database. Scientific Data, 3, 160035
4. Ofstad, E. H., Frich, J. C., Schei, E., Frankel, R. M., & Gulbrandsen, P. (2016). What is a medical decision? A taxonomy based on physician statements in hospital encounters: a qualitative study. BMJ open, 6(2), e010098. https://doi.org/10.1136/bmjopen-2015-010098
5. Gehrmann S, Dernoncourt F, Li Y, Carlson ET, Wu JT, Welt J, et al. (2018) Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PLoS ONE 13(2): e0192360. https://doi.org/10.1371/journal.pone.0192360
6. Moseley, E., Celi, L. A., Wu, J., & Dernoncourt, F. (2020). Phenotype Annotations for Patient Notes in the MIMIC-III Database (version 1.20.03). PhysioNet. https://doi.org/10.13026/txmt-8m40.
7. Elgaar M., Cheng J., Vakil N., Amiri H., Celi LA. (2024). MedDec: A Dataset for Extracting Medical Decisions from Discharge Summaries. In Findings of the Association for Computational Linguistics: ACL 2024.
8. Amiri, H., Vakil, N., Elgaar, M., Cheng, J., Mohtarami, M., Wong, A., ... & Celi, L. A. G. (2024). Analysis of Race, Sex, and Language Proficiency Disparities in Documented Medical Decisions. medRxiv, 2024-07.