MIMIC-Eye: Integrating MIMIC Datasets with REFLACX and Eye Gaze for Multimodal Deep Learning Applications

Abstract

Deep learning technologies have been widely adopted in medical imaging due to their ability to extract features from images and make accurate diagnoses automatically. Medical imaging technologies are particularly useful because they can be trained to detect subtle differences in images that are hard to detect for human radiologists. In the real world, radiologists must rely on various types of patient information to assess medical images confidently. However, most DL applications in medical imaging only utilize image data, mainly because the literature on medical datasets combining different data modalities is scarce. In this study, we present MIMIC-EYE, a dataset that encompasses a comprehensive integration of several datasets related to MIMIC. This dataset includes a comprehensive range of patient information, including medical images and reports (MIMIC CXR and MIMIC JPG), clinical data (MIMIC IV ED), a detailed account of the patient's hospital journey (MIMIC IV), and eye tracking data containing gaze information and pupil dilations together with image annotations (REFLACX and EYE GAZE). Integrating eye tracking data with the various MIMIC modalities may provide a more comprehensive understanding of radiologists' visual search behavior patterns and facilitate the development of more robust, accurate, and reproducible deep-learning models for medical imaging diagnosis.

Background

Medical image diagnosis is the task of identifying lesions and diseases based on medical images, such as chest X-ray images, CT scans or MRI scans. The skills and knowledge of performing this task are owned by specialists, such as doctors and radiologists. However, statistics showed a 6.4 million, 30.6 million, and 2.9 million worldwide shortage of physicians, nurses, and pharmacists respectively in 2019 [1]. Due to the disproportionate impact of the pandemic on healthcare workers, the situation has become more dire. One potential solution to alleviate this shortage is to adopt AI-driven diagnostic systems to facilitate the process.

In AI-driven systems, deep learning (DL) is a popular technique for delivering promising results. Deep learning is based on neural networks, which is a computing system that imitates the biological nerve systems in animal brains. In medical image diagnosis, DL approaches have proven effective and efficient. Several studies have achieved or even surpassed human-level performance by applying DL to medical image tasks, such as breast imaging [2, 3], left ventricular assessment [4, 5], dermoscopy analysis [6, 7],and chest X-Rays [8 – 11]. DL learns knowledge from the dataset it is trained on. This means its performance is highly commensurate with the quality and quantity of the dataset.

Except for providing more robust data to DL models, exposing other aspects of instances to the model can also be beneficial. Multi-modal learning, multi-task learning, and contrastive learning are three popular deep learning technologies. A variety of modalities are employed to better describe or extend the scenarios to models, which enhances their performance and generalisation. Multimodal learning involves more than one input modality, which allows the model to perceive various aspects of the input phenomenon and comprehend the scenario better. The term "modality" is then used to describe the collected information from each sensor. When several different sensors are set up to observe a phenomenon, the information collected is known as multi-modal data [12, 13]. In multi-task learning, the model is trained through multiple tasks, which require various types of labels. When a model is learned from more than one task, it will accommodate each task and generalise better. Contrastive learning is a self-supervised technique, which trains the model to contrast its inputs against each other by mapping different modalities to the same semantic vector space. All these techniques take advantage of the additional information given by the variety of modalities.  

The Medical Information Mart for Intensive Care (MIMIC) IV dataset [14] is the dataset used in this work, which is sourced from two in-hospital database systems, a custom hospital wide EHR and an ICU specific clinical information system, at Beth Israel Deaconess Medical Center (BIDMC) from 2011 to 2019. Since the popularity of the MIMIC-IV dataset, several of its subsets have been built to provide additional information and modalities. Two categories of additional data are crucial to include in research. The first type is the patient's clinical data. Clinical data is highly informative and essential for radiologists to diagnose precisely [15]. The second type of data is human-centric data, which is gathered when radiologists are making diagnoses. This includes eye-tracking data, audio recordings, and time-stamped transcriptions. Since medical diagnosis is a skill owned by experts, it is beneficial to study and analyse their diagnosis patterns. Human-centric data allows us to explore the decision-making process that radiologists undertake while interpreting medical images.

In this work, we present MIMIC-Eye, which integrates valuable modalities from MIMIC-IV [14] and its subsets. Several studies have attempted to involve more than one modality in their training process since multi-modal, multi-task, and contrastive learning became popular and effective. However, the approach to construct a multi-modal dataset for MIMIC-IV has not been standardized. Studies applied different preprocessing and integration strategies, which hinders comparisons between them. To enhance the reproducibility and convenience of medical image diagnosis, it is indispensable to have a dataset with all available modalities, which results in the MIMIC-Eye dataset. 

Five datasets are used to construct the MIMIC-Eye dataset, including MIMIC-IV [14], MIMIC-IV-ED [16], MIMIC-CXR [17], Eye Gaze [18], and REFLACX [19] datasets. Each of them contains useful information about patients or radiologists. MIMIC-IV and MIMIC-IV-ED provide clinical data about patients. MIMIC-CXR provides Chest X-Ray (CXR) images and radiology reports. The radiology reports can be used with Nature Language Processing (NLP) labelers to generate labels for CXR images. The REFLACX and Eye Gaze datasets collected eye-tracking data and audio when radiologists were reading images. The REFLACX dataset also asked radiologists to manually annotate lesions using bounding ellipses. In this work, we integrate the mentioned modalities from different datasets to create the MIMIC-Eye dataset. The key contributions are:

• MIMIC-Eye allows researchers to explore and study the relationships between modes.
• It has integrated clinical data, which simulates the same situation as radiologists have in practice. 
• MIMIC-Eye contains human-centric data, which allows the exploration of diagnosis patterns by radiologists.
• We fixed some data quality issues in the REFLACX and EyeGaze datasets.
• MIMIC-Eye is a patient-level dataset that each patient has their own folder to store information and modalties related to them. This is more intuitive and memory-efficient for debugging and researching.
• We provide a single-source and ready-to-train dataset to ensure reproducibility and facilitate the training of AI models.

Methods

In this section, we first present you a brief introduction to the valuable modalities in the MIMIC-IV dataset. Then we propose a strategy for preprocessing and integrating various modalities in the MIMIC-IV, MIMIC-IV-ED, MIMIC-CXR, REFLACX, and Eye Gaze datasets. The challenges and the issues we encountered are also mentioned in this section. 

Clinical and Human-Centred Data

The MIMIC-Eye dataset was initially motivated by two types of data. The first type is the patient's clinical information relevant to the chest X-ray image. By including clinical data in the dataset, this allows the model to take into account clinical information related to patients, such as body temperature, heart rate, and blood pressure. This is the same approach radiologists apply in their daily work. To make an accurate diagnosis, they need clinical data to provide comprehensive information. For example, while Ateletasis looks identical to Consolidation on the radiograph, Consolidation usually comes with infection. Body temperature plays a crucial role in classifying them in this case. 

Totally, ten clinical features are used in this work. The MIMIC-IV Core patients table includes only two clinical attributes, age and gender. And, the other eight clinical features are extracted from the MIMI-IV ED traige table. The explanations for these eight clinical features in the MIMIC-IV documentation are:

1. temperature: The patient’s temperature in degrees Fahrenheit.
2. heartrate: The patient’s heart rate in beats per minute.
3. resprate: The patient’s respiratory rate in breaths per minute.
4. o2sat: The patient’s peripheral oxygen saturation as a percentage.
5. sbp, dbp: The patient’s systolic and diastolic blood pressure, respectively, measured in millimetres of mercury (mmHg).
6. pain: The level of pain self-reported by the patient, on a scale of 0-10.
7. acuity: An order of priority. Level 1 is the highest priority, while level 5 is the lowest priority.

The second type of data is human-centred data, which are collected while radiologists are reading chest X-ray images and making diagnoses. There are primarily two modalities included in the REFLACX and EyeGaze datasets, including time-stamped transcripts and eye-tracking data. Additionally, the bounding boxes annotated by radiologists that indicate lesions can be accessed from the REFALCX dataset. As the MIMIC-IV dataset only includes global labels extracted from corresponding radiology reports, it can only train the model to make diagnoses from a global perspective. With the ground-truth bounding boxes from REFLACX, the model can be used to perform object/lesion detection to locate pathology.

IDs used in the MIMIC-IV

Before explaining the creation process, it is necessary to introduce some important IDs and tables in the MIMIC-IV dataset. Four important IDs are used in MIMIC-IV to link the information across tables. They are:

• subject_id (patient_id): ID specifying an individual patient.
• stay_id: ID specifying a single emergency department stay for a patient.
• study_id: ID specifying a radiology report written for the given chest x-ray. It is rarely mentioned because we do not use the report as the groundtruth label in this paper.
• dicom_id: ID specifying a chest x-ray image (radiograph).

MIMIC-Eye Construction

In this part, we describe the methods we used to build the MIMIC-Eye dataset. The overall process is divided into the following three phases:

1. stay_id identification: This process is inspired by the EyeGaze dataset. In their work, they showed us the approach to identifying stay_id for CXR images. After this, we also updated the stay_id in EyeGaze since they used an outdated version of the dataset, which is incompatible with the stay_id in v2.0.
2. Pre-processing: In this phase, we pre-processed datasets to make them ready for merging with others. The issues we encountered will also be mentioned and fixed here. To ensure that the dataset is scalable and flexible, a minimum amount of preprocessing is done for MIMIC-Eye, which allows users to choose the preprocessing they need for their task. And MIMIC-Eye only fixes bugs or errors that can be solved with a particular approach. 
3. Integration: At the end, we combine all the datasets and modalities into a single source with a specific folder structure.

Phase 1 - stay_id identification

As we mentioned in previously the stay_id is used to identify stays in the emergency department. However, the MIMIC-CXR dataset only offers images' subject_id but not their stay_id, which causes a dilemma in retrieving the corresponding clinical data from MIMIC-IV-ED. To solve this issue, two tables, MIMIC-IV-ED edstays and MIMIC-IV-CXR-JPG metadata, are used to identify stay_id for CXR images through the following steps:

1. For each CXR image, we first search for stays that belong to the patient. 
2. For each stay, we simply search their time-spans in the MIMIC-IV-ED edstays table, which can tell us when is the start (intime) and end (outtime) time of a stay.
3. Once we obtain the time-span of the stay, we then check what time the radiograph was captured. In the MIMIC-CXR-JPG metadata table, the StudyDate and StudyTime columns can tell us the exact time that the radiograph was taken.
4. Having the duration of the stay and the time the radiograph was taken, we can determine whether this radiograph was taken during the stay. If the time point of the radiograph falls within the time-span, then its $stay\_id$ is found. If not, we continue looking for the patient's stays with the time-span containing the time point.

Mathematically, Let $T^{CXR}_{p,i}$ be to be the time that a CXR image taken for patient $p$ at unknown stay $i$. The $i$ is determined by:

where $T^{\text{start}}_{i,p}$ and $T^{\text{end}}_{j,p}$ are starting time and ending time of a stay respectively.

At this step, 55.99% of CXRs could not identify their stay_id, since the CXRs may have been taken after they left ED. To explore this issue, we ran different conditions to test how many instances would be lost in each condition. We found that when we increased the condition to 7 days, we could identify 23.26% more CXRs. The goal of this experiment is to use clinical information that is close to the time of the CXR. Therefore, only the CXRs taken within the range are used.

Phase 2 - Pre-processing

In this section, we describe the preprocessing we have done for each module. For most of the data quality issues in the parent datasets, we left them to users to decide what techniques to apply. We only provide minimum preprocessing to a few issues that significantly affect the dataset. Below, we list the modules that we have preprocessed:

• REFLACX: The REFLACX dataset provides two versions of labels for radiologists to use when annotating the same lesion. The REFLACX dataset contains 3052 cases in total. 2757 of them are annotated with v1 labels, and the other 295 cases use v2 labels. To make the dataset concise, we interviewed radiologists and used their opinion to merge the labels from two versions, which resolves conflicts. At the end, we have 21 types of classes (lesions) left. 
• EyeGaze: Because we consider raw gaze information to be equally valuable to investigate as fixations, we have also included it in the MIMIC-Eye dataset. However, some values of DICOM_ID in eye_gaze.csv had been trimmed with the last 5 digits, which makes them unrelatable to their chest X-ray images. In some cases, both incorrect and valid IDs exist in the table. And the erroneous one is usually shorter than the correct one. In order to fix the data quality issue, we then delete the faulty ID if both the valid and the faulty ID are present. When only the faulty ID appears in the table, it is replaced with the valid one.

Phase 3 - Integration

MIMIC-Eye is designed so that each patient has their own folder in which they can store their own information. These folders are named as subject_id. Using a patient-level folder structure instead of a stay-level folder structure is found to be more intuitive and more convenient for human users. Information related to the patient can now be easily accessed in the same folder, which considerably simplifies the debugging process. For radiologists, it is easier for them to check the patient's clinical data or history. Additionally, we have another folder, called spreadsheets, that contains information that is not related to a specific patient. For example, some tables store the definition of codes used in other tables.

Totally, seven MIMIC-IV modules are used to create the MIMIC-Eye dataset, and the integration strategy we used is available on the GitHub repository [20]. They are assigned to respective folders according to their purposes mentioned previously. Upon integration, MIMIC-Eye contains 3,689 chest X-ray images linked to different modalities. The original MIMIC-IV dataset includes 315,460 patients. For MIMIC-Eye, we only retrieve patients that are either from the EyeGaze or REFLACX datasets since they provide human-centered data. MIMIC-Eye totally contains 3,192 patients. 2,199 and 1,038 of them are in the RELFACX and EyeGaze datasets respectively. And only 10 patients exist in both datasets. In terms of ED stays, 447,712 stays are recorded in the MIMIC-IV-ED dataset. However, only 1,644 stays_ids are identified for chest X-ray images in the MIMIC-Eye dataset. For the chest X-ray images, the MIMIC-IV-CXR dataset consists of 377,110 images of chest X-ray, while 96,161 of them are Posterior-Anterior (PA) view images. In MIMIC-Eye, 3,689 images are used, and 1,683 of them can be linked to patients' clinical data.

Data Description

The MIMIC-Eye dataset contains a total of 3,192 patients, 1,644 stays, and 3,689 chest X-ray images. To organise these data, folders in the root folder are arranged according to the patient's ID, as shown above. As mentioned in the previous section, we designed two types of folders in MIMIC-Eye:

• patient_{subject_id}: This type of folder contains information related to a specific patient. There are up to 7 subfolders in a patient folder. Each sub-folder stores the data extracted from the respective MIMIC-IV module. 
• spreadsheets: While the patient folders hold information about patients. Other information is stored in this spreadsheet folder, such as the definition of codes. Other than that, this folder contains spreadsheets extracted directly from the module. Instead of dividing patients into their own folders, these spreadsheets contain information about all patients. Processing this kind of spreadsheet may occupy a huge amount of computer memory, but it can also minimise retrieval time in certain scenarios. 

Patient

In each patient folder, several folders can be seen. Each of them contains tables extracted from a module or dataset. If the patients miss some folders, then they don't have related information in that module. The following describes the information contained in these folders:

• Core: This folder is derived from the MIMIC-IV [14] Hosp module, which contains demographics for patients, tracking information for ward stays, and measurements during the hospital stay.

• ICU: This folder is retreived from the MIMIC-IV [14] ICU module including ICU level data contains information related to items and events occurring during the ICU stay, including patients' input and output events.

• ED: This folder is obtained from MIMIC-IV-ED [16], which contains the patients' data collected while they are in the emergency department. The triage assessment in this folder is a vital piece of clinical evidence for human radiologists to use when diagnosing patients.

• MIMIC-CXR: This folder contains radiology reports retrieved from the MIMIC-CXR [17] dataset. This dataset contains chest X-ray images in DICOM format, which is the standard format used in hospitals. Nevertheless, this format is not feasible for machine learning models. We then retrieve the images from the MIMIC-CXR-JPG dataset instead, which has CXRs in JPG format.

• MIMIC-CXR-JPG: The folder includes chest X-ray images in JPG format, which are retrieved from the MIMIC-CXR-JPG [23] dataset. This format is ready for machine models to process. Their DICOM metadata is also converted into CSV files and stored here.

• REFLACX: The folder contains information collected from five radiologists while they were reading chest X-ray images. REFLACX [19] asked radiologists to annotate the lesions in CXR images using ellipses. During the process of interpretation, radiologists' eye movements and utterances are also collected. The utterances are then transformed into time-stamped transcriptions. Lastly, this folder also contains chest bounding boxes indicating the lung and heart areas.

• EyeGaze: The EyeGaze folder contains data extracted from the EyeGaze dataset [18]. As with the REFLACX dataset, it also records audio and eye movements during interpretation. The Eye Gaze dataset only contains information generated by a radiologist, while REFLACX has five radiologists involved. Moreover, this folder provides segmentation and bounding boxes for anatomical structures as supplemental sources for further analysis of the correlation with anatomical structures.

Spreadsheets

We also created a spreadsheet folder to store information or tables that are not related to a specific patient. The folder is located at the same level of the directory as the folders for other patients. The file, cxr_meta.csv, is extended from the original metadata.csv in MIMIC-CXR-JPG. In this file, three columns are added:

• stay_id: With this stay_id associated with chest X-ray images, the image can then be linked to corresponding clinical data in the MIMIC-IV-ED dataset.
• in_reflacx: A boolean value indicating whether this image is used in the REFLACX dataset.
• in_eye_gaze: A boolean value indicating whether this image is used in the Eye Gaze dataset

The following table shows the structure of MIMIC-Eye:

patient_{patient_id}/
     Hosp/
          admissions.csv
          diagnoses_icd.csv
          drgcodes.csv
          labevents.csv
          microbiologyevents.csv
          omr.csv
          pharmacy.csv
          poe.csv
          poe_detail.csv
          prescriptions.csv
          procedures_icd.csv
          services.csv
          transfers.csv
     ICU/
          chartevents.csv         
          datetimeevents.csv      
          icustays.csv            
          ingredientevents.csv    
          inputevents.csv         
          outputevents.csv        
          procedureevents.csv
     ED/
          diagnosis.csv   
          edstays.csv     
          medrecon.csv    
          pyxis.csv       
          triage.csv      
          vitalsign.csv
     CXR-JPG/
          cxr_chexpert.csv
          cxr_meta.csv
          cxr_negbio.csv
          cxr_split.csv
          s{study_id}/
               {dicom_id}.jpg
     CXR-DICOM/
          s{study_id}.txt
     REFLACX/
          gaze_data/
               {reflacx_id}/
                    gaze.csv
          main_data/
               {reflacx_id}/
                    anomaly_location_ellipses.csv
                    chest_bounding_box.csv
                    fixations.csv
                    timestamps_transcription.csv
                    transcription.txt
          metadata.csv
     EyeGaze/
          audio_segmentation_transcripts\
               {dicom_id}\
                    aortic_knob.png
                    left_lung.png
                    mediastanum.png
                    right_lung.png
                    audio.mp3
                    audio.wav
                    transcript.json
          bounding_boxes.csv
          eye_gaze.csv
          fixations.csv
          master_sheet.csv

spreadsheets\
     CXR-JPG\
          cxr_chexpert.csv
          cxr_negbio.csv
          cxr_split.csv
     Hosp\
          d_hcpcs.csv
          d_icd_diagnoses.csv
          d_icd_procedures.csv
          d_labitems.csv                        
     REFLACX\
          metadata.csv              
     EyeGaze\ 
          bounding_boxes.csv
          eye_gaze.csv
          fixations.csv
          master_sheet_with_updated_stayId.csv                               
     ICU\
          d_items.csv                            
     cxr_meta.csv
     cxr_meta_with_stay_id_only.csv
           

Usage Notes

This dataset includes a variety of inputs and labels. The user is then advised to use different combinations of inputs and labels to perform multi-modal, multi-task, and contrastive learning. A minimum pre-processing is only applied to the REFLACX and EyeGaze data for severe issues as described in the Method section Phase 2 - Pre-processing. No preprocessing is performed on other parent datasets. The missing data and data quality issues presented in MIMIC-Eye are inherited from parent datasets. The user can choose the preprocessing techniques that are suitable for their own task, which not only provides flexibility but also ensures consistency with parent datasets. An example of using clinical data and chest X-ray images to perform multi-modal learning is described in MDF-Net [22], and the implementation can be found in the GitHub repository [21].

Ethics

MIMIC-Eye is a reconstructed version of MIMIC-IV and MIMIC-IV-ED. This dataset exists under the same IRB.

Acknowledgements

This material is based upon work supported by the UNESCO Chair on AI&XR; and the Portuguese Fundação para a Ciência e Tecnologia (FCT) under grants no. 2022.09212.PTDC (XAVIER) and no. UIDB/50021/2020.

Conflicts of Interest

The author(s) have no conflicts of interest to declare.

References

1. Haakenstad, A. et al. Measuring the availability of human resources for457 health and its relationship to universal health coverage for 204 countries and458 territories from 1990 to 2019: a systematic analysis for the global burden459 of disease study 2019 (2022). URL https://doi.org/10.1016/s0140-6736(22)460 00532-3.
2. Maicas, G., Bradley, A. P., Nascimento, J. C., Reid, I. & Carneiro, G.Pre and post-hoc diagnosis and interpretation of malignancy from breast DCE-MRI. Medical Image Analysis 58 (2019). https://doi.org/10.1016/j.media.2019.101562 .
3. Shen, L. et al. Deep learning to improve breast cancer detection on screening mammography. Scientific Reports 9, 2045–2322 (2019) .
4. Liu, X. et al. Deep learning-based automated left ventricular ejectionfraction assessment using 2-d echocardiography. Journal of Physiology Heart and Circulatory Physiology 321, H390–H399 (2020) .
5. Medley, D. O., Santiago, C. & Nascimento, J. C. Cycoseg: A cyclic collaborative framework for automated medical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (11), 8167–8182 (2022). https://doi.org/10.1109/TPAMI.2021.3113077 .
6. Pham, T.-C., Luong, C.-M., Hoang, V.-D. & Doucet, A. Ai outperformed every dermatologist in dermoscopic melanoma diagnosis, using an opti461 mized deep-cnn architecture with custom mini-batch logic and loss function. Scientific Reports 11, 17485 (2021) .
7. Haenssle, H. et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Annals of oncology 29, 1836–1842 (2018) .
8. Irvin, J. et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison, 590–597 (2019).
9. Rajpurkar, P. et al. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. CoRR abs/1711.05225 (2017). https://arxiv.org/abs/1711.05225 .
10. Rajpurkar, P. et al. Deep learning for chest radiograph diagnosis: A retro473 spective comparison of the chexnext algorithm to practicing radiologists. PLOS Medicine 15 (11), 1–17 (2018). https://doi.org/10.1371/journal.pmed.1002686 .
11. Yates, E., Yates, L. & Harvey, H. Machine learning “red dot”: open source, cloud, deep convolutional neural networks in chest radiograph binary normality classification. Clinical Radiology 73 (9), 827–831 (2018). https://doi.org/10.1016/j.crad.2018.05.015 .
12. Lahat, D., Adali, T., and Jutten, C. (2015). Multimodal data fusion: An overview of methods, challenges, and prospects. Proceedings of the IEEE, 103(9):1449–1477.
13. Ramachandram, D. and Taylor, G. W. (2017). Deep multimodal learning: A survey on recent advances and trends. IEEE Signal Processing Magazine, 34(6):96–108.
14. Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L. A., & Mark, R. (2022). MIMIC-IV (version 2.0). PhysioNet. https://doi.org/10.13026/7vcr-e114.
15. Castillo, C., Steffens, T., Sim, L. & Caffery, L. The effect of clinical information on radiology reporting: A systematic review. Journal of Medical Radiation Sciences 68 (1), 60–74 (2021). https://doi.org/10.1002/jmrs.424 .
16. Johnson, A., Bulgarelli, L., Pollard, T., Celi, L. A., Mark, R., & Horng, S. (2022). MIMIC-IV-ED (version 2.0). PhysioNet. https://doi.org/10.13026/as7t-c445.
17. Johnson, A., Pollard, T., Mark, R., Berkowitz, S., & Horng, S. (2019). MIMIC-CXR Database (version 2.0.0). PhysioNet. https://doi.org/10.13026/C2JT1Q.
18. Karargyris, A., Kashyap, S., Lourentzou, I., Wu, J., Tong, M., Sharma, A., Abedin, S., Beymer, D., Mukherjee, V., Krupinski, E., & Moradi, M. (2020). Eye Gaze Data for Chest X-rays (version 1.0.0). PhysioNet. https://doi.org/10.13026/qfdz-zr67.
19. Bigolin Lanfredi, R., Zhang, M., Auffermann, W., Chan, J., Duong, P., Srikumar, V., Drew, T., Schroeder, J., & Tasdizen, T. (2021). REFLACX: Reports and eye-tracking data for localization of abnormalities in chest x-rays (version 1.0.0). PhysioNet. https://doi.org/10.13026/e0dj-8498.
20. Chihcheng H. (2022). MIMIC-Eye Integration Strategy. Available from: https://github.com/ChihchengHsieh/MIMIC-Eye
21. Chihcheng H. (2022). Multi-modal Dual-Fusion Network (MDF-Net). Available from: https://github.com/ChihchengHsieh/multimodal-abnormalities-detection
22. Hsieh, C., Nobre, I. B., Sousa, S. C., Ouyang, C., Brereton, M., Nascimento, J. C., Jorge, J., and Moreira, C. (2023). Mdf-net: Multimodal dual-fusion network for abnormality detection using cxr images and clinical data.
23. Johnson, A., Lungren, M., Peng, Y., Lu, Z., Mark, R., Berkowitz, S., & Horng, S. (2019). MIMIC-CXR-JPG - chest radiographs with structured labels (version 2.0.0). PhysioNet. https://doi.org/10.13026/8360-t248.