OpenOximetry Repository

Abstract

The OpenOximetry Repository is a structured database designed to store clinical and laboratory pulse oximetry data and allows for consolidation of data sets held by collaborating organizations. Matched or independent readings of oxygen saturations, arterial blood gas samples, high frequency processed and unprocessed waveforms, and other physiologic metadata are supported (e.g. skin color readings). Contribution of data to the repository is encouraged; a data dictionary and standardized data collection protocols are provided to ensure consistent data archival and interpretation. The goal of this dataset is to enable researchers to interrogate the effects of physiologic and device variables on pulse oximeter performance. 

A set of controlled desaturation studies is included in the initial release with ongoing updates planned as more data are available from both controlled lab desaturation studies and real world prospective clinical trials. Data captured during these studies may include paired pulse oximeter and arterial blood gas readings throughout a range of oxygen saturations, patient demographics and comorbidities, longitudinal quantitative and qualitative skin color measurement (using multiple methods  at multiple anatomic locations), finger diameter, continuous ECG, arterial blood pressure, EtCO2/EtO2, unprocessed photoplethysmography (PPG), pH, PaCO2, hemoglobin, met-Hgb, CO-Hgb, respiratory rate and multiple other variables depending on the data source

Background

Pulse oximetry is a standard of care tool for measuring hemoglobin oxygen saturation in the blood and is used in all medical contexts worldwide to directly inform diagnosis, monitoring and treatment decisions. Though they have been used for many years, there is still much unknown about how this technology interacts with different patient characteristics. For instance, it is known that pulse oximeters can read higher the actual saturation (as measured in the blood) in patients with darker skin tones [1-3]; this can have real clinical impact [4]. Further still, there are a host of other variables that are not thoroughly investigated, such as finger diameter, probe location, the effects of anemia, the pulsatility amplitude, temperature, device characteristics like probe type and more. 

Differences in pulse oximeter performance are opaque and information is difficult for consumers to retrieve. Devices exist that are marketed for consumer use outside the healthcare setting, or have not met existing FDA or ISO standards. The performance of these devices can vary considerably, but publicly available data demonstrating this is rare [5, 6].

Data needed to investigate questions of performance and the impact of variables on performance is difficult to acquire. The specialized lab conditions required to compare against the gold standard (arterial blood sampling) and safely test devices in desaturated patients are expensive and require expertise to use. Likewise, collection of high quality, real time clinical data is time consuming and logistically difficult, and datasets tend to be small. Retrospective data sets help fill a gap but can be limited by their ability to produce contemporaneous observations.

This repository serves as a location to bring together both laboratory based desaturation studies and data from real world prospective clinical trials. By leveraging a common data structure, otherwise disparate data sets can be harmonized to help answer these import questions on pulse oximetry. The initial release contains a data set of laboratory based controlled desaturation studies. Healthy volunteers are desaturated in a controlled environment while obtaining repeated arterial blood gas samples, allowing for the comparison of pulse oximeter SpO2 readings to the gold standard of the arterial blood gas SaO2 over a range of saturations.

This longitudinal data set is released to help researchers and device developers answer questions regarding the impact of patient variables on pulse oximeter performance, as well as the impact of these factors across different brands and models over time.

Methods

Data acquisition:

Data are collected in one of two ways: 

1. Data are collected from healthy humans undergoing controlled desaturation ‘breathe down’ studies in accordance with protocols [7].
2. Data are collected from prospective studies in real world clinical settings that adhere to protocols [8].

Briefly, during a typical study participant encounter, baseline demographics and measurements, such as qualitative and quantitative skin tone, finger diameter, gender, height, weight and blood pressure are obtained.  Radial arterial access is obtained (or already in place if in a clinical trial). Oximeters are confirmed to be correctly positioned on the participant or patient, and optically shielded from each other. 

In a laboratory controlled desaturation study, the participant breathes via a closed circuit system whereby levels of medical air, oxygen, nitrogen, and carbon dioxide are adjusted. The blend of air and nitrogen is titrated to desaturate the participant to stable target oxygen saturation (SaO2) plateaus distributed over the range of SaO2 70-100% in accordance with ISO80601 standards and US FDA Guidance. When the target plateau stability is reached, a series of arterial blood gas samples are obtained and run immediately on a calibrated co-oximeter (e.g. Radiometer ABL90 Flex Plus). At the time of each blood sample, the saturation reading of each test device is recorded, and is denoted incrementally as the “sample number” within an encounter.

Detailed information on the Hypoxia Lab study protocol can be found on the OpenOximetry website [7]. 

For data captured in prospective, real world clinical studies, adult or pediatric patients are enrolled in accordance with methods approved by local Institutional Review Boards (IRBs). Methods may vary in accordance to study design, but will include measurement of clinical and study pulse oximeter values and arterial blood gas SaO2 values (and the time interval between observations, where applicable).

Detailed information on the OpenOx Clinical Trial Protocol can be found online [7].

Additional information is captured at each encounter. This includes:

• Finger diameter measured with calipers
• Monk skin tone scale
• Quantitative skin color via spectrophotometer 
• Please see the OpenOximetry Skin Color Assessment Protocol for further information [9]

Data sources:

Deidentified data are uploaded to this OpenOx Repository from one of two sources:

1. Data collected by the UCSF Hypoxia Lab or UCSF EquiOx Clinical Trial are de-identified and directly uploaded on an ongoing basis
2. Data on pulse oximetry that adhere to OpenOx data collection standards/protocols and have been independently uploaded to Physionet will be reviewed by the OpenOx team and merged into this dataset upon request by the submitting team (NOTE: the review process examines only the data structure and the reported protocol for acquisition. The OpenOx team takes no responsibility and makes no claims to the accuracy or quality of those data). 

Devices that are tested by the Hypoxia Lab with data published in this data repository were independently chosen and purchased by the Hypoxia Lab for the purpose of independent evaluation. These devices were chosen based on relevance and prevalence in global markets. Device selection for other studies that may be included in this dataset was done at the discretion of the PIs for those studies. All studies are conducted in accordance with local IRBs for each contributing study.

Data updates:

Data will be regularly added to this data repository as they become available from controlled desaturation studies as well as prospective real world clinical studies. Fields and tables may be updated as new data is contributed that does not map to an existing field.

Please note: in the current version, laboratory desaturation data is provided unprocessed as collected. In upcoming releases of the repository, we will implement a standardized data cleaning process to filter out potential artifacts or inaccuracies introduced during data collection. Details regarding the filtering criteria will be published on this page when the data is updated.

Data Description

Data Tables

The OpenOximetry Repository is a relational database consisting of a number of tables linked by common identifiers. For an in-depth description of the specific fields in each table, please consult the data dictionary included (OpenOximetry_Data_dictionary_v1.0.0.csv )

Patients

The patients.csv file contains information about individual patients/participants.  Basic, relatively immutable information is included in this file, with each participant having one row in the participants file.  Participants are given a unique de-identified participant ID so that they can be identified between encounters (i.e. each participant may have many encounters), and to preserve privacy. Additionally, information about the site that is conducting the study is included in this file so that participants can be selected and stratified by site if desired during analysis.

Encounter

The encounter.csv file contains information from each controlled desaturation “breathe down” encounter or clinical patient sample.  Each row of the encounter table includes information from a single encounter (so examining a single encounter requires looking at only one row of the encounter table; this is not the case for other files).  Each encounter corresponds to one participant (i.e. each row has one participant ID).  Information about which device is used on which finger/location, finger diameter, and measurements made during the encounter are included in this file. Qualitative skin color measurements are also included here (e.g. Monk Skin Tone Scale). In clinical studies where these measurements occur at enrollment only, this data will be included at the initial encounter only. Dates in the encounter table are offset into the future by a random offset (set per participant) while attempting to preserve seasonality to de-identify the dataset. This preserves intra-participant chronology while inter-participant chronology is obscured.

Pulse oximeter

The pulseoximeter.csv file contains pulse oximeter saturation readings for the different test devices. Each row contains one saturation reading from one pulse oximeter, corresponding to a specific controlled desaturation encounter (identified by encounter_id), and a specific blood gas (identified by sample_number).

Blood gas

The bloodgas.csv file contains blood gas measurements and data obtained at the same time as the blood gas. Each row represents the results from a single blood gas sample and clinical data from that time. These samples are linked to pulse oximeter readings and other data during the same encounter via sample_number and encounter_id.

Device

The devices.csv file contains descriptive data about the pulse oximeters used in testing. These data will be published when available, though may not always be available and are not required for publishing data in this repository. Each row contains data for one pulse oximeter model and includes a unique device ID number and light transmission mode of the oximeter.

Spectrophotometer

The spectrophotometer.csv file contains quantitative skin color data. Each row represents one spectrophotometer measurement at a single anatomic site. Spectrophotometer measurements are typically taken three times at each anatomic site, and repeated at each encounter. They are linked to the encounter via encounter_id, and to the patient via patient_id. Multiple spectrophotometers may be used for each subject. Further information can be found on the OpenOximetry protocols page [9].

Waveform Files

Continuous waveform data is stored in WFDB or CSV format, with file names corresponding to the encounter.

Three different waveform files are available:

1. (150hz, 'encounter_id'.dat): Continuous data contained here includes ECG, oxygen and carbon dioxide in mmHg, respiratory rate, and arterial blood pressure in mmHg. Recordings are made to 3 decimal places.
2. Reference oximeter readings (2Hz, file suffix _2hz.csv): Continuous data from devices used as known reference oximeters during an encounter are included as csv files. These files contain SpO2, HR, and pulsatility index (PI) recorded at 2Hz. The calculated saturation (ScalcO2) is the instantaneously estimated SaO2 based on ETO2. Blood sample draws are labeled and correspond to the ‘sample’ number in other tables. 
3. Raw photoplethysmograph (PPG) (86hz, file suffix _ppg.dat): Raw PPG is included for certain encounters, and PPG files are named according to their corresponding encounter. However in this version of the database, they are not time-synchronized to the other waveform data from the same encounter.

Usage Notes

Users may filter and join tables to obtain the data set required. For instance, to examine the paired SpO2 and SaO2 measurements for a specific pulse oximeter, the user would first filter devices.csv based on device_id. This table would then be joined on encounter_id and device_id to the pulseoximeter.csv file(to obtain SpO2 values), and to the abg.csv file on encounter_id and sample (to obtain the ABG results for each sample during the encounter). Demographics or skin color data could be added by joining again on the patient.csv or spectrophotometer.csv files. 

Within the waveform directory, intermediate directories 0-9 and a-f contain all waveform records for encounters that begin with that character.  For example, the waveform records for encounter b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9 would be found in the b intermediate directory.  Additionally, raw PPG data in this release is available for only a subset of encounters as a separate, non-time-synced WFDB file (suffix _ppg), recorded at 86 Hz. 

There are many potential use cases for this data that have not yet been explored. The data set contains many physiologic variables that could affect pulse oximeter performance but have not been explored. In addition, questions regarding the interaction of these variables and quantitative and qualitative skin color are possible. As a repository of skin color data, separate questions may be asked regarding the relationship of quantitative and qualitative skin color measurements. Finally, analysis of high frequency PPG, respiratory and blood pressure waveforms can be analyzed for their interaction on pulse oximeter performance or the physiologic effects of hypoxemia.

Contributors with pulse oximeter performance testing data who wish to be included in this database may contact the correspondence address above for more information. Contributed data must adhere to the existing data dictionary. Contributors with fields that do not yet exist in the data dictionary or that do not fit the existing schema are also encouraged to contact our team. Our goal is to include all relevant variables to pulse oximetry and we will work with you and your data to do so.

Release Notes

Version 1.0.1 - Methods text updated to clarify current and planned data filtering steps. Data remains the same as initial release.

Ethics

All data included in this repository were collected with approval from the University of California, San Francisco IRB (#23-40212), including written informed consent for study participants. 

Acknowledgements

The Open Oximetry Project has been supported by the Gordon and Betty Moore Foundation, The Patrick J. McGovern Foundation, the Robert Wood Johnson Foundation, UNITAID/PATH through UNITAID, UCSF and USAID STAR.

Contributors of data to this repository include (this list is updated regularly): 

• The UCSF Hypoxia Lab

Conflicts of Interest

There are no conflicts of interest to declare. The UCSF Hypoxia Research Laboratory receives funding from pulse oximeter manufacturers/sponsors to test the sponsors’ devices for the purposes of product development and regulatory performance testing. Data in this repository do not include sponsors’ study devices. All data are collected from devices procured by the Hypoxia Research Laboratory for the purposes of independent research. No pulse oximeter company provides direct funding of the Open Oximetry Project, participates in study design or analysis, or is involved in creation of this data repository. None of the investigators who maintain this database own stock or equity interests in any pulse oximeter device companies.

Disclaimer

This OpenOximetry.org Project is a collaborative effort by representatives of multiple institutions, and not all collaborators are directly involved in the creation or maintenance of this repository. The information herein are intended and designed for academic purposes. The creators and hosts of this dataset make no claims or warranties as to the currency, accuracy, or suitability of this information for any purpose. Any reference to specific equipment, pharmaceuticals, or other medical devices in this repository is not meant as an endorsement of such items. The use of any information from this repository is undertaken solely at your own risk, and the creators and hosts of this repository shall not be liable for any damages, losses, or other injury caused by the use of any information from this repository, nor for any reliance on the accuracy or reliability of such information. 

References

1. Feiner JR, Severinghaus JW, Bickler PE. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender. Anesth Analg. 2007;105: S18–S23. doi:10.1213/01.ane.0000285988.35174.d9
2. Bickler PE, Feiner JR, Severinghaus JW. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology. 2005;102: 715–719. doi:10.1097/00000542-200504000-00004
3. Shi C, Goodall M, Dumville J, Hill J, Norman G, Hamer O, et al. The accuracy of pulse oximetry in measuring oxygen saturation by levels of skin pigmentation: a systematic review and meta-analysis. BMC Med. 2022;20: 267. doi:10.1186/s12916-022-02452-8
4. Sjoding MW, Dickson RP, Iwashyna TJ, Gay SE, Valley TS. Racial Bias in Pulse Oximetry Measurement. N Engl J Med. 2020;383: 2477–2478. doi:10.1056/NEJMc2029240
5. Leeb G, Auchus I, Law T, Bickler P, Lipnick M, The Performance of 11 Fingertip Pulse Oximeters During Hypoxemia in Healthy Human Subjects with Varied, Quantified Skin Pigment. Available at SSRN: https://ssrn.com/abstract=4578562 or http://dx.doi.org/10.2139/ssrn.4578562
6. Lipnick, Michael S., John R. Feiner, Paul Au, Michael Bernstein, and Philip E. Bickler. 2016. “The Accuracy of 6 Inexpensive Pulse Oximeters Not Cleared by the Food and Drug Administration: The Possible Global Public Health Implications.” Anesthesia and Analgesia 123 (2): 338–45.
7. Laboratory-based, controlled hypoxemia pulse oximeter human study protocol. Open Oximetry: Study protocols. Available from: https://openoximetry.org/study-protocols/
8. Real world clinical study of pulse oximeter performance in critically-ill adults and pediatric patients. Open Oximetry: Study Protocols. Available from: https://openoximetry.org/study-protocols/
9. Skin color quantification for pulse oximeter human study protocol. Open Oximetry: Study Protocols. Available from: https://openoximetry.org/study-protocols/