Skip to main content
Data Source
N/A
Organizer
Dr. Christopher James Doig, Dr. Christiane Job McIntosh

Background

Patients admitted to the ICU are the sickest and most complex in the health care system often requiring expensive lifesaving technologies and interventions (e.g., invasive monitoring, intubation and mechanical ventilation, vasoactive medications, dialysis etc.)(Jacobs and Noesworthy, 1990). Further, care within an intensive care unit is provided by a collaborative team of health professionals, including nurses, respiratory therapists, pharmacists, occupational and physiotherapists, and social workers. Many efforts have been made to develop ICU metrics to reflect the performance of a healthcare system. One definition of a physician’s clinical performance (PCP) is “the quantitative assessment of physician performance based on the rates at which their patients experience certain outcomes of care and the rates at which physicians adhere to evidence-based processes of care during their actual practice of medicine”(Street, 2006). Traditionally, physicians participate in performance evaluations that are called 360 evaluations. The 360 model of feedback utilizes information from self-assessment, colleagues, non-physicians and patients. These assessments are useful, but fail to consider additional sources of data (e.g. patient specific outcomes including ICU or hospital length of stay, complications, and mortality) that may provide a more encompassing picture of physician performance.  We are interested in developing a model for assessing the performance of an individual physician. Performance is a confluence of multiple factors, particularly knowledge, skills, and behavioral competence. PCP assessment is beneficial to ensuring high-quality medical care, and may inform quality improvement initiatives amongst physicians.

Physician Participants:

In this dataset, each physician had 360 feedback evaluations completed anonymously by a random sample of allied health professionals working in the ICU, and peer physicians including medical directors, physicians from non-critical care disciplines. Questions are linked to 7 domains (Medical Expert, Communicator, Leader, Advocate, Professional, Scholar, and Collaborator). Each question was scored on a 5-points likert scale.  


Physician characteristics available included: (1) age, (2) background medical discipline before critical care training, (3) an administrative leader defined as a physician, (4) an education leader in the university, (5) university rank.

Patient Participants:

At hospital discharge, trained health record technicians abstract patient-specific data, including demographic, clinical, and administrative information. All multisystems ICU’s conform to the same standards for nursing, respiratory therapy and other allied health care professional training, have uniform departmental policies and protocols, standard equipment (eg, monitoring, drug infusion, and ventilators), and a standard drug formulary. However, each ICU might have different working environments, including variability in patient profiles (some which may not be measured/available e.g. socio-economic status, ethnic background, chronic disease burden). 

Patient Status and Outcome Scores:

APACHE II is an ICU-specific acuity of illness score calculated at the time of ICU admission based on the derangement of 12 physiological variables, age, and a chronic health score.(Naved et al., 2011) 
The SOFA (Sepsis-associated organ failure assessment) score was developed by an expert consensus panel to describe the severity of organ failure at and following ICU admission. In addition, SOFA was designed to complement the acuity of illness score.(Lambden et al., 2019).

 

Research Question

Data Description

The data to be utilized in this case study challenge is ‘mock’ data that has been developed in consultation from ICU healthcare service professionals. Data descriptions are provided below.

Case Study Challenge

Based on the data sets provided, your challenge is to develop valid quality indicators to evaluate the performance of critical care physicians. For example, early appropriate medically expert diagnoses may result in earlier expert interventions, such as resuscitation, and antibiotic treatment, which are known to decrease patient mortality rates. Alternatively, even if the physician is an expert, most interventions are dependent on allied health professionals to carry out the interventions recommended by the physician, therefore, it may be the physician that is the expert team leader/communicator/collaborator that has the best outcomes.   

Patients admitted to the ICU are assigned to the attending physician on duty for the duration of their stay.  One possible way to approach this case study challenge would be to examine the variability between ICU physicians on the outcomes of their critically ill patients. One may examine the data sets provided below to determine if the physician 360 evaluations explain any observed differences in patient outcomes.  Alternatively, individuals that work in a particular ICU may have developed a ‘team culture’ and therefore examining outcome by ICU might be another approach. 

How you choose to address the challenge presented above is up to you.  If you are unsure whether your research question(s) would meet the case study competition criteria, please check with the organizers. You may also consider any data or parameters found from publicly available sources and/or published literature.

 

Variables

N/A

 

Data Access

The datasets described above can be downloaded from here:

  • Link for physician 360-degree evaluation data
  • Link for patient characteristic data at ICU admission
  • Link for patient SOFA trajectory data
  • Link for doctor characteristic data
  • Link for Data Dictionary

Organizer Contact Information

This case study was prepared by Dr. Christopher James Doig, Dr. Christiane Job McIntosh, and Dr. Chel Hee Lee with help and guidance from the other members of the case study committee of the Statistical Society of Canada. Any questions or concerns can be directed to: christiane.jobmcintosh@ahs.ca or chelhee.lee@ahs.ca

Award

We are pleased to announce that an award of $750 will be given to the winning team by the Department of Critical Care Medicine, Alberta Health Services & University of Calgary. In addition to the financial award there may be potential for research opportunities/collaborations for the successful team members.

 

References

Jacobs, P., & Noseworthy, T. W. (1990). National estimates of intensive care utilization and costs: Canada and the United States. Critical Care Medicine, 18(11), 1282–1286.
Lambden, S., Laterre, P. F., Levy, M. M., & Francois, B. (2019). The SOFA score—development, utility and challenges of accurate assessment in clinical trials. Critical Care, 23(1), 1–9.
Naved, S. A., Siddiqui, S., & Khan, F. H. (2011). APACHE-II score correlation with mortality and length of stay in an intensive care unit. Journal of the College of Physicians and Surgeons Pakistan, 21(1), 4.
Street, A. (2006). Future of quality measurement in the National Health Service. Expert Review of Pharmacoeconomics & Outcomes Research, 6(3), 245–248.