Derivation and validation of a novel severity classification for intraoperative adverse events

Background: There is currently no systematic approach to evaluating the severity of intraoperative adverse events (iAEs).

Study design: A 3-phase project was designed to develop and validate a novel severity classification scheme for iAEs. Phase 1 created the severity classification using a modified Delphi process. Phase 2 measured the classification's internal consistency by calculating inter-rater reliability among 91 surgeons using standardized iAEs scenarios. Phase 3 measured the classification's construct validity by testing whether major iAEs (severity class ≥3) correlated with worse 30-day postoperative outcomes compared with minor iAEs (severity class <3). This was achieved by creating a matched database using American College of Surgeons NSQIP and administrative data, querying for iAEs using the Patient Safety Indicator #15 (Accidental Puncture/Laceration), and iAE confirmation by chart review.

Results: Phase 1 resulted in a 6-point severity classification scheme. Phase 2 revealed an inter-rater reliability of 0.882. Of 9,292 patients, phase 3 included 181 confirmed with iAEs. All preoperative/intraoperative variables, including demographics, comorbidities, type of surgery performed, and operative length, were similar between patients with minor (n = 110) vs major iAEs (n = 71). In multivariable logistic analysis, severe iAEs correlated with higher risks of any postoperative complication (odds ratio [OR] = 3.8; 95% CI, 1.9-7.4; p < 0.001), surgical site infections (OR = 3.7; 95% CI, 1.7-8.2; p = 0.001), systemic sepsis (OR = 6.0; 95% CI, 2.1-17.2; p = 0.001), failure to wean off the ventilator (OR = 3.2; 95% CI, 1.2-8.9; p = 0.022), and postoperative length of stay ≥7 days (OR = 3.0; 95% CI, 1.5-5.9; p = 0.002). Thirty-day mortalities were similar (4.5% vs 7.1%; p = 0.46).

Conclusions: We propose a novel iAE severity classification system with high internal consistency and solid construct validity. Our classification scheme might prove essential for benchmarking quality of intraoperative care across hospitals and/or individual surgeons.