A core outcome set for damage control laparotomy via modified Delphi method =========================================================================== * Saskya Byerly * Jeffry Nahmias * Deborah M Stein * Elliott R Haut * Jason W Smith * Rondi Gelbard * Markus Ziesmann * Melissa Boltz * Ben L Zarzaur * Miklosh Bala * Andrew Bernard * Scott Brakenridge * Karim Brohi * Bryan Collier * Clay Cothren Burlew * Michael Cripps * Bruce Crookes * Jose J Diaz * Juan Duchesne * John A Harvin * Kenji Inaba * Rao Ivatury * Kevin Kasten * Jeffrey D. Kerby * Margaret Lauerman * Tyler Loftus * Preston R. Miller * Thomas Scalea * D Dante Yeh ## Abstract **Objectives** Damage control laparotomy (DCL) remains an important tool in the trauma surgeon’s armamentarium. Inconsistency in reporting standards have hindered careful scrutiny of DCL outcomes. We sought to develop a core outcome set (COS) for DCL clinical studies to facilitate future pooling of data via meta-analysis and Bayesian statistics while minimizing reporting bias. **Methods** A modified Delphi study was performed using DCL content experts identified through Eastern Association for the Surgery of Trauma (EAST) ‘landmark’ DCL papers and EAST ad hoc COS task force consensus. **Results** Of 28 content experts identified, 20 (71%) participated in round 1, 20/20 (100%) in round 2, and 19/20 (95%) in round 3. Round 1 identified 36 potential COS. Round 2 achieved consensus on 10 core outcomes: mortality, 30-day mortality, fascial closure, days to fascial closure, abdominal complications, major complications requiring reoperation or unplanned re-exploration following closure, gastrointestinal anastomotic leak, secondary intra-abdominal sepsis (including anastomotic leak), enterocutaneous fistula, and 12-month functional outcome. Despite feedback provided between rounds, round 3 achieved no further consensus. **Conclusions** Through an electronic survey-based consensus method, content experts agreed on a core outcome set for damage control laparotomy, which is recommended for future trials in DCL clinical research. Further work is necessary to delineate specific tools and methods for measuring specific outcomes. **Level of evidence** V, criteria * abdominal injuries * laparotomy * patient outcome assessment ## Introduction Damage control laparotomy (DCL) is a well-established tool in the surgical armamentarium since its original 1908 description in liver trauma by Dr J. Hogarth Pringle.1 It subsequently achieved renewed interest and acceptance in the 1970s2 and 1980s.3 The term ‘damage control’ was inspired by the naval military experience and captured the concept of containment without definitive repair, specifically in reference to ‘the capacity of a ship to absorb damage and maintain mission integrity’.4 Damage control in abdominal surgery is initial control of hemorrhage and contamination by followed rapid and temporary closure, further resuscitation to normal physiology in the intensive care unit and subsequent definitive re-exploration.5 DCL generally involves a temporary abdominal closure, frequently with pending bowel anastomoses and/or vascular reconstructions to perform. This description is also often referred to as an abbreviated laparotomy. The key elements are rapid control of bleeding and contamination, and in the unstable, acidotic, coagulopathic cold patient, resuscitation prior to performing definitive injury repairs. Adoption of an abbreviated laparotomy has since spread to other surgical fields including emergency general surgery and obstetrics. DCL is associated with decreased mortality5 in patients with the lethal triad (ie, hypothermia, acidosis, and coagulopathy), yet it also has known or posited associated complications including failure to achieve fascial closure, surgical site infection, fascial dehiscence, and enterocutaneous and enteroatmospheric fistula formation.6–10 This has subsequently created elective surgical practices to address such surgical complications. As the DCL experience and number of survivors continue to grow, studies to refine current practice are needed. The National Academies of Medicine suggest that we should approach these problems using an evidence based, data-driven approach.11 However, this can be challenging when reported outcomes in the literature are inconsistent and differ between studies. One way to approach this challenge is by developing a core outcome set (COS)12–14 for all future studies. A COS is a minimum set of outcomes reported that does not preclude investigators from reporting data on additional outcomes.15 This allows for future comparisons via meta-analysis or Bayesian statistics. Furthermore, it minimizes bias since these measures were prespecified and disallow selective reporting of major outcomes. The Core Outcome Measures in Effectiveness Trials (COMET) Initiative16 seeks to develop relevant resources, facilitate the exchange of ideas and information, and foster methodological research to avoid unnecessary duplication of effort, raise awareness of current problems with outcomes in clinical trials, and encourage evidence-based COS development. The Eastern Association for the Surgery of Trauma (EAST) clinical practice management guideline committee has brought attention to the limitations caused by lack of standardized outcomes across published manuscripts in trauma.17 Therefore, the purpose of this study was to develop a COS for future studies regarding DCL via an ad hoc task force of experts. ## Methods The Damage Control Laparotomy Core Outcomes Set (DCL COS) study was developed following the COMET tool and conducted in accordance with recommendations from the Core Outcome Set-Standards for Development and Reporting (COS-STAD).18 Additionally, the Core Outcome Set Standards for Reporting were used to ensure transparency and clarity in reporting outcomes.19 The study was registered with the COMET database. Delphi methodology was selected as the preferred consensus method when compared with nominal process and consensus development. Delphi is time-efficient and cost-efficient and was popularized by the RAND Corporation in the 1940s. Advantages to Delphi technique include elimination of bias by maintaining anonymity, avoiding compromised decision versus an actual consensus, and obviating specific meeting times, which allows respondents to make thoughtful decisions at a convenient time. Disadvantages include participant burnout as rounds increase, and judgment may be influenced by researcher feedback. Panel members were identified based on contribution to the peer-reviewed DCL literature. All EAST Landmark Papers20 for DCL were initially evaluated by a scoping review (MZ and MB performed). All papers that were not primary literature (eg, reviews) were removed, as were papers with a topic-focus other than open-abdomen decision-making or outcomes. The first and last authors of the EAST landmark papers were considered content experts; additional content experts were selected by members of the EAST ad hoc COS task force. Twenty-eight experts were identified based on this method. In the first round, DCL content experts were queried for suggested COS without any limitations. These were collated and grouped by the DCL COS task force with duplicates removed. In the second round, experts were asked to rank each variable using the Grading of Recommendations Assessment, Development and Evaluation scale of 1–9, with scores of 1–3 signifying *a lesser important variable*, 4–6 *important but not critical* and 7–9 *a critically important variable*.21 Consensus was defined as >70% of scores ranging from 7 to 9 and <15% of scores ranging from 1 to 3 as in previous clinical Delphi studies.22 23 Each panelist was provided with their answers as well as the distribution of the group and histograms along with the intraclass correlation (ICC) and current consensus COS of the group. We allowed panelists to include additional COS candidates as well as anonymously give feedback that they felt the other panelists should consider. This de-identified feedback was given to the group and a third round commenced including the voting on the non-consensus COS from the prior round, as well as two yes/no comments related to semantics and definitions brought up previously by the panelists that were prespecified as requiring supermajority of 70% to take effect. The flow diagram is viewable in figure 1. At no time were the panelists given information regarding the identities of the other panelists. Two rounds were required and this has previously been determined to be optimal,24 with more rounds requiring explanation. ![Figure 1](http://tsaco.bmj.com/http://tsaco.smart01.highwire.org/content/tsaco/7/1/e000821/F1.medium.gif) [Figure 1](http://tsaco.bmj.com/content/7/1/e000821/F1) Figure 1 Core outcome set (COS) for damage control laparotomy flow diagram. Statistical analysis was performed by measuring ICC. ICC estimates and their 95% CIs were calculated using R statistical package ‘ICC’ V.2.3.025 based on a mean-rating (k=12), absolute-agreement, 2-way mixed-effects model. Accepted definitions of agreement: <0.50= poor; 0.50–0.74=moderate; 0.75–0.90=good; 0.90–1.00=excellent. All statistical tests were performed in the R V.4.0.2 environment.26 ## Results Twenty-eight content experts were identified and were invited to participate. Round 1 was conducted between December 15 and December 31, 2020 with a response rate of 20/28 (71%). Each of the 20 participants (online supplemental file 1) proposed between 3 and 10 outcomes by free response, resulting in 36 unique proposed outcomes (box 1). Several of the proposed entries were demographics and not actual outcomes (‘study population’, ‘primary diagnosis’, ‘indication for abbreviation of laparotomy’) and were removed in subsequent rounds. There were four variables included for consideration for mortality (in-hospital, 28-day, 30-day and 90-day) and these were not considered mutually exclusive. Round 2 was conducted between January 1 and January 31, 2021, with 20/20 (100%) participants. Ten outcomes achieved consensus: in-hospital mortality, 30-day mortality, fascial closure, days to fascial closure, abdominal complications, major complications requiring reoperation or unplanned re-exploration following closure, gastrointestinal anastomotic leak, secondary intra-abdominal sepsis (including anastomotic leak), enterocutaneous fistula, and 12-month functional outcome. Results were shared with the group along with individual feedback. ICC (two-way mixed-effects model with the average of k=20 raters) was 0.87, 95% CI (0.81 to 0.92), p<0.001. Email feedback was encouraged from participants to identify any additional outcomes and to provide additional feedback supporting or opposing any particular candidate outcome. A definition of intra-abdominal infection (IAI) was proposed to combine terms of gastrointestinal anastomotic leak, abscess/deep surgical site infection, secondary intra-abdominal sepsis, and abdominal complication. Feedback also challenged the contention that abdominal closure was synonymous with fascial closure. Two additional proposed COS (IAI and maximum lactate in first 24 hours) were submitted along with feedback regarding six existing proposed outcomes. These were provided to the group as well in preparation for round 3 voting. ### Supplementary data [[tsaco-2021-000821supp001.pdf]](pending:yes) Box 1 ### Thirty-six ‘write-in’ proposed core outcomes by content experts in round 1* * Study population * Primary diagnosis * Indication for abbreviation of laparotomy * Blood product transfusion (whole blood vs component transfusion) * 24-hour packed red blood cells * Time to normalize lactate * Time to hemostasis (surgical control of bleeding) * Time interval between operations * Involvement of specialty surgeon at second operation * Time to enteral feeding * Fascial closure (vicryl mesh is NOT a fascial closure) at index hospitalization * Abdominal closure * Days to fascial closure * (Planned) ventral hernia * Postoperative ventral hernia formation 12–24 months follow-up * 90-day postoperative complication rate * Complications/Abdominal complications * Major complications requiring reoperations/unplanned re-exploration following closure * Sepsis/Intra-abdominal sepsi * Intra-abdominal abscess/surgical site infection III/deep space infection * Gastrointestinal anastomotic leak, if applicable * Secondary intra-abdominal sepsis (including anastomotic leak) * Enterocutaneous fistula * Acute kidney injury/presence and degree of organ failure * Acute respiratory distress syndrome * Deep vein thrombosis * Hospital length of stay * Intensive care unit length of stay * Ventilator days * Intensive care unity length of stay with ventilator free days * Hospital mortality * 28-day mortality * 30-day mortality * 90-day mortality (not sooner) * Long-term physical and psychological functional assessment * Functional outcome at 12 months (return to work, pain score, etc) * *Responses are displayed exactly as received by the content experts in round 1 without any edits or modifications. The Delphi process for round 3 was conducted between February 15 and April 1, 2021. No new COS were identified through consensus and neither the semantics challenge nor new definition met the supermajority (70%) requirement to change the language of the study. The overall response rate was 95% (19 of 20), with one expert not responding in the format requested. ICC (two-way mixed-effects model with the average of k=19 raters) was 0.83, 95% CI (0.73 to 0.90), p<0.001. With no further improvement in consensus, the Delphi study was concluded (box 2). Box 2 ### Ten outcomes achieving consensus for damage control laparotomy core outcome set #### Core outcomes for damage control laparotomy * Fascial closure at index hospitalization * Days to fascial closure * Abdominal complications * Major complications requiring reoperation or unplanned re-exploration following closure * Gastrointestinal anastomotic leak * Secondary intra-abdominal sepsis (including anastomotic leak) * Enterocutaneous fistula * In-hospital mortality * 30-day mortality * Functional outcome at 12 months (return to work, pain score, etc) ## Discussion DCL is an established tactic in emergency surgery for trauma, and its long-term consequences and outcomes are not entirely well-defined. The EAST ad hoc COS task force identified DCL as an important topic that would benefit from a COS to improve future research. Using an accepted consensus method, 19 content experts participated in this study to define a 10 item COS (box 2), which represent the minimum outcomes any future DCL study should include. This is the first COS developed for DCL with the intention of facilitating higher quality studies,27 with more easily combined results for pooled analysis. We recommend DCL studies measure these outcomes *at minimum* and also encourage investigators to include other outcomes relevant to their hypotheses. The goal of DCL is to stop hemorrhage, re-establish critical vascular perfusion, and control contamination while delaying definitive surgery once the patient’s physiology is optimized. Given the ultimate goal of DCL is survival, it is not surprising that two core outcomes are in-hospital mortality and 30-day mortality. Mortality was one of five outcomes reported in a recent DCL systematic review and meta-analysis along with days to fascial closure and abdominal complications which were also identified as core outcomes by our expert panel.28 Fascial closure at index hospitalization, major complications requiring reoperation or unplanned re-exploration following closure, gastrointestinal anastomotic leak, secondary intra-abdominal sepsis (including anastomotic leak), and enterocutaneous fistula were all identified as core outcomes. Functional outcome at 12 months was also identified as a COS variable and was the only outcome (besides 30-day mortality, potentially) that usually extends beyond the patient’s hospital course. Patient-reported outcomes have recently been identified as an important future area of study.29 ## Limitations Factors such as the participant panel, number of survey rounds, feedback between rounds, and the ability of panelists to add their own views must be considered according to COS-STAD.18 While we attempted to follow best practices outlined via COMET and Delphi guidelines, it should be noted that there are several limitations to this study. First, there are limitations specific to Delphi compared with other consensus methods. Expert panels are ideally 6–11 participants; our panel was larger, with 20 participants. Prior studies identify panel sizes of 6–11 as ideal. Dagenais30 reported a monotonic increase in reliability of Delphi as the size of the panel increases. His study ended at 11 participants on the panel with reliability index of 0.76. Nair *et al*31 suggested that a number of experts on a panel ‘can be hundreds, but at least 10–30’, and that <6 had limited reliability and groups over 12 had insignificant reliability. This larger cohort makes consensus harder to achieve but alternatively, smaller groups usually have fewer novel suggestions.24 Unfortunately, we were unable to include DCL survivors due to recruitment and organizational restraints. These would be important stakeholders to include in future COS development as the importance of patient-centered outcomes is paramount to the next evolution of surgical care. In the future, formalized community programs using national organizations such as the Coalition for National Trauma Research32 may facilitate patient participation in this important effort. Clear definitions of our trauma outcomes are not well-defined based on the several definition clarifications we had to make within the modified Delphi process among our panel experts and while the group defined functional outcome as a consensus outcome, we did not strictly define which measure should be reported in the literature. Finally, the authors acknowledge that the technique of damage control is not standardized within the trauma community. Although achieving consensus about the ideal technique is outside the scope of this core outcome study, this represents an area ripe for future research. ## Conclusion Our Delphi process using 20 content experts of damage control laparotomy achieved consensus on 10 core outcomes. This study is the first to standardize important outcomes for future DCL research and facilitate pooling of results via systematic review, meta-analysis, and/or Bayesian analysis. ## Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. ## Ethics statements ### Patient consent for publication Not applicable. ### Ethics approval The study was deemed exempt according to University of Tennessee Health Science Center Institutional Review Board, Reference 908142, and all involvement with human participants were in accordance with the ethical standards of the institutional committee and with the 1964 Helsinki Declaration and its later amendments. ## Footnotes * Contributors SB made substantial contributions to the conception and design of the study and to acquisition, analysis, and interpretation of the data. She drafted the article and revised it critically for important intellectual content and gave final approval of the version to be published as the guarantor of the article. JN, DMS, ERH, JWS, RG, MZ, MB, BLZ, and DDY made substantial contributions to the conception and design of the study, and to analysis and interpretation of the data. They revised the article critically for important intellectual content and gave final approval of the version to be published. MB, AB, SB, KB, BC, CCB, MC, BC, JJD, JD, JAH, KI, RI, KK, JDK, ML, TL, PM, and TS made substantial contributions to the acquisition and interpretation of the data. They revised the article critically for important intellectual content and gave final approval of the version to be published. * Funding EH reports research funding from The Patient-Centered Outcomes Research Institute (PCORI), the Agency for Healthcare Research and Quality (AHRQ), the NIH/NHLBI, the DOD/Army Medical Research Acquisition Activity, and the Henry M. Jackson Foundation for the Advancement of Military Medicine (HJF). EH receives royalties from Lippincott, Williams, Wilkins for a book—'Avoiding Common ICU Errors'. EH was a paid speaker for the Vizient Hospital Improvement Innovation Network (HIIN) VTE Prevention Acceleration Network. TL was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number K23 GM140268. * Provenance and peer review Not commissioned; internally peer reviewed. [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/). ## References 1. Pringle JH, Jh P. V. notes on the arrest of hepatic hemorrhage due to trauma. Ann Surg 1908;48:541–9.[doi:10.1097/00000658-190810000-00005](http://dx.doi.org/10.1097/00000658-190810000-00005)pmid:http://www.ncbi.nlm.nih.gov/pubmed/17862242 [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1097/00000658-190810000-00005&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=17862242&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) [Web of Science](http://tsaco.bmj.com/lookup/external-ref?access_num=000201621400049&link_type=ISI) 2. Lucas CE, Ledgerwood AM. Prospective evaluation of hemostatic techniques for liver injuries. J Trauma 1976;16:442–51.[doi:10.1097/00005373-197606000-00003](http://dx.doi.org/10.1097/00005373-197606000-00003)pmid:http://www.ncbi.nlm.nih.gov/pubmed/778397 [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1097/00005373-197606000-00003&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=778397&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) [Web of Science](http://tsaco.bmj.com/lookup/external-ref?access_num=A1976BW50100002&link_type=ISI) 3. Stone HH, Strom PR, Mullins RJ. Management of the major coagulopathy with onset during laparotomy. Ann Surg 1983;197:532–5.[doi:10.1097/00000658-198305000-00005](http://dx.doi.org/10.1097/00000658-198305000-00005)pmid:http://www.ncbi.nlm.nih.gov/pubmed/6847272 [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1097/00000658-198305000-00005&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=6847272&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) [Web of Science](http://tsaco.bmj.com/lookup/external-ref?access_num=A1983QQ64700005&link_type=ISI) 4. Survivability SS. Naval war publication 3–20.31. Washington: DC Dep Def, 1996. 5. Rotondo MF, Schwab CW, McGonigal MD, Phillips GR, Fruchterman TM, Kauder DR, Latenser BA, Angood PA. 'Damage control': an approach for improved survival in Exsanguinating penetrating abdominal injury. J Trauma 1993;35:375–82.pmid:http://www.ncbi.nlm.nih.gov/pubmed/8371295 [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1097/00005373-199309000-00008&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=8371295&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) [Web of Science](http://tsaco.bmj.com/lookup/external-ref?access_num=A1993LY24200008&link_type=ISI) 6. Hatch QM, Osterhout LM, Podbielski J, Kozar RA, Wade CE, Holcomb JB, Cotton BA. Impact of closure at the first take back: complication burden and potential overutilization of damage control laparotomy. J Trauma Acute Care Surg 2011;71:1503–11.[doi:10.1097/TA.0b013e31823cd78d](http://dx.doi.org/10.1097/TA.0b013e31823cd78d) 7. Pommerening MJ, Kao LS, Sowards KJ, Wade CE, Holcomb JB, Cotton BA. Primary skin closure after damage control laparotomy. Br J Surg 2015;102:67–75.[doi:10.1002/bjs.9685](http://dx.doi.org/10.1002/bjs.9685)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25393208 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 8. Harvin JA, Wray CJ, Steward J, Lawless RA, McNutt MK, Love JD, Moore LJ, Wade CE, Cotton BA, Holcomb JB. Control the damage: morbidity and mortality after emergent trauma laparotomy. Am J Surg 2016;212:34–9.[doi:10.1016/j.amjsurg.2015.10.014](http://dx.doi.org/10.1016/j.amjsurg.2015.10.014)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26754456 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 9. Fischer PE, Fabian TC, Magnotti LJ, Schroeppel TJ, Bee TK, Maish GO, Savage SA, Laing AE, Barker AB, Croce MA. A ten-year review of enterocutaneous fistulas after laparotomy for trauma. J Trauma 2009;67:924–8.[doi:10.1097/TA.0b013e3181ad5463](http://dx.doi.org/10.1097/TA.0b013e3181ad5463)pmid:http://www.ncbi.nlm.nih.gov/pubmed/19901649 [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1097/TA.0b013e3181ad5463&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=19901649&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) [Web of Science](http://tsaco.bmj.com/lookup/external-ref?access_num=000271729500005&link_type=ISI) 10. Teixeira PGR, Inaba K, Dubose J, Salim A, Brown C, Rhee P, Browder T, Demetriades D. Enterocutaneous fistula complicating trauma laparotomy: a major resource burden. Am Surg 2009;75:30–2.[doi:10.1177/000313480907500106](http://dx.doi.org/10.1177/000313480907500106)pmid:http://www.ncbi.nlm.nih.gov/pubmed/19213393 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=19213393&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) [Web of Science](http://tsaco.bmj.com/lookup/external-ref?access_num=000262332000006&link_type=ISI) 11. Haut ER, Mann NC, Kotwal RS. Military trauma care’s learning health system: the importance of data driven decision making. Comm Mil Trauma Care’s Learn Heal Syst its Transl to Civ Sect 2016. 12. Blackwood B, Ringrow S, Clarke M, Marshall JC, Connolly B, Rose L, McAuley DF. A core outcome set for critical care ventilation trials. Crit Care Med 2019;47:1324–31.[doi:10.1097/CCM.0000000000003904](http://dx.doi.org/10.1097/CCM.0000000000003904)pmid:http://www.ncbi.nlm.nih.gov/pubmed/31356474 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 13. Tong A, Elliott JH, Azevedo LC, Baumgart A, Bersten A, Cervantes L, Chew DP, Cho Y, Cooper T, Crowe S, et al. Core outcomes set for trials in people with coronavirus disease 2019. Crit Care Med 2020;48:1622–35.[doi:10.1097/CCM.0000000000004585](http://dx.doi.org/10.1097/CCM.0000000000004585)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32804792 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 14. Rose L, Burry L, Agar M, Campbell NL, Clarke M, Lee J, Marshall JC, Devlin JW, Blackwood B, Needham DM, et al. A core outcome set for research evaluating interventions to prevent and/or treat delirium in critically ill adults: an international consensus study (Del-COrS). Crit Care Med 2021;49:1535–46.[doi:10.1097/CCM.0000000000005028](http://dx.doi.org/10.1097/CCM.0000000000005028)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33870914 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 15. Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials 2012;13:1–8.[doi:10.1186/1745-6215-13-132](http://dx.doi.org/10.1186/1745-6215-13-132) [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1186/1745-6215-13-1&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=22214287&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 16. COMET. Core Outcome Measures in Effectiveness Trials (COMET) Initiative [Internet]. [https://www.comet-initiative.org](https://www.comet-initiative.org). 17. Haut ER. Eastern association for the surgery of trauma (East) practice management guidelines and the perpetual quest for excellence. J Trauma Acute Care Surg 2020;89:1–10.[doi:10.1097/TA.0000000000002709](http://dx.doi.org/10.1097/TA.0000000000002709)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32251261 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 18. Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, Williamson PR. Core outcome Set-STAndards for development: the COS-STAD recommendations. PLoS Med 2017;14:e1002447. [doi:10.1371/journal.pmed.1002447](http://dx.doi.org/10.1371/journal.pmed.1002447)pmid:http://www.ncbi.nlm.nih.gov/pubmed/29145404 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 19. Kirkham JJ, Gorst S, Altman DG, Blazeby J, Clarke M, Devane D, Gargon E, Williamson PR. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials 2015;16:1–6.[doi:10.1186/s13063-015-0913-9](http://dx.doi.org/10.1186/s13063-015-0913-9) [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1186/1745-6215-16-1&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=25971836&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 20. EAST. Landmark Papers in Trauma and Acute Care Surgery. [https://www.east.org/education-career-development/publications/landmark-papers-in-trauma-and-acute-care-surgery](https://www.east.org/education-career-development/publications/landmark-papers-in-trauma-and-acute-care-surgery). 21. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, Alderson P, Glasziou P, Falck-Ytter Y, Schünemann HJ. Grade guidelines: 2. framing the question and deciding on important outcomes. J Clin Epidemiol 2011;64:395–400.[doi:10.1016/j.jclinepi.2010.09.012](http://dx.doi.org/10.1016/j.jclinepi.2010.09.012)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21194891 [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2010.09.012&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=21194891&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 22. Avery KNL, Chalmers KA, Brookes ST, Blencowe NS, Coulman K, Whale K, Metcalfe C, Blazeby JM, . ROMIO Study Group, CONSENSUS Esophageal Cancer Working Group. Development of a core outcome set for clinical effectiveness trials in esophageal cancer resection surgery. Ann Surg 2018;267:700–10.[doi:10.1097/SLA.0000000000002204](http://dx.doi.org/10.1097/SLA.0000000000002204)pmid:http://www.ncbi.nlm.nih.gov/pubmed/28288055 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 23. Knaapen M, Hall NJ, Moulin D, van der Lee JH, Butcher NJ, Minneci PC, Svensson JF, St Peter SD, Adams S, Nah SA, et al. International core outcome set for acute simple appendicitis in children: results of a systematic review, Delphi study, and focus groups with young people. Ann Surg 2020. [Epub ahead of print: 29 Dec 2020].[doi:10.1097/SLA.0000000000004707](http://dx.doi.org/10.1097/SLA.0000000000004707)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33630468 24. Waggoner J, Carline JD, Durning SJ. Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad Med 2016;91:663–8.[doi:10.1097/ACM.0000000000001092](http://dx.doi.org/10.1097/ACM.0000000000001092)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26796090 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=26796090&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 25. Wolak ME, Fairbairn DJ, Paulsen YR. Guidelines for estimating repeatability. Methods Ecol Evol 2012;3:129–37.[doi:10.1111/j.2041-210X.2011.00125.x](http://dx.doi.org/10.1111/j.2041-210X.2011.00125.x) [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1111/j.2041-210X.2011.00125.x&link_type=DOI) 26. R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2020. [https://www.R-project.org/](https://www.R-project.org/). 27. Kerwin AJ, Haut ER, Burns JB, Como JJ, Haider A, Stassen N, Dahm P, . Eastern Association for the Surgery of Trauma Practice Management Guidelines Ad Hoc Committee. The eastern association of the surgery of trauma approach to practice management Guideline development using grading of recommendations, assessment, development, and evaluation (grade) methodology. J Trauma Acute Care Surg 2012;73:S283–7.[doi:10.1097/TA.0b013e31827013e9](http://dx.doi.org/10.1097/TA.0b013e31827013e9)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23114482 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=23114482&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 28. Sharrock AE, Barker T, Yuen HM, Rickard R, Tai N. Management and closure of the open abdomen after damage control laparotomy for trauma. A systematic review and meta-analysis. Injury 2016;47:296–306.[doi:10.1016/j.injury.2015.09.008](http://dx.doi.org/10.1016/j.injury.2015.09.008)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26462958 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 29. Sakran JV, Ezzeddine H, Schwab CW, Bonne S, Brasel KJ, Burd RS, Cuschieri J, Ficke J, Gaines BA, Giacino JT, et al. Proceedings from the consensus conference on trauma patient-reported outcome measures. J Am Coll Surg 2020;230:819–35.[doi:10.1016/j.jamcollsurg.2020.01.032](http://dx.doi.org/10.1016/j.jamcollsurg.2020.01.032)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32201197 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 30. Dagenais F. The reliability and convergence of the Delphi technique. J Gen Psychol 1978;98:307–8.[doi:10.1080/00221309.1978.9920886](http://dx.doi.org/10.1080/00221309.1978.9920886)pmid:http://www.ncbi.nlm.nih.gov/pubmed/28136700 [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 31. Nair R, Aggarwal R, Khanna D. Methods of formal consensus in classification/diagnostic criteria and Guideline development. Seminars in arthritis and rheumatism. 41: Elsevier, 2011:95–105.[doi:10.1016/j.semarthrit.2010.12.001](http://dx.doi.org/10.1016/j.semarthrit.2010.12.001) [CrossRef](http://tsaco.bmj.com/lookup/external-ref?access_num=10.1016/j.semarthrit.2010.12.001&link_type=DOI) [PubMed](http://tsaco.bmj.com/lookup/external-ref?access_num=21420149&link_type=MED&atom=%2Ftsaco%2F7%2F1%2Fe000821.atom) 32. CNTR. Coalition for national trauma research clinical trials network. [https://www.nattrauma.org/](https://www.nattrauma.org/).