Machine Learning Model May Reduce Unnecessary Type-and-Screen Orders in Surgery

A personalized machine learning model to estimate surgical transfusion risk demonstrated strong external validity across diverse hospital settings and may improve blood utilization if adopted widely, according to findings from a large, multi-center study published in JAMA Network Open.

Researchers from Washington University School of Medicine externally validated the Surgical Personalized Anticipation of Transfusion Hazard (S-PATH) model using data from more than 3.2 million surgical cases across 45 U.S. hospitals. Unlike the standard maximum surgical blood ordering schedule (MSBOS), which bases testing recommendations on procedure type, S-PATH incorporates both patient- and procedure-specific variables to guide preoperative type-and-screen decisions.

Compared with MSBOS, S-PATH recommended type-and-screen testing for a median of 17.9% fewer patients (32.5% for S-PATH versus 51.6% for MSBOS) while maintaining 96% sensitivity. The model also showed superior performance in predicting transfusion needs, with a higher median area under the receiver operating characteristic curve (AUROC, 0.929 versus 0.857) compared with MSBOS.

Additionally, the investigators found that the model performed consistently across academic and community hospitals without requiring local retraining, although they observed some variation in S-PATH performance across hospitals, especially among hospitals with lower surgical volumes. They believe this may result from smaller hospitals having different practice patterns, reduced access to transfusion resources and different transfusion preferences.

“These results suggest that model validation within local contexts will continue to be necessary before implementation of S-PATH as clinical decision support,” the researchers wrote, “especially for smaller hospitals.”

According to the authors, S-PATH’s robust performance across institutions may be attributed to several factors. First, the model uses historical transfusion rates for each procedure as an input, which allows for model customization to specific hospitals. Second, S-PATH was also developed using a large, multi-institutional dataset with manually validated data elements, which may have increased the likelihood of the model learning robust relationships between the input variables and the transfusion outcome. Third, the model relies on only 20 clinically relevant input variables that are consistent with clinician judgment and practical to extract from electronic health records.

While further validation may be warranted, the investigators believe “S-PATH’s generalizability and robustness and provide evidence to support its potential for pragmatic clinical value in improving resource allocation if implemented broadly.”