A randomized evaluation at a large community hospital produced a result that is becoming familiar: a well-performing model, faithfully deployed, with no detectable effect on the outcome it was meant to improve.
The model identified deteriorating inpatients with an area under the curve around 0.87 on prospective data — genuinely good. Alerts fired to a rapid response team that was already operating at capacity. Response times to model-generated alerts averaged forty-three minutes, against nine minutes for nurse-initiated calls.
The investigators are careful in their framing: this is not evidence that the model does not work. It is evidence that prediction without capacity is not an intervention. The paper's discussion section argues that trials of clinical AI should be required to report the response resource as an explicit study arm.
The hospital has kept the model and changed the workflow, routing alerts to a dedicated overnight nurse rather than the shared team. A follow-up evaluation is underway. The initial cost of that nurse exceeded the annual license fee for the model by a factor of roughly six.
