About this Resource
The Center’s clinical deployment program serves as the conduit for connecting transformative digital image analysis with patients to achieve better outcomes at lower cost.
Who Can Use This Resource
UCSF scientists and doctors can access the resources of the clinical deployment program.
AI Algorithm Clinical Deployment Evaluation Framework
This is a general framework for evaluation of AI algorithms for clinical deployment. Not all points in this framework will be relevant to every algorithm; each algorithm is likely to have certain unique considerations. This framework tries to give form to the evaluation for approval by covering the most common aspects that should be evaluated, discussed and considered
- Ready for Clinical Deployment = potentially ready to put into place where they will be used in regular clinical workflow
- Algorithms that are in clinical validation mode = initial development of algorithm is complete but do some additional validation in a clinical development and not ready to use for clinical decision making
Analysis – Benefits
- Breadth – how many cases/patients effected?
- Depth – extent of effect on each case both on patients and radiologists
Risk & Costs - Erroneous Results
- Impact on patients
Risk & Costs - Risk Spectrum
- Low – prioritization/triage/alerting with little to no impact on patient
- Moderate – decision/diagnosis support, workflow requires radiology evaluation
- High – decision/diagnosis support that doesn’t require radiology evaluation
- Highest – algorithms that produce clinical outputs that where there is no ability for radiologist to crosscheck; zero detectability and correctability of errors
Monitoring & QA
- Follow-up evaluation of algorithm performance
- Degree of monitoring correlates to Risk Spectrum