Coalition Proposes ‘Nutrition Label’ for Health AI
The nonprofit Coalition for Health AI (CHAI) has released details about how it will certify independent assurance labs that will vet AI algorithms. It plans to standardize the output of these labs through CHAI "model cards," which it describes as being like ingredient and nutrition labels.
The CHAI certification process and model card design are expected to be available by the end of April 2025, following a process of review and feedback by CHAI members, partners and the public.
In a recent interview with Healthcare Innovation, Brian Anderson, M.D., CEO of CHAI, explained the timeliness of this work. “When you have health systems that are delivering care and building out increasing number of use cases with AI, they want to be able to partner with technology vendors and ensure that those AI tools are not going to be discriminatory, that those AI tools are not going to harm people,” he said. “And in order for health systems to have a transparent dialog with technology vendors, they need to have common definitions. So if I'm a health system, and I want to ask a technology vendor for the specifics on what went into their training data, it would be helpful for an organization like CHAI to describe what those standards are, what is the bare minimum that needs to go into the description of your training data, and that's particularly important in the generative AI space.”
Anderson added that if a health system wants to make sure that the model isn't discriminatory or doesn’t have unjustified bias to particular populations, they would want to be able to go to their technology vendor and say, ‘Show me the testing evaluations for bias against specific populations.’ “We don't have an ability to do that in generative AI, because we don't have those common definitions yet,” he explained. “So this assurance standards guide is building those common definitions, those common frameworks, so that health systems can have more informed and more rigorous dialog with technology vendors to bring greater transparency to some important parts of models development.”
In the past, CHAI has said that it expects a federated network of as many as 30 assurance labs to be created to vet AI algorithms.
The draft CHAI certification program framework was created working with the ANSI National Accreditation Board and several emerging quality assurance labs using ISO 17025, the predominant standard for testing and calibration laboratories worldwide. Among the requirements are mandatory disclosure of conflicts of interest between assurance labs and model developers and the protection of data and intellectual property.
Additionally, the certification program integrates data quality and integrity requirements derived from FDA's draft guidance on use of high-quality real-world data, CHAI testing and evaluation metrics sourced from the various working groups, and alignment with the National Academy of Medicine’s AI Code of Conduct.
The organization said the draft CHAI model card presents a standard template to provide a degree of transparency with key information to support the evaluation of AI solution performance and safety. The model card includes the identity of the developer, intended uses, targeted patient populations, AI model type, data types, key performance metrics, security and compliance accreditations, maintenance requirements, known risks and out-of-scope uses, known bias and ethical considerations and third-party information (e.g. relevant clinical studies).
The model card was designed by a workgroup representing a range of stakeholders including regional health systems, EHR solution vendors, medical device makers and health AI leaders and start-ups.
It was designed as a starting point for those reviewing AI models during the procurement process and for electronic health records (EHR) vendors who need to comply with the ONC Health IT Certification Program (HTI-1). CHAI completed an assessment of all of the HTI-1 requirements and gathered consensus recommendations from clinicians, health system organizational data custodians, and developers about what additional information should be included beyond the existing regulatory requirements.
The initial drafts of the certification program and model cards are being presented at the CHAI Global Summit at the HLTH 2024 meeting in Las Vegas. CHAI said it would engage stakeholders across the healthcare ecosystem including patient advocates, under-resourced local health systems and start-ups for additional feedback.
