FDA Seeks Feedback on Oversight of Generative Artificial Intelligence

The FDA has posted a request for information (RFI) for the use of generative artificial intelligence (generative AI), asking stakeholders to opine on matters such as the risks inherent to the use of this class of software. The agency is hinting that its expectations for postmarket surveillance for generative AI may be exceptional due to difficulties in evaluating this type of software in the premarket setting.

Much of the content of the FDA discussion paper attached to the Aug. 18, 2026, RFI is based on an advisory hearing conducted roughly two years ago. The November 2024 meeting of the digital health advisory committee led to several specific recommendations related to risk. Among these were that developers of generative AI products should provide the FDA with information on hallucination rates and estimates of the uncertainty of the algorithm’s output. The committee also suggested that the FDA may need new strategies and new regulatory frameworks to properly oversee several types of risk, including those associated with ethical use of generative AI and the usability of generative AI devices.

The discussion paper indicates that FDA staff at the Center for Devices and Radiological Health (CDRH) believe that new methodologies may be required to evaluate the performance of generative AI device functions. Even with the help of new methodologies, it may be difficult to adequately evaluate some generative AI products in the premarket setting because of the prospect that the algorithm will undergo continuous adjustment. Other considerations, including but not limited to broad intended use statements, prompted CDRH staff to conclude that it may be impractical to attempt to evaluate every possible input that will be introduced to the algorithm during routine clinical use.

For these and other reasons, CDRH stated that it may have little choice but to require that premarket evidence be augmented by robust postmarket evidence generation. This might be accomplished by monitoring methods that can track the performance of the generative AI. The degree of this type of monitoring activity would be based on the risks inherent to the design of the generative AI, which in turn are determined by two types of factors.

One of these factors is the severity of the consequences of a malfunction of the generative AI and the other the degree of autonomy of the generative AI’s function. The agency does not state that a combination of full autonomy and severe consequences would make a generative AI product a class III, high-risk device, but neither does the discussion draft disavow such a possibility.

One method of controlling the reliability of generative AI for medical use may be a credentialing process for health care professionals. This notion is supported by the argument that generative AI is too adaptive and opaque to rely on conventional FDA premarket review, and thus medical licensure requirements and board certification evaluations may aid in assurances that generative AI can be deployed with a reasonable assurance of safety and effectiveness. While the FDA cannot compel state boards of medical licensure to mandate generative AI training for physicians, the agency can stipulate that any purchasers or licensees of the generative AI have undergone training for the use of the product.

The discussion draft also examines the possibility that device benchmarking processes would allow the agency to evaluate the performance of the generative AI. A benchmarking methodology might be periodically deployed in the postmarket realm as well, along with a routine clinician review of the generative AI. CDRH indicated it is also amenable to the use of performance degradation monitoring techniques that would aid in detection of output drift.

However, responsibility for this kind of surveillance would not fall entirely on the developer, the FDA paper states, citing healthcare institutions, payers and standard-setting bodies as entities that might also play a role in tracking the performance of generative AI. The agency is taking comment on the discussion paper through Oct. 19, 2026.