Pentagon concludes pilot program using chatbots for military medicine

The US Department of Defense’s Chief Digital and Artificial Intelligence Office (CDAO) has concluded a pilot focused on using AI chatbots in military medical services.  

In the Jan. 2 announcement, the DoD said the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program pilot focused on the use of large-language models (LLM) for clinical note summarization and as medical advisers in the military.

It comes as more AI firms have begun offering their products to the US military and defense contractors to investigate their usefulness in military applications.

According to the DoD, the pilot was a red teaming effort conducted by technology nonprofit Humane Intelligence.

It attracted over 200 independent external participants, including clinical providers and healthcare analysts, who compared three prominent chatbot models.

Analysts from the Defense Health Agency and the Uniformed Services University of the Health Sciences also collaborated with the other participants, testing for potential system weaknesses and flaws while the chatbots were used.

According to the DoD, the pilot discovered a few hundred possible issues when using chatbots in military medical applications.

“The exercise uncovered over 800 findings of potential vulnerabilities and biases related to employing these capabilities in these prospective use cases.”

“This exercise will result in repeatable and scalable output via the development of benchmark data sets, which can be used to evaluate future vendors and tools for alignment with performance expectations,” the DoD said.

The Chief Digital and Artificial Intelligence Office’s lead for the initiative, Matthew Johnson, said the results will also be used to shape future DoD research and development of Generative AI (GenAI) systems that may be deployed in the future.

The CDAO was established in June 2022 to oversee and advance the integration of digital and artificial intelligence technologies within the US military and defense operations.

Related: Google touts ‘new agentic era’ with AI agents powered by Gemini 2.0

Last November, a bipartisan US congressional commission said the country should focus on developing an initiative similar to the Manhattan Project to advance artificial intelligence development amid growing competition with China.

Among a list of specific recommendations, the commission said the US secretary of defense should mark AI projects with the highest national priority designation.

Meanwhile, social media and tech firm Meta has started offering its artificial intelligence model Llama to the US military and defense contractors for national security purposes.

Magazine: How crypto laws are changing across the world in 2025