The Department of Defense’s (DoD) artificial intelligence (AI) development and deployment efforts thus far lacks the communication, strategy, and metrics necessary for success, according to a December 2019 RAND Corporation report mandated by Congress.
“The RAND report provides a thorough and thoughtful critique of the state of AI within DoD,” CIO Dana Deasy said in a statement. “We will take this assessment along with other recent reports by the National Security Commission on AI and the Defense Innovation Board to forge a way forward for the multi-generational task of accelerating the fielding of AI-enabled capabilities throughout the U.S. military.”
As a part of the 2019 National Defense Authorization Act (NDAA), RAND was selected to research the state of AI at DoD and make recommendations for improvement. RAND concluded that: current technology to support AI is optimized for commercial use rather than DoD specific needs; the state of AI verification, validation, test, and evaluation (VVT&E) cannot support the performance and safety of AI systems; and DoD should maintain realistic expectations about AI deployment.
Those concerns permeated all factors of the agency’s AI strategy, RAND said. Lack of AI baselines and metrics, disparate communication channels between AI developers and users, and processes hindering innovation were all cited as concerns. Additionally, poor access to data and difficulties cultivating talent dampen AI progress, RAND said.
“Many different technologies underpin AI,” Danielle Tarraf, lead author of the RAND report said in a press release. “The current excitement, and hype, are due to leap-ahead advances in Deep Learning approaches. However, these approaches remain brittle and artisanal – they are not ready yet for prime time in safety-critical systems,” she said.
RAND made seven recommendations to the Pentagon:
- DoD should adapt AI governance to align authorities and resources with the mission of scaling AI;
- DoD should advance the practice of AI VVT&E alongside industry and academia;
- All AI funding at DoD should include a budget for VVT&E;
- All DoD agencies should have mechanisms for connecting AI researchers, developers, and operators;
- DoD should recognize data as critical resources, institute practices for collection and curation, and increase sharing while resolving data protection issues;
- The DoD Chief Data Officer should make select data sets available to the AI community to foster innovation; and
- DoD should embrace permeability and openness to enhance AI talent.
At DoD’s Joint Artificial Intelligence Center (JAIC), a lack of visibility and authority prevents the department from carrying out its present role. Further, a lack of long-term strategy and funding could hinder the department’s success. Three additional recommendations addressed JAIC specifically:
- JAIC should develop a five-year strategic road map to scale AI;
- JAIC, alongside DoD partners, should carry out annual or biannual portfolio reviews of AI investments; and
- JAIC should organize a technical workshop showcasing AI programs.
“The DoD recognizes that AI could be a game-changer and has set up organizational structures focusing on AI…But currently the JAIC doesn’t have the authorities or resources it needs to carry out its mission,” Tarraf said.
JAIC spokesperson Lt Cmdr. Arlo Abrahamson told MeriTalk that the department has made continuous efforts to improve.
“Since the time the RAND report was commissioned, the Department of Defense has not been standing still,” Abrahamson said. “The department, along with Congress, have arrived on some of the same conclusions that the RAND report recommends. With enacted and proposed legislation, along with internal actions within the department, we believe we are making incremental steps that will allow the JAIC to fulfill its mandate.”
