The intelligence community’s (IC) stock in trade has always been knowing what nobody else knows. Now it’s looking to tap into new technology to expand its ability to forecast geopolitical events in several ways, including finding out what everybody knows.
The Intelligence Advanced Research Projects Activity (IARPA), the IC’s lead research arm, is looking to bring artificial intelligence to bear in a number of intelligence-gathering areas. This includes two projects at opposite ends of the spy game: at one end, intercepting and interpreting classified communications, and at the other, finding a way to read the tea leaves in massive amounts of public information.
IARPA recently launched a competition to support its Mercury program, which focuses on the more familiar (to the public anyway) element of spycraft known as signals intelligence. SIGINT, as it’s usually called, covers the collection and exploitation of transmissions from communication systems, radars, and weapon systems, according to the CIA’s definition. Although SIGINT has long been a staple of the IC’s activities, IARPA notes that one thing it hasn’t done with SIGINT is to see if it can be used to help predict significant events around the world with a good degree of accuracy and timeliness. In outlining the competition, IARPA cited events such as the fall of the Berlin Wall, the Arab Spring, and Russia’s deny-in-plain-sight incursion into Ukraine as major events that came as a surprise.
The agency–and military intelligence gatherers throughout history–have often relied on publicly available information to help it foresee events, and that’s the focus of another program, Open Source Indicators (OSI), which scans an ever-growing landscape of digital information for signs of coming events.
In both cases, machine learning and AI come into play.
With the prize-based Mercury Challenge, the agency is enlisting technologists, data scientists, and machine learning engineers from industry and academia to develop new machine learning and artificial intelligence techniques that can analyze SIGINT to forecast events involving military action, civil unrest, or infectious diseases, specifically in Arabic-speaking countries in the Middle East and North Africa.
IARPA wants automated, continuous analysis of SIGINT, built on sociological models of population behavior with cutting-edge techniques for data-extraction, and the processing and analysis of streaming data, the agency said. (The challenge is offering prizes totaling $100,000 but, as with other government-sponsored tech competitions, IARPA is promoting the idea that the challenge’s collaborative environment and groundbreaking potential will benefit participants, too.)
OSI, instead, deals with open source intelligence. This might broadly be described as the “word on the street,” but these days the street encompasses websites, chat rooms, social media feeds, and other online communications, as well as TV and radio broadcasts. It involves a nearly unlimited amount of material–Twitter, where hash tags or commonly used phrases can provide clues that something is afoot–hosts more than 500 million tweets a day, on average. Content of interest can include government announcements, academic papers, and community events, along with rumblings on social media.
The sheer volume of data and the wide range of formats calls for automated processes, since it’s impossible for analysts to sort through with any precision. And AI is essential in making it work, since IARPA isn’t just looking for processing speed but wants to incorporate other disciplines such as inductive logic, cognitive biases, informal and probabilistic reasoning, and human judgement. The machines need to understand the subtleties of what’s being discussed in order to help human analysts make any kind of prediction.
While OSI has developed promising methods of making predictions, the lag times and noisy nature involved with public information limits accuracy, hence the goal of taking a similar approach to SIGINT.
IARPA currently has more than a half-dozen projects looking to develop anticipatory intelligence for one reason or another, from recognizing the most promising new technologies to detecting insider threats, in addition to analyzing signals and open-source intelligence. If the IC is to keep up with the increasingly digital world around it, AI and machine learning will be a big part of the program.
