The National Science Foundation (NSF) is ahead of schedule in developing a new chatbot that will assist the research community in navigating NSF program websites and matching their ideas with current solicitations, according to a top agency official.
NSF Chief Data Officer and AI Official (CAIO) Dorothy Aronson explained today that the chatbot is the result of NSF’s AI Pilot program – which is separate from the National Artificial Intelligence Research Resource (NAIRR) pilot. The pilot began on Oct. 1, 2023, and will end on Sept. 30, 2024, in alignment with fiscal year 2024.
The agency is working with a vendor to design the chatbot so that the pilot can “move as quickly as possible,” Aronson said. Per the vendor’s request, NSF provided 50 sample questions for the vendor to build the chatbot, and the agency now has a proof of concept.
“We actually were able to develop the proof of concept, the initial one, faster than we expected,” Aronson said during today’s AI/Automation Workshop hosted by Nextgov/FCW. “So, we were able to get the data provided to the vendor, we were able to build the initial tool, we were able to tune it for performance, and then we went to a small, friendly group of customers and they were very happy with it.”
Some examples of questions that a customer could ask the chatbot are “What is the status of my proposal?” or “Is my proposal relevant to any open solicitations?”
However, Aronson said that NSF soon learned that 50 questions “is not nearly enough” to build a chatbot.
“We went to a larger group of customers, and they were very unhappy with it,” she said. “So, right now we’re at this point where we’re like, ‘Well, why were they unhappy?’ They’re unhappy because even though it answers the original set of questions pretty nicely, people aren’t going to ask a controlled set of questions.”
Aronson compared the current chatbot to a one-year-old “baby AI,” saying that “no one wants to ask a one-year-old a question about how to work with NSF.” However, she said people may want to ask a 40-year-old a question about how to work with NSF.
“How do we grow the one-year-old to be 40 in a few months? That’s the challenge,” Aronson said. “This is the moment where I stand right now on the precipice of making this decision about how to move forward with this.”
NSF is in the process of further building and fine-tuning the chatbot to better meet customer needs, the CAIO explained.
The agency expects to finish building and testing the chatbot by the end of July. Then, it hopes to publicize the chatbot throughout August and September with a road show – making improvements as necessary.
“This has been a very exciting effort for us. We learn a lot every step of the way. We’ve learned so much about the process of developing AI … and it’s been a joy,” Aronson said. “So, how do we make it win? That’s my next project. Can’t talk about that today, we have three months to figure that one out.”
