The Office of Management and Budget (OMB) wanted to improve how Federal agencies conducted their hiring practice by adding in skills-driven assessments. To convince agencies to make the shift, OMB partnered with the General Services Administration (GSA) to develop a new hiring data dashboard.
During the April 6 Government Summit 2021, sponsored by Tableau, Federal experts – Stephanie Grosser, team lead for the Subject Matter Expert Qualification Assessments (SME-QA) hiring reform initiative at OMB, Ivan Metzger, program manager at GSA, and Elizabeth Keyes, senior analyst at GSA – discussed how OMB, GSA, and the Office of Personnel Management (OPM) partnered to develop and roll out a new hiring data dashboard.
Grosser kicked off the roundtable by discussing the previous hiring status quo at Federal agencies. Essentially, agencies relied on candidate self-assessments, rather than a pass/fail exam that tested key competencies. To improve the quality of applicants being considered, OMB wanted agencies to work with SMEs to create interview assessments for agencies to use.
Initially, Grosser said her team was asking itself “how do we track compliance” with OMB’s guidance on assessments. But, her team quickly realized they should be asking, “how do you push the right behavior?” This sparked the idea of creating a data dashboard, which would enable OMB to not only understand what assessments agencies were using, but also create visuals to help encourage agencies to adopt new types of assessments.
OMB then had to wrestle with whether the data should be available to only Federal employees or the general public. Grosser said if they wanted to keep the data available to only Federal employees, they would need to create a password-protected login system. Her team was worried that this would serve as a barrier to entry. So for that reason, and a desire to promote government transparency, the data was made publicly available.
Digging into their desired outcome – increasing agency use of hiring assessments – Grosser said her team was trying to emphasize behavior change over compliance. They wanted to make sure they weren’t penalizing agencies for not using assessments, but instead, encourage them to use assessments in the future.
OMB wanted to roll out the new dashboard in under two months – a tall order for a platform tracking governmentwide data. To do so, OMB didn’t turn to an external contractor, but rather looked internally to see what resources were available elsewhere in the government. In the end, OMB worked closely with GSA and OPM due to their experience with similar platforms.
Metzger said the team relied heavily on user feedback while building the dashboard. During the feedback process, GSA, OMB, and OPM conducted seven user interviews in a week and collected feedback on a prototype dashboard from 10 users representing six agencies. Once the feedback was collected, the team reviewed it with stakeholders and prioritized what to act on and how to act on it.
During the webinar, Metzger walked through the dashboard, which starts by giving users a “plain language, data-driven narrative” for the data being displayed. The team decided to start the dashboard with a narrative because the data area the dashboard covers is “a bit wonky,” and the team “came to the conclusion that users could need a bit of context to really be set up for success in terms of interpreting the rest of dashboard.”
The dashboard also examines what agencies are using different assessments and which assessments are proving successful in terms of resulting in qualified hires. “We’re not interested in just having agencies adopt additional assessments for their own sake,” Metzger said. “It’s important [that agencies] are adopting effective assessments that are associated with more [hires].” The dashboard also helps advise agencies as to which job categories may “desperately need more effective assessments to improve [hiring] rates.”
While the team was able to quickly roll out the new dashboard, it wasn’t without hiccups along the way. Keyes explained that they were trying to integrate two data sets that weren’t meant to be integrated by pulling data from USAjobs.gov and Monster.com. Because the two data sets weren’t intended to integrate, Keyes said that many of the data fields didn’t match up. So, the team worked across agencies and with Monster.com to merge the data sets into one clean source by using a combo of scripts and data pipeline software.
Keyes said the ability to automate this process with scripts is “incredibly beneficial” as the team looks to do monthly refreshes of the database. She acknowledged that while additional changes may be needed in the future as data evolves and changes, the team has “a good foundation” for that process.
Grosser concluded the session by asking “Can open data change behavior? I think the answer is definitely yes.” She said agencies have always had access to the data displayed on the dashboard within their talent acquisition systems, but it was very difficult to use. “By making the data, open and available at people’s fingertips, we really have the ability to change behavior,” she said.
