- GAO has issued a draft guide outlining best practices for testing practitioners, program managers and procurement officials
- The guide provides a testing and evaluating process agencies can follow before purchasing COTS systems
- GAO has identified weaknesses in agency testing through previous reviews
The Government Accountability Office released an exposure draft guide laying out a five-step process for testing and evaluating commercial off-the-shelf systems before federal agencies buy them, with public comments due by Dec. 14.
What Does the Guide Cover?
In a report published Monday, GAO said the guide focuses on operational testing of complete, integrated and technically sophisticated hardware systems that generally include software, and it excludes standalone software along with basic components such as nuts and bolts. It lays out a five-step process agencies can use across the board for planning, executing and reporting T&E results, along with best practices meant to help agencies standardize their approach, manage risk and keep vendors informed. GAO said the guide draws a line between mission-critical systems, whose failure could carry severe consequences including loss of life, and noncritical systems that call for a lighter testing load.
Who Is the Guide Meant to Help?
Agency testing and evaluation practitioners, program managers, and procurement officials stand to gain the most direct benefit, with GAO saying they can put the guide’s practices to work toward sharper evaluations, technology that actually holds up once purchased and fewer budget or timeline surprises down the road. Organizations without their own T&E policies can use the guide to build one, while those with existing guidance can use it to fill in gaps. The guide also gives GAO and other oversight bodies a yardstick for judging agency testing, and manufacturers can use it to see how their products measure up against agency requirements.
What Problem Is GAO Trying to Solve?
Federal agencies spend billions of dollars each year on commercial hardware, and GAO said its own prior work turned up shortcomings in how agencies test those systems, with no official government community of practice in place for sharing lessons learned between agencies. GAO has also identified testing-policy gaps at the Department of War, including a lack of practices such as early tester involvement, iterative testing and ongoing user feedback. The exposure draft is GAO’s attempt to close that gap, aimed at helping agency officials make sound purchasing decisions and avoid the cost overruns and schedule delays that come with ineffective technology.
