Like state governments, the federal government faces many challenges when changing the processes and tools used to develop the software that agencies depend on. Waterfall methodology may appear to be less risky, but is it?

The Risks of Waterfall Methodology

For a long time, the waterfall process has been considered reliable, methodical, and less risky than the DevOps “move fast and release often” approach.

In a waterfall-based software development process, stakeholders spend considerable time documenting requirements and contingencies ahead of time because it may be years between when a software contract is awarded and the software is delivered. Even within the vendor doing the development work, developers work in isolation before handing off a release for testing and security validation. Any errors or vulnerabilities found at this stage send the release back to the developers; unfortunately, so much of the code is intertwined that attempts to debug and fix the errors create additional work or even new bugs.

These delays can add years and millions to the cost of a software project. For example, the Air Force awarded a 2013 contract to update its Air Operations Center (AOC) software. By 2016, the project was three years behind schedule and $371 million over budget, yet still needed to deliver working code. Not only were these costs wasteful, but they also continued to put Americans at risk in wartime environments.

Common Challenges to Moving to DevOps

The federal government is sometimes synonymous with the word “bureaucracy,” and breaking down silos in a bureaucracy is challenging. Fragmented budgets, strict procurement guidelines that limit tool choice, compliance, and audits are just some of the obstacles that would-be DevOps champions face when implementing a change.

DevSecOps can add focus on the importance of security and compliance. Meanwhile, more agencies have proven the value of DevOps and DevSecOps practices, so obtaining buy-in from leaders is becoming easier.

DevOps Saves Lives and Millions of Dollars

Fed up with the AOC software project, the Air Force ran a trial in agile development with a project named Jigsaw Tanker Planning Software. Unlike the AOC software, Jigsaw took mere months to develop. As a result of fuel efficiencies from using the software in planning, the Air Force realized a cost savings of $750,000 to $1 million every week. The trial was a success, and Project Kessel Run was born to explore DevOps in other development projects. Because of the investment in DevOps teams, tools, and methodology, the Air Force quickly resolved a scaling issue in its production software that enabled it to extract thousands of Americans, interpreters, and refugees from Kabul in 2021.

Reduced Downtime and Easier Audits

When the US Patent and Trade Office piloted DevOps for nine months, they were able to fix defects and roll out improvements fast with zero downtime. Under the waterfall method, changes used to incur twelve hours of downtime to implement—the ability to deliver while maintaining rigorous standards convinced even the most vocal skeptics.

Another benefit is that the USPTO is now better positioned to comply with audit requests. By automating standard processes, auditors can pull logs showing precisely what was completed and whether it matches the expected procedures. Automation reduces the chance of human error on the front end but also reduces the need to stop work and sit in on an audit interview.

Reallocate Time and Resources for High-Value Projects

When agencies save millions in costs, like the Air Force, or reduce outages as the USPTO did, they can reallocate resources to other projects. As software becomes more reliable, other departments also benefit from the freed-up time and can focus on innovation or more complex problem-solving.

Other Federal DevOps Initiatives

If you’re interested in exploring initiatives beyond the Air Force and USPTO examples, here are some reading recommendations:

• General Services Administration is expanding its DevOps program to DevSecOps
• NIST National Cybersecurity Center of Excellence’s Software Supply Chain and DevSecOps project.

Wherever you are in your DevOps journey, ReleaseTEAM has experience working with government agencies, is a DevOps tool reseller, and provides training. Read how we helped the Colorado Secretary of State migrate multiple legacy tools and implement DevSecOps principles in this case study.