The Circular Firing Squad of Defense Acquisition Rhetoric
June 30, 2017
Good People Get Bad Press
To say the Department of Defense (DoD) gets bad press regarding its acquisition system is like saying the sun rises in the east. It’s frankly an easy target for those looking to score political points or clicks on websites. So, it’s worth distinguishing reality from sensationalism.
Over decades, a highly specialized group of government and industry officials have become skilled at navigating the myriad of public and private interests, regulations, and political toxicity surrounding defense acquisition programs. A relatively small number of people operate a system that is much closer to a nationalized defense industry than an open marketplace, yet they are judged in terms that are irrelevant to the quality of their work or the performance of the acquisition system as a whole.
For our acquisition system to work perfectly, it must:
- Correctly identify the most relevant current and future operational needs;
- Survey all of industry and government to identify the best technical solutions to meet operational needs;
- Impartially award contracts that optimize tax payer investment;
- Assure that contractors achieve goals for performance, cost, and schedule;
- Ensure—to an extraordinary degree—that the end product is safe and reliable;
- Affordably sustain the product throughout its lifecycle; and
- Exhaustively document all the above for review by various overseers.
Each step is at risk for error and frequently affected by external factors. Yet, naysayers point to any real or perceived error to reaffirm “the defense acquisition system is broken.” There are endemic flaws in the way the Defense Department acquires weapon systems, but there are also sound practices. Rather than dwell on the good and bad elements of defense acquisitions, the focus here is on the terms of our national discourse.
First, it’s important to keep perspective. Any nation on the planet would gladly trade its Defense Department for ours. They want our people and our weapon systems; they would probably take our bureaucracy as part of the deal. Given that reality, it’s fair to start with a little good news. In the most recent “Report on the Progress of the Defense Acquisition System,” the former undersecretary of defense for acquisition, technology and logistics (USD AT&L), Frank Kendall, notes that the 5-year moving average of cost growth on our largest and highest-risk programs is at a 30-year low. In clear, data-driven terms, the defense acquisition system is producing positive results.
Chewing on the “Defense Acquisition System is Broken” Bone
The U.S. Government Accountability Office (GAO) added acquisition of weapons systems to its High Risk List (HRL) in 1990. In its 2015 update, GAO asserts that the DoD has no comprehensive action plan to remove it. Government watchdogs perennially cite defense programs as underperforming. Excessive bureaucracy is regularly cited as part of the problem, for example “… programs GAO surveyed spent, on average, over two years completing numerous information requirements for their most recent milestone decision, yet acquisition officials considered only about half of the requirements as high value.”
GAO’s opinion aside, the defense acquisition reputation continues to suffer from lingering memory of a series of well-publicized acquisition disasters. A Google search on “defense acquisition disaster” returns the “Top Ten Failed of the RMA (Revolution in Military Affairs) Era,” including: #5, 2011 cancellation of USAF’s Airborne Laser @15 years and $5B; #4, 2011 cancellation of NOAA/USAF National Polar-Orbiting Environmental Satellite System @17 years and $5.8B; on-going, Joint Tactical Radio System @19 years and $6+B (depending how you count the waste); 2004 cancellation of the Army’s RAH66 Comanche Helicopter @22 years and $6.9B; and 2009 cancellation of the Army’s Future Combat System @10 years and $19B. According to the author:
“Over time, this dynamic has resulted in the most technologically advanced military on the planet. Unfortunately, it is also by far the most expensive military and increasingly involves years of delays and cost overruns. Like the Zumwalt-class destroyer, current defense programs such as the Ford-Class Carrier, the F-22 Raptor and the much-maligned F-35 Joint Strike Fighter are producing incrementally fewer platforms and exponentially higher cost.”
However, the most recent cancellation of an “acquisition disaster” was more than five years ago. The JTRS program has been de-scoped with some of its sub-elements now apparently on track. Despite dire concerns, the Air Force successfully fielded its variant of the F-35 last year and the unit cost is on track to drop at least 15 percent. The Air Force’s Operational Control Segment (OCX), which is a critical capability to manage future global positioning system (GPS) signals, has received a lot of bad press but is now showing notable signs of progress. Even the GAO HRL acknowledges that DoD initiatives, such as “Better Buying Power” and the annual “Report on the Performance of the Defense Acquisition System,” are substantial steps in the right direction.
So, it’s time to declare “Mission Accomplished” in reforming defense acquisitions…right?
Of course not. No rational person would claim that our acquisition system doesn’t need major improvement. In fact, virtually every secretary of defense since World War II has launched an acquisition reform initiative. During that period, at least 150 significant watchdog reports on acquisition reform were delivered; the directive describing the defense acquisition system, DoD Instruction 5000.02, has been reissued every few years; and the following major changes have been implemented:
- Establishing the Federal Acquisition Regulation (FAR);
- Establishing Defense Acquisition University to train the acquisition workforce;
- Instituting a clear management chain (program manager, program executive officer, service acquisition executive, undersecretary of defense) to foster accountability and authority…then restructuring that management chain;
- Implementing a milestone decision process to improve oversight;
- Requiring independent cost estimates to improve budget forecasting;
- Establishing a joint requirements board to improve requirements development and eliminate duplicative programs;
- Increased use of commercial technology; and
- Using congressionally approved multiyear procurements to lower costs.
If repeating the same process and expecting a different result is the definition of insanity, and if the optimism expressed in the latest AT&L report on acquisition performance is not crazy, then: (1) the latest DoD acquisition reforms must—this time—represent fundamentally effective changes; or (2) despite a bum rap to the contrary, historical changes to the acquisition process have, in fact, led to improvements to acquisition outcomes. Either way, outcomes are positive.
Unfortunately, outcomes don’t feel positive. Mind-blowing commercial innovations arrive every day, and our military adversaries adopt advancing technologies faster than ever. Meanwhile, the vast majority of U.S. companies shudder at the prospect of pursuing business opportunities with the Defense Department. This doesn’t just cause us to miss out on the latest widget; we’re missing out on the United States’ vast intellectual capital. The cost and timeline to implement advanced technology into DoD systems seems to grow perpetually, and every acquisition failure, endemic or not, reminds us of the limitations of our bureaucracy.
The Ugly Truth of Vanity Metrics
How do we square the very real data that show our acquisition system performing well with the very real observations that technologies in defense systems evolve glacially and, generally, at much greater cost than in industry? Some of the answer comes from how we measure and discuss acquisition performance. Mr. Kendall alludes to this in the AT&L Performance Report:
“In the case of defense acquisition, the primary outcome is the value of operational capabilities delivered in time for our warfighters to address threats. Unfortunately, it is very difficult to measure the final operational performance and value of our systems across systems and commodities.”
We should measure value delivered—what commercial industry would call Return on Investment (RoI)—but that’s too hard, so we focus on other proxy metrics. In contrast, commercial industry has found ways to measure actual value delivered and, in turn, measure and control the “acquisition” processes that deliver their products and services. As an enterprise, the Defense Department cannot objectively predict or measure how much utility it delivers, per dollar spent, per time it takes to deliver. Discussions in and out of the Pentagon definitely don’t address defense acquisitions in these terms.
Instead, the most trivial conversations about DoD acquisitions deal in raw dollar amounts and timelines to deliver systems. Huge dollar figures get thrown around with little quantifiable consideration for the utility those systems are intended to provide. More sophisticated conversations address cost and schedule growth against predetermined baselines instead of raw cost and time values. The most sophisticated conversations address cost and schedule growth and mention utility by equating it to predefined “Key Performance Parameters” (KPPs).
Indeed KPPs are important, and they are measurable—major Air Force programs, for example, are meeting their KPPs. It is also conceivable to evolve KPPs into a utility metric by directly correlating them to cost and schedule. However, establishing these types of interrelationships is difficult to accomplish and not required. Most congressional interest centers only on cost and schedule growth for major defense programs.
Yet, cost and schedule growth are less measures of enterprise-level acquisition system performance and more measures of individual contract performance. By focusing on secondary factors, we take for granted the intrinsic error in each prior and subsequent step of the acquisition process. To accept cost and schedule growth as meaningful enterprise-level metrics, we must accept prima facie that baseline cost estimates for each system are, in fact, the best costs achievable and that schedule estimates are the best achievable schedules for each system—not just the best achievable by the Defense Department but by any organization. We must also assume that capabilities delivered to our warfighters are, in fact, the best capabilities deliverable at the time they are delivered and that these capabilities will be the most affordable and reliable possible across the acquisition lifecycle. Only then would it be true that managing cost and schedule growth on contracts is tantamount to having an effective acquisition system.
Fundamental acquisition system improvements are likely hindered by this disconnect between what the Defense Department measures and the information necessary to inform enlightened change. Without measures of value that can objectively compare the utility, cost, and delivery time for multiple disparate products, services, and processes, the Defense Department, industry, Congress, and the public will continue to talk—subjectively—past each other. Indeed, we can and should have robust and open debate on all aspects of national defense, including our acquisition system. However, we will achieve real progress only with meaningful data.
Ending by Example
Our acquisition system performance measures and, similarly, the terms of our collective rhetoric around acquisition system performance are disconnected from the outcomes they intend to achieve. It takes little effort to observe how this spurs inefficiencies and frustration, but it’s useful to highlight examples.
In 2014, senior Air Force leaders launched the Bending the Cost Curve (BTCC) Initiative to address the perception that the Air Force, over time, pays more to receive less capability. Notwithstanding that this perception is more about a “value curve” than a “cost curve,” several Air Force and industry leaders took to saying it was not enough to “bend” the cost curve. They wanted to smash and obliterate it. Some skeptical industry representatives chose to describe the part of the human anatomy where cost curve would end up pointing. Ultimately, BTCC became a catch-all initiative for a variety of acquisition reform efforts and had several meaningful successes. As the Air Force official tasked with developing and executing BTCC from its early stages, my first inclination was to address a fundamental question: What is the current cost curve?
The short answer is that nobody knows. There are very few, if any, relevant longitudinal studies to be found in literature that correlate cost, capability, and delivery time for common classes of defense systems. In most cases, the Air Force (and, presumably, the entire Defense Department) does not have ready access to the data necessary to objectively assess whether the systems it is acquiring today are a good value compared to those being acquired 10, 20, or 30 years ago. Such data are not required of the Air Force, not demanded by the public, and, thus, not collected in any organized manner. We are left to our collective biases to draw conclusions, based on caricatures of our defense apparatus.
It is revealing that our distinction of acquisition categories (ACAT) is not principally based on the relative value or mission criticality but on how much money is expected to be spent—the most expensive programs are ACAT I, then ACAT II, then ACAT III. There isn’t even a definitive accounting of the number of ACAT II and III programs, let alone their performance under current metrics or their relative national security value. It is highly likely that some ACAT II and III programs have greater military value than existing ACAT I programs. Yet, the allocation of investment funds, high-quality personnel, and Internet chatter inherently skews towards ACAT I programs simply because of our focus on dollars spent over value delivered.
Aside from the philosophical shortcomings of our metrics and rhetoric, there can be notable real-world consequences that evade awareness. Last year, DoD evaluated the OCX program and found the contractor lacked modern software development tools and control processes. As a result, it took longer than it should to evaluate whether developed code increments functioned properly; developers did not employ key cybersecurity best practices; and the program fell well behind schedule and over budget. Based on outputs from these reviews, the Air Force and Raytheon are making changes. OCX is gradually developing into a good news story…but it’s an incomplete story.
Certainly, OCX-like stumbles jeopardize national defense. Yet, we should be equally concerned about what might have happened if the program, developed with substandard techniques, had delivered an archaic and proprietary system on budget and schedule. In that case, no alarms would have sounded, but lifecycle utility, cost, and reliability issues could have easily been worse than the cost and schedule challenges that have been faced so far. Airmen would have borne the brunt of an expensive lemon, while the acquisition system metrics would have registered a victory. This is more than a hypothetical; this scenario is a primary cause for the Defense Department’s ballooning sustainment expenses.
Many in the defense acquisition enterprise recognize these issues and are establishing more value-based decisionmaking tools. The Value Adjusted Total Evaluated Price (VATEP) source selection methodology is a great example that is finding growing use in aircraft and space acquisitions. But much more work is needed to make defense acquisition conversations, metrics, and processes less about managing contracts and more about managing outcomes.
Bottom Line at the Bottom
We must employ more relevant metrics of acquisition performance to get beyond our mundane and self-defeating rhetoric around defense spending. This is a challenging but not impossible task, and much of the groundwork has been laid. Along the way, let’s acknowledge that our nation’s undeniable military superiority is largely due to an unheralded and often maligned acquisition workforce that every day provides U.S. and allied warfighters around the world with the most sophisticated and effective weapon systems ever seen.
Camron Gorguinpour is a senior associate with the International Security Program at the Center for Strategic and International Studies in Washington, D.C., a Principal at Wōden, LLC, and former director of transformational innovation in the Office of the Assistant Secretary of the Air Force for Acquisitions.
Commentary is produced by the Center for Strategic and International Studies (CSIS), a private, tax-exempt institution focusing on international public policy issues. Its research is nonpartisan and nonproprietary. CSIS does not take specific policy positions. Accordingly, all views, positions, and conclusions expressed in this publication should be understood to be solely those of the author(s).
© 2017 by the Center for Strategic and International Studies. All rights reserved.