The New Metrics of Afghanistan
August 7, 2009
No one who works with the unclassified data on Afghanistan can fail to be aware of how poor and contradictory much of that data now are. In general, no NATO/ISDAF government – including the United States – has yet provided an honest or meaningful picture of the war. Far too often, official reporting has been tailored to report success when the Taliban, Hekmatyer, and Haqqani were actually scoring major gains. In other cases, key problems in the Afghan government, the NATO/ISAF effort, and the economic aid effort were ignored or disguised as successes.
In other cases, governments have simply reported metrics designed for bureaucratic purposes in other contexts, and which simply are not relevant to war fighting. Far too much economic reporting, for example, has ignored the real world poverty and needs of the Afghan people and reported classic econometric data. The bulk of reporting on aid has ignored the realities of warfighting except to excuse focusing aid on areas where it has little or no impact on areas with significant risk or outside the immediate perimeter of military protection. Most aid reporting has focused on funding and money spent and projects started, not on whether they meet the requirements of either broad national development or war fighting, or whether they have any meaningful or enduring effectiveness in actually serving the Afghan people.
CSIS has been tracking the data that are made available by NATO/ISAF, the US, other allied countries, the UN, available for several years. A survey of the key maps, graphics, and other data that are now provided is available on the CSIS web site at :
A review of these data reveals critical problems that call the integrity of most public Western reporting on the Afghan conflict into question. It also shows that clear needs exist for more objective reporting and measures of effectiveness:
Metrics That Show the True Nature of Taliban and Insurgent Successes
Virtually all NATO/ISAF and member country combat reporting has focused on military clashes between NATO/ISAF and insurgent forces. Until recently, almost all of this data has been tied to analyses that grossly minimize insurgent success by focusing on the limited number of districts (13 out of 364 in a recent US report) where there have been major clashes.
Yet, between 2004 and 2009, the insurgents generally avoided engaging NATO/ISAF forces unless they had a strong political motive to fight and concentrated on expanding their political and security control over the countryside. NATO/ISAF was winning largely meaningless tactical clashes without securing or “holding” population centers while the insurgents succeeded in locking NATO/ISAF and Afghan government efforts into steadily smaller areas. They took over much of southern Afghanistan; expanded their influence in the east, center, north, and west; steadily reduced the areas in which aid and PRTs could operate; and steadily won the war of political control that really counted.
For years, NATO/ISAF, the US, and other NATO/ISAF countries failed to provide unclassified data and maps showing this progress. They talked about victory measured solely in terms of the outcome of tactical clashes with insurgent forces from 2002 through much of 2008. Even today, government spokesmen talk about stalemates at a time the insurgency has scored more than half a decade of steady gains in winning the war it is seeking to fight: a battle of ideology, political influence and control in Afghanistan, and political attrition in undermining NATO/ISAF support for the war.
The only reporting that began to show what was actually happening was a series of UN maps that showed the massive expansion of insurgent influence in terms of risk to aid works. The UN reporting showed entire districts at risk when insurgent influence was often more limited, and this tended to exaggerate Taliban and other insurgent influence. It has generally been correct, however, in showing that it expanded from some kind of active presence somewhere close to 30 districts in 2003 to around 150 today – some 40% of the country and more than ten times the figures where NATO/ISAF reported on major attack incidents.
It is only now, after some eight years of warfare, that NATO/ISAF is truly beginning to shift its strategy to fight the war necessary to win, and is focusing on control of key population centers. This shape, clear, hold, and build strategy seeks to defeat the Taliban and insurgents in the war they have actually been fighting by:
- Shape: Create the military conditions necessary to secure key population centers; limit the flow of insurgents;
- Clear: Removing insurgent and anti-government elements from a given area or region, thereby creating space between the insurgents and the population;
- Hold: Maintaining security, denying the insurgents access and freedom of movement within the given space; and,
- Build: Exploiting the security space to deliver humanitarian relief and implement reconstruction and development initiatives that will connect the Afghan population to its government and build and sustain the Afghanistan envisioned in the strategic goals.
Yet, NATO/ISAF, the US, and other NATO/ISAF governments still provide zero unclassified reporting on the details of Taliban, Hekmatyer, and Haqqani control and influence in given areas of Afghanistan. Providing this kind of reporting is critical to any assessment of NATO/ISAF vs. insurgent success.
So is breaking out casualty data and significant attack data by key area of combat or threat activity, and providing maps that color code the level of insurgent control and clearly highlight gains and losses over time. Tying such data to satellite mapping of population density and terrain provides a far clearer picture of where insurgent activity is directly relevant in winning or losing the Afghan people. Much of this reporting effort already exists at various levels, and the reality is scarcely unknown to the threat. Making it public in general enough terms not to disclose tactical details is largely a matter of political embarrassment and hardly a matter of security.
Accordingly, the following metrics are critical to implementing a shape, clear, hold, and build strategy:
- Provide well-defined, topographically shaped, estimates of areas of insurgent influence and do this by province in ways that show whether the areas are expanding or contracting over the previous year. Also show areas where we lack clear data in grey. If possible, map polling results in some scale for support/opposition to insurgents.
- Map progress in shape, clear, hold, and build in the same depth. Get off the past emphasis on kinetics as the test of insurgent influence and apply the Abrams test from Vietnam: If you cannot go there in a normal vehicle, you are not yet at hold. If it isn't safe overnight, you are not yet at build. Again, use polling data where possible, rather than just our ratings.
- Show such data relative to population density in key areas that are the focus of our current strategy. These also are areas where NATO/ISAF and the US need to poll and map Afghan perceptions in detail. If we have a population-oriented strategy, we need simple maps to show its popular impact.
Reporting on the Insurgent Presence in Pakistan and the Patterns of Infiltration of Afghan and Foreign Fighters
The war has not been a purely Afghan conflict since early 2002. Al Qa’ida and Afghan insurgent groups, increasingly mixed with Pakistani Deobandi and “Taliban,” have established major sanctuaries and operational centers in the FATA and Baluchi areas of Pakistan. NATO/ISAF does not cover these developments. The war mysteriously has stopped at the Afghan border, and such reporting is inherently misleading and dishonest. No metrics than only cover Afghanistan can possibly be adequate.
These critical shortfalls in reporting have been compounded by the fact that elements of the Pakistani military and ISI have continued to support Al Qa’ida and Afghan Pashtun insurgents in both FATA and especially in Baluchistan (The Omar-led Taliban operates freely in the Quetta area). Metrics which do not show the known areas of Afghan insurgent operations in Pakistan, and which do not show the actual level of Pakistani military operations and border activity to deal with them, are equally inherently misleading and dishonest.
It is not enough to improve the metrics on the fighting in Afghanistan. These same metrics need to be applied in ways that show activity in the FATA and Baluchi areas of Pakistan and the entire real-world theater of combat.
Frank and Open Analysis of Afghan Successes versus Failures and Corruption.
A strategy of shape, clear, hold, and build must deal with the reality that the war is being lost as much through the corruption and incapacity of the Afghan central government as through any strengths on the part of the Taliban and other Jihadist insurgents. The war cannot be won where the Afghan government is not honest or present, and where NATO/ISAF and international aid efforts do not supplement or substitute for Afghan government corruption and incapacity.
Largely because of US influence in setting up Afghanistan’s present structure of government, Afghanistan has one of the most over-centralized structure of government in the world, and at the same time one of the least capable and most corrupt. Virtually all government funds flow through the control of the President and corrupt and incapable ministries to unelected provincial governments and district leaders – many of who are blatantly corrupt and incapable, and many of whom have ties to ex-warlords, narcotrafficers, corrupt powerbrokers, and sometimes to insurgents.
This level of corruption and incapacity is as much a threat as the Taliban and has done much to empower it by alienating the Afghan people and creating a de facto power vacuum.
These problems also threaten to cripple efforts to change NATO/ISAF strategy from a focus on warfighting or “kinetics” to shape, clear, hold, and build. It is only possible to hold and build where either the Afghan central government is present and honest or it is bypassed by NATO/ISAF and civil aid efforts that will work with local officials, the Afghan security forces, and respected local leaders.
They have long created a need for suitable metrics:
o Show which ministries are honest and effective and which are corrupt and incapable. What are the trends by key areas of capability? Suitable metrics in the form of tables and chants have been developed and used for many other countries.
o Show what provinces and districts in Afghanistan have honest and capable provincial and district governments and what areas have incapable and dishonest officials. These are easy to map and easy to name.
o Map quality of government activity, ROL, and corruption by district. Show where there is no effective government presence or the situation is unknown. Consider mapping provinces by district showing the quality of governance as polled.
- Map quality of governance in broad terms.
- Map level of corruption and impact of power brokers.
- Map quality of ROL as determined by availability of prompt and effective justice.
- Map areas of major narcotics growing and narco-trafficking/major impact from organized crime.
o Map GIRoA budget, aid flows, and impact. Map economic progress. Again, key metrics seem to be status of roads, electricity, water, irrigation, medical services, and education.
o Develop ratings for key population centers/cities in addition to provinces. NATO/ISAF is adopting a strategy based on population centers and not provinces. The rating system should focus at least as much on performance in the key areas of the campaign as provinces per se.
o Show the impact of aid in governance, ROL, and economics. Get both a rating based on official judgments and a summary score/rating based on polling. Here, it might be useful to rate what most Afghans seem to care most about: roads, electricity, water, irrigation, education, and medical. Add a question on perceived corruption and waste in the aid process as a control.
o Show how areas of insurgent influence relate to the effective presence of Afghan government officials, security forces, and functioning court systems and a rule of law. (This latter point is critical because the Taliban is now the functioning source of justice in many districts in the country, and often because the Western focus on a formal justice system has meant ignoring the informal justice system that is actually present while providing no meaningful services in many areas.) All of these are easy to map – and often have been mapped but without any circulation of the results.
o Show how NATO/ISAF and aid activities relate to areas of insurgent influence. The war is being fought a local level. It will be won or lost on the basis of whether there is a lasting security presence, and enough aid and government services to create jobs and success agriculture. Once again, it is easy to map where NASTO/ISAF and aid efforts actually support the hold and build phases and where they do not.
o Track aid disbursements and establish summary metrics for how a combination of NATO/ISAF, Afghan government, and aid efforts do and do not provide the key services Afghans want most: jobs and markets, basic security, prompt justice, water, roads, electricity, schools, and clinics.
Provide Data on the Integrity, Relevance and Effectiveness of International Aid.
US, UN, and international aid efforts are critical to successful war fighting, and to the hold and build phases of the new strategy. Afghanistan is not in a position to focus on longer term aid objectives until it is secure and stable. It needs aid to both win and to develop. Such aid, however, must be honest and effective and tied to valid requirements. In far too many cases, any valid metrics would show that the aid effort is currently as corrupt and ineffective as the Afghan government and does not serve either the immediate needs of counterinsurgency or valid, achievable goals for development.
There is an unforgivable lack of transparency and integrity in the US and international aid community, sometimes mixed with direct corruption. Organizations, countries, and NGOs do not report full financial data, do not validate requirements and programs by showing they have Afghan support and meet Afghan needs, do not comply with Ministry of Finance reporting requirements, or have useful measures of completion or effectiveness – particularly in terms of overall regional and national needs. This lack of reporting ensures that there are no remotely reliable numbers in many critical areas. However, Matt Waldman of Oxfam estimated in March 2008 (ACBAR Aid Effectiveness in Afghanistan, p. 5) that,
- There is an aid shortfall of $10 billion – equivalent to thirty times the annual national education budget: donors committed to give $25 billion in aid since 2001 but have only delivered $15billion.
- An estimated 40% of aid goes back to donor countries in corporate profits and consultant salaries – some $6 billion since 2001.
- Largely due to lack of coordination and communication, the Afghan government does not know how one-third of all aid since 2001 – some $5 billion – has been spent.
- The US military spends close to $100m a day in Afghanistan; yet the average volume of aid spent by all donors since 2001 is just $7million per day.
- Over half of aid is tied, requiring the procurement of donor-country goods and services.
- Over two-thirds of all aid bypasses the Afghan government.
- According to the latest OECD figures less than 40% of technical assistance is coordinated with the government and only one-third of donor analytical or assessment work is conducted jointly.
- Profit margins on reconstruction contracts for international and Afghan contractor companies are often 20% and can be as high as 50%.
- Most full time, expatriate consultants, working in private consulting companies, cost $250,000-$500,000 a year.
There are no quick answers to correcting these problems but there are some good ways to start:
o Insist that all aid reporting show that a valid requirement existed for the aid effort, the level of completion, and whether the project or program had sustained success.
o Require reporting that allows a direct tracking by Afghan population center and by key area for shape, clear, hold, and build operations.
o Blacklist contractors, Afghan officials and power brokers, and NGOs that have proven to be corrupt and/or failed to complete projects and ensure that they met Afghan requirements.
o List national and international aid efforts that do not comply with Afghan Ministry of Finance reporting requirements, do not publicly file annual statements and audits, and no not report on effectiveness.
Reshape Economic and Aid Reporting to Deal with the Realities of Shape, Clear, Hold, and Build.
There also is a need to provide far better reporting on the economics of Afghanistan and how aid impacts the insurgency and shape, clear, hold, and build. In most cases, governments and air organizations have reported metrics designed for bureaucratic purposes in other contexts, and which simply are not relevant to war fighting. Far too much economic reporting, for example, has ignored the real world poverty and needs of the Afghan people and reported classic econometric data.
Similarly, most reporting on aid efforts has ignored the realities of warfighting except to excuse focusing aid on areas where it has no little or impact on areas with significant risk or outside the immediate perimeter of military protection. Most aid reporting focusing on funding and money spent and projects started, not on whether they meet the requirements of either broad national development or war fighting, or whether they have any meaningful or enduring effectiveness in actually serving the Afghan people.
The war is being fought in a country torn by decades of conflict, abject poverty, massive urban refuge problems, intense demographic pressures, and chronic unemployment and under employment. There is little meaningful reporting on these realities and what is reported tends to be reported in national trends, rather than the problems and needs of specific areas, and in ways that show the level of economic stability and government services in high risk or conflict areas.
There is remarkably little focus on the needs of a population which is some 70% involved in agriculture, but faces major demographic pressure on the land from a young population some 2.5 times the population when the USSR invaded; years of drought and the destruction of much of the irrigation system; a lack of roads and local markets, and a need to rely on drug crops to survive. This is the population most at risk to the Taliban, and where hold and build will require immediate aid efforts. It is, however, virtually forgotten by many aid efforts or only affected by limited or pilot projects in secure areas or where PRTs have direct military protection.
A strategy of shape, clear, hold, and build cannot be based on aid efforts that ignore the basic economic and military realities of the country. Metrics needs to be designed to reflect the regional and local economics of the war, and to show whether and how well economics, aid, and governances really support the hold and build phases of shape, clear, hold, and build.
There is also a critical need for another kind of transparency and metric. Success or failure will depend heavily on the real world level of the unity of civil-military efforts in the field. The lack of unity of effort, and integrated civil-military effort, has better a critical factor in crippling the real world effectiveness of NATO/ISAF efforts since the start of the war, and one where member country governments have systematically avoided coming to grips with the real-world need for effective operations in the field.
It requires integrated civil-military campaign plans at the national and NATO/ISAF level. The past failures of governments, NATO/ISAF, and UNAMA have also shown that it requires public reporting that forces this level of unity on stovepiped and reluctant bureaucratic and military efforts, and measures of effectiveness that give as much emphasis to the civil aspects of hold and build as to the military efforts in shape and clear. While some details of military operations need to be kept secret, the need to develop and make a joint civil-military campaign plan requires significant public exposure. So does the reporting of tailored metrics on economics, employment, governance, and rule of law in each major populated area where shape, clear, hold, and build are being implemented.
Show Whether Adequate Resources Are Being Provided for an Effective Military and Civil Effort.
There are no magic force ratios or troop to task ratios that can be used to calculate how many NATO/ISAF and Afghan forces are needed to win in Afghanistan, what level of funding is needed, and what degree of unity of effort is required. It is clear, however, that the war will almost certainly be lost if NATO/ISAF countries do not provide the level of resources the theater commander wants, if Afghan forces are not built up to the levels where they can take over the hold mission and then the lead, and if national caveats and restrictions prevent effective unity of effort.
o Map where NATO/ISAF and Afghan are strong enough to implement shape, clear, hold and building; and to directly compare their areas of influence with those of the insurgents are the regional level down to key population centers and areas of military activity. The fact such net assessments have not been provided to date is not a function of their difficulty, but rather of the lack of integrity and transparency in NATO/ISAF, US, and allied government reporting. Basing metrics on military judgment scarcely presents a key problem when the entire war is based on military judgment.
o Show the size of total NATO/ISAF and Afghan force strength vs. estimated requirements.
o Provide similar summary estimates of actual US budgets vs. requirements, and here funding the civil side of the US Embassy and aid will be as important as funding US military forces.
o Honestly and publicly report on the lack of unity caused by national caveats and restrictions on individual NATO/ISAF forces and related Provincial Reconstruction Teams (PRTs) and aid efforts. These caveats and restitutions are a major problem in much of Afghanistan and one of the worst kept secrets that national governments and media have failed to address. Publicly listing such caveats and restrictions by country, summarizing their impact, and mapping the areas affected is as much key metric of effectiveness as providing similar data on the problems in Afghan government and security force activity.
o Expand current reporting of Afghan security force readiness from the “CM,” or Capability Milestone, system to mapping the areas where army and police forces are adequate to hold and build. Such estimates can be simple and straightforward. Using color-coding to show the relative level of capability is a simple adaptation of estimating types of metrics.
o Provide better data on the adequacy of funding for the development of Afghan forces, and show whether the required numbers of trainers, mentors, and partner units is adequate. These data are already provided in some reporting. Tying such reports to new force goals will provide key measures of effectiveness.
Measure Success in Terms of Afghan Perceptions
The war is essentially a war for Afghan perceptions. Far too often polling data are not made public or used in selective ways that disguise the full nature of Afghan views or produce positive results. There are a number of ways to correct this situation:
o Level of corruption and perceived corruption: Compare NATO/ISAF and US judgments with polling or focus group data showing Afghan views. This is one key to understanding Afghan perceptions of legitimacy. Excessive corruption and abuses by power brokers are so critical in interfering with the mission, and losing the support of the Afghan people, that NATO/ISAF and US should rate corruption and the impact of power brokers for each element of governance and the ANSF in each province and area.
o Quality of law enforcement and prompt justice. Again, show NATO/ISAF and US judgments of the status of the rule of law in every critical area, and poll or sample Afghan views. This is a critical area that clearly dominates Afghan perceptions and where the Taliban often scores gains.
o Perceptions of security with trend polling: Afghan perceptions of legitimacy and security are a key to tracking progress in shape, clear, hold, build. Implement a civil-military approach to creating a NATO/ISAF or US rating of progress in clear, hold, build, and some definitions of key polling questions to cover the topic in ways that show Afghan perceptions.
- A summary rating of Afghan perceptions of insurgent violence, including the lower levels of violence and intimidation that allow the insurgents to seek control over the population. The Significant Acts of Violence approach to reporting is close to useless in this regard.
- Summary ratings of Afghan perceptions of the US, UNAMA/Aid, and NATOL/ISAF.
- Comparisons of perceptions of threat posed by/violence from actions of NATO/ISAF vs. insurgents. NATO/ISAF may be the good guys but we need to know that Afghans see this.
o Perceptions on employment and economic well-being. Classic economics are fine for some purposes, but again, the focus of our strategy and actions has to be on winning popular support. The economic side is critical and there should be a break out of young male Afghans as a special category. Their perceptions of jobs and a stable economic future, and of their economic well being is critical to knowing the real operational climate and its impact on the people.
The Challenge of Complexity and Transparency
In some ways, this is a daunting list of reporting requirements. In practice, however, all of the resources required are already available. The problem is that they are being misused to produce metrics of little or negative value in uncoordinated ways that disguise their lack of military relevance and the lack of civil-military unity of effort. It is one thing to avoid reporting requirements that serve no operational purpose, it is another to avoid reporting requirements that show when resources are being wasted, how they can be better allocated, and contribute directly to winning a war.
The exercise in armed nation building in Afghanistan is also extremely complex. It cannot be analyzed simply using a few key indicators, particularly if analysis is to provide valid controls on what is actually happening in operational terms. Moreover, the Afghan War has raised fundamental questions in public policy about the integrity of western governments in reporting to their legislatures, their publics, and their media. Detailed public metrics are one way to help force integrity and efficiency on dysfunctional national and international efforts.
For related reports, please see:
The Afghanistan Campaign: Can We Win?
Winning in Afghanistan: Creating Effective Afghan Security Forces
The Dynamics of the "AfPak" Conflict: Metrics and Status Report