Back to the "Body Count:" The Lack of Reliable Data on the Wars in Iraq, Afghanistan, and Pakistan
September 11, 2012
There is no easy way to put war in perspective, and there are many different ways to measures trends and intensity. As long as one only uses one source, there seem to be useful data on the levels of combat and terrorist activity in Afghanistan, and valid comparisons with the similar combat and terrorist activity in Iraq and Pakistan. The moment one states comparing official NCTC, DoD, ISAF, and UN estimates, however, is becomes obvious that no clearly reliable data exist.
These problems are analyzed in detail, along with the credibility of current official reporting on the Afghan conflict, in a new report by the Burke Chair at CSIS. It is entitled Back to the “Body Count:” The Lack of Reliable Data on the Wars in Iraq, Afghanistan, and Pakistan. It is available on the CSIS web site at https://csis-website-prod.s3.amazonaws.com/s3fs-public/legacy_files/files/publication/120910_Iraq_Afghan_Terrorism.pdf.
- Chart One shows that the NCTC estimates that Afghanistan ranked highest in terms of global terrorism in 2011 in both the number of attacks (2,872) and deaths (3,353), followed by Iraq and Pakistan.
It should be noted, however, that the Afghan totals for attacks and deaths in 2011 – the peak year so far in the Afghan war – were only marginally smaller than the totals for Iraq (2,265 attacks and 3,063 dead) – a nation where the war is supposed to have been won. Pakistan ranked third with about half the number of attacks (1,436) and roughly than two-third of the deaths (2,033) that took place in Afghanistan and Iraq.
The problem is, as the following comparisons show, there is no way to no what these number really mean or estimate their reliability.
- Chart Two shows the NCTC estimate of the number of kidnappings It shows the dangers of relying on counts from even a single source. The patterns in kidnappings followed very different patterns from the patterns in attacks and deaths. Somalia led with 2,527. Afghanistan was next with 902, about one-third of the Somalia total. Pakistan had 430, and Iraq had 111. This illustrates one of many studies showing that deaths alone do not provide a meaningful metric of the violence in combat, particularly in political wars like insurgencies.
- Chart Three begins to provide a baseline for showing the problems in today’s equivalent of the “body counts” in Vietnam. It shows that the NCTC estimates that the Iraq War produced far more attacks and deaths at its peak than Afghanistan – with roughly six times as many in Iraq 2007 as Afghanistan had in 2011. As noted earlier, however, “peace” in Iraq in 2011 was close to being as violent as war in Afghanistan in both total numbers and relative to total population.
As becomes clear from the Charts that follow, however, other sources for Afghanistan raise critical questions as to what the NCTC data mean and their utility.
- Chart Four shows the Department of Defense estimate of the total number of security incidents in Afghanistan in 2009-2011. It illustrates just how radically different counts of combat activity can be. If the DoD counts of security incidents are compared with the NCTC counts in Chart Three, the monthly counts in the DoD data for 2010 and 2011 often equally or exceeded the total annual counts for the NCTC. Unfortunately, neither count is defined well enough to explain the differences.
- Chart Five shows the latest ISAF counts of enemy initiated attacks. These are notably lower than the totals for 2010 and 2011, and could track with the DoD data – although the different ways DoD and ISAF count IED attacks makes this highly uncertain.
The ISAF counts of enemy initiated attacks should be closer to the counts the NCTC makes of terrorism – which seems to be applied to most insurgent combat activity and not an actual count of terrorism. Once again, however, there is no comparison. The ISAF monthly count of enemy initiated attacks during 2010 and 2011 often equals or exceeds the NCTC count for the entire year.
- Chart Six shows the latest ISAF count of IED explosions – not total IED activity – and mine strikes. The patterns for 2010 and 2011, and for the first half of 2011 do not show the same trends as total number of security incidents or enemy initiated attacks. This again illustrates why one set of metrics is not a meaningful measure of combat activity or intensity, and can be used to spin a level of success that may not actually exist. This is particularly true if the insurgency is focusing on the political dimension of war and limited direct engagement with its enemy.
- Chart Seven shows the UN count of total security incidents in Afghanistan in 2009, 2010, and 2011. The counts compare as follows: For 2009, the NCTC count of attacks is 2,124 and the UN count is 11,524 – or 443 percent higher. For 2010, the NCTC count of attacks is 3,346 and the UN count is 19,403 – or 480 percent higher. For 2011, the NCTC count of attacks is 2,872 and the UN count is 22,903 – or 697 percent higher.
The UN counts could conceivably track with the DoD and ISAF counts, but there is no way to tell. They cannot conceivably track with the NCTC counts. The NCTC attack counts are also the only ones showing a peak in 2010, and comparisons with later counts of deaths and injured make it even harder to see any consistent patterns or possible explanations. These may exist, but the is no way any user of the information can know this, or what any given source and count really means.
- Charts Eight and Nine show the ISAF counts of civilian deaths in Afghanistan. The total ISAF count of insurgent-inflicted deaths for 2011 in Chart Nine is compared with the annual NCTC estimate for 2011 in Chart Three. Chart Three shows the 2011 NCTC estimate of terrorism related deaths to be 3,353, and the ISAF Charts Eight and Nine report a count of 3,150 civilian deaths in 2011 in Afghanistan.
- Chart Ten shows the UN count of total civilian deaths in Afghanistan by year. The UN figure for total deaths in 2011 is 3,021. The ISAF figure for insurgent-caused deaths is approximately 3,150 -- when it should be much lower if the UN and ISAF counts were comparable. The NCTC count is 2033. Once again, there is no correlation between the differences in counts of attacks in each source and the differences in counts of deaths.
- Chart Eleven shows the NCTC count of attacks, attacks producing at least one major act of violence, and people killed, injured, and kidnapped as a result of terrorism. The NCTC data illustrate a key point constantly forgotten in official US war reporting, and most media. The number of dead is not a useful measure of the intensity of fighting or the number of people who suffer, and is of little value in measuring the impact of a political struggle like an insurgency.
The totals for people killed, injured, and kidnapped are far higher than for killed, and do not correlate closely to the trends in deaths shown in earlier charts. Moreover, Iraq is notably more violent than Afghanistan in 2011. Iraq is 33% higher in killed, injured, and kidnapped while Afghanistan is 9% higher in deaths.
- Chart Twelve reinforces this point. The UN count of deaths and injuries is also far higher than of deaths alone and reveals sharply different patterns in cause. Once again, however, the comparisons between the UN data and NCTC data are virtually inexplicable The NCTC totals for people killed, injured, and kidnapped cover the entire year, while the UN counts for civilian deaths and injuries IEDs, target killings, and suicide/complex attacks only cover six months.
The NCTC total for 2009, however, is 7,588 for the entire year, and the UN total is 1,895 for the first six months. The NCTC total for 2010 is 9,035, and the UN count for the first six months is only 2,318. The NCTC total for 2012 2011 is 9,171, and the UN count for the first six months is only 2,009. Even if one more than doubles the UN count to allow for the differences in the campaign season, , there is no way the NCTC and UN counts can be remotely comparable.
- Chart Thirteen shows latest the UN estimate of who is responsible for civilian deaths. These do still show that the insurgents are clearly responsible for 80%. And since most “unknown” are either insurgents or the result of local fights, the US and ISAF are unlikely to be much higher than 10-12% in 2012, versus around 15% in 2010. Once again, however, there is no consensus or transparency, and the methodology seems likely to grossly overstate US and ISAF responsibility for Afghan civilian suffering since the insurgents produce far more wounded with IEDs and their attack methods.
It must be stressed that these problems do not show that any of the different sets of counts did not come out of not valid efforts to try to measure the intensity and trends in the fighting. It is obvious, however, that none are properly explained, and that perceptible effort has been to examine the methods and counts in other sources in depth. The data also warn that counts of given types of attacks alone, or that focus on deaths rather than the overall impact of combat and terrorism, are inherently too limited to be valid metrics for summarizing key trends.