New report on recording casualties of drone strikes

First published 2nd Nov 2012

The Human Rights Clinic at Columbia Law School in the United States has done a cross comparison research of organisations who have been recording compiling and publishing casualties of U.S drone strikes in Pakistan. The  report examined the methodology of three organisations whose casualties figures are the among the most quoted figures in the debate about the effectiveness and humanitarian costs of drones. 

The report evaluated the recording methodology of the Long War Journal, the New America Foundation and the Bureau of Investigative Journalism (the latter a member of the International Practitioners Network) according to standards which the authors of the report have identified and explained. The three organisations studied have widely varying casualty figures. For instance for the 2011 period, the New America Foundation recorded between 3 to 9 civilian deaths, the Long War Journal recorded 30 civilians had died and the Bureau of Investigative Journalism reported between 68 to 157 civilian casualties. Such a stark difference in casualty figures demonstrates the difficulty in trying to obtain reliable information on the use of drones by the U.S in Pakistan and other conflicts. This report, however, contributes a great deal to the understanding of how such figures are complied at determined, which provides an important backdrop for commentators and others in this debate

The authors of the study also did their own recount of reported casualties  using  media sources and other sources available on U.S drone strikes in Pakistan for 2011. The recount done by the Human Rights Clinic at Columbia Law had significant discrepancies in the civilian casualties count with the New America Foundation and the Long War Journal and was significantly higher than the latter two organisation. In fact the report's recount of civilian deaths was much more similar to the Bureau's count with the report counting only 5.9% more minimum civilian casualties.

The principal reasons the study identifies for the discrepancies in casualty figures between organisations is:

  • The Bureau of Investigative Journalism's use of non-media sources; 
  • The New America Foundation and Long War's Journal decision not to update figures in some cases where media sources change their casualty counts; 
  • The organisations decisions not to include some data from their own listed sources.

Another reason which the study identifies as a limitation of the media, but can also explain the discrepancy in casualty counts, particularly of civilians, is that there is no standard definition of media sources  qualifying a person as a militant or a civilian. It is not clear how media sources determine this information and in-turn it becomes more difficult for the recording organisations themselves to make that call.

Finally, the authors of the study are concerned about and warn against the over reliance on recording organisations who use media reports as their sources and then subsequently are cited by the same media. The authors point to methodological flaws of these organisations which can be fixed but also to inherent flaws when using media sources. The biggest one being limited access by journalists and others, in the particular region where drone strikes are occurring  Although the authors argue that "the US government has the responsibility to step in and describe its own accounting of civilian casualties," which is critical, we believe that such recording organisations, who use a transparent and sound methodology (as identified by the study), provide important information that would otherwise be absent from the debate and can provide the government a basis on which to investigate civilian casualties.  

The full report can be found here