Focus on Good Science, not Scientists
We recently learned about an article by Dr Nafeez Ahmed that criticizes the methods and conclusions of the Iraq Body Count (IBC) and the work of Professor Michael Spagat. Dr Ahmed cites our work extensively in support of his arguments, so we think it’s useful for us to reply.
We welcome Dr Ahmed’s summary of various points of scientific debate about mortality due to violence, specifically in Iraq and Colombia. We think these are very important questions for the analysis of data about violent conflict, and indeed, about data analysis more generally. We appreciate his exploration of the technical nuances of this difficult field.
Unfortunately, parts of Dr Ahmed’s article focus on sources of funding that IBC and Professor Spagat have received, and on speculation about how such funding might affect their substantive conclusions. We find such criticism to distract from the important points of scientific debate.
Policy arguments that draw on scientific findings will inevitably include scientists who have personal opinions and political associations. Those personal opinions often influence which areas we choose to research. The challenge for all scientists is to do rigorous, reviewable, transparent work that supports or rebuts existing theories — or advances a new theory — about how the world works. Such work should be done without the scientists working in fear of personal attacks.
Good scientific work does not depend on who did the science. Every scientific finding must be assessed by the quality and appropriateness of the data and methods, and their relevance to the theory being tested. Science depends on anonymous peer review precisely so that the reviewers are not biased by who the authors might be. The personalities and biographies of the scientists involved must be irrelevant to the quality and veracity of the scientific result. Similarly, science is tested by replication: given a finding, another scientist should be able to use the same data and method to reach the same result. Anonymous review and replication are fundamental to science precisely in order to distinguish scientific knowledge from the scientists who do the work.
In particular, we think the article’s criticism of the United States Institute of Peace (USIP) is excessive and inaccurate. Our colleague Professor Amelia Hoover Green, the lead author of our white paper criticizing the Dirty War Index (DWI) measure cited extensively in Dr Ahmed’s article, has been supported by USIP grant funding. Our sister project, Martus, has received funding from USIP. Many other academics whose work is deeply critical of U.S. policy, Colombian policy, and the academic proponents thereof have received USIP support. (For full disclosure, HRDAG’s current and past funders are listed here.)
HRDAG has many times disagreed with Professor Spagat and IBC, and at other times, we have agreed. HRDAG and Professor Spagat have had productive debates, and we expect to have more with him in the future, because that is how scientific research works and advances. We appreciate that Professor Spagat and IBC have been forthcoming with information and data when HRDAG has requested it. This is noteworthy because Professor Spagat and IBC are fully aware that we are likely to disagree with them.
Lastly, HRDAG would like to correct two of Dr. Ahmed’s points. He repeatedly asserts that IBC might be missing many large events (e.g., massacres). HRDAG’s analysis, and a 2008 RAND study cited by Dr Ahmed, leads HRDAG to conclude that IBC is likely missing many smaller events with fewer victims, while missing relatively few large events. Further, Dr Ahmed refers to HRDAG as based in Los Angeles. Although our fiscal sponsor, Community Partners, is based in Los Angeles, we are happy to report that we are based in San Francisco.