The UDHR Turns 70
As we celebrate Human Rights Day, commemorating 70 years since the signing of the Universal Declaration of Human Rights (UDHR), our team is reflecting, unsurprisingly, on data collection and analysis. Specifically, we’re thinking about how rigorous analysis can fortify debates about components of our criminal justice system such as cash bail, pretrial risk assessment and what “fairness” even means in these contexts.
One of the goals of the UDHR is to protect those who enter the criminal justice system. Consider, for example, UDHR articles 7, 9, and 11, and how they are implemented (or not) in the United States:
Article 7.
All are equal before the law and are entitled without any discrimination to equal protection of the law. All are entitled to equal protection against any discrimination in violation of this Declaration and against any incitement to such discrimination.
Article 9.
No one shall be subjected to arbitrary arrest, detention or exile.
Article 11.
(1) Everyone charged with a penal offence has the right to be presumed innocent until proved guilty according to law in a public trial at which he has had all the guarantees necessary for his defence.
(2) No one shall be held guilty of any penal offence on account of any act or omission which did not constitute a penal offence, under national or international law, at the time when it was committed. Nor shall a heavier penalty be imposed than the one that was applicable at the time the penal offence was committed.
Last year, Kristian Lum, Erwin Ma, and Mike Baiocchi used statistical tools appropriate for drawing causal (not just correlative) conclusions to find that “setting bail results in a 34% increase in the likelihood of conviction.” Perhaps most importantly, they concluded “[t]he real world implications of this are that there are likely many people– disproportionately, poor people– who have been convicted of crimes simply because bail was set.” This analysis, based on data collected by the New York Legal Aid Society, showed empirically the ways in which the cash bail system is violating Article 11 of the UDHR. The findings are being used to advocate for changes in the cash bail system in New York City.
Meanwhile, the use of cash bail was abolished by a new law in California, SB10. However, the law replaces bail with pretrial risk assessment, requiring the use of predictive algorithms which may introduce their own violations of the articles above. Again, statistical analysis can help us examine the affect of these models on different segments of the population. A recent article by Laurel Eckhouse, Kristian, Cynthia Conti-Cook, and Julie Ciccolini outlines the different layers of bias that need to be addressed in quantitative risk assessment.
More generally, our colleague Shira Mitchell and her co-authors Eric Potash and Solon Barocas have been outlining the different, and sometimes contradictory, ways to rigorously and mathematically define fairness. As conversations around risk assessment and other predictive models are beginning to center on issues of fairness, it’s critical that these discussions clarify which definition of fairness is being prioritized and why.
We welcome these conversations, and we are especially gratified to see participation and input across a variety of communities, including academia, advocacy groups, and most importantly, individuals who are directly affected by the implementation of these models. It is this last group that we believe more policy questions around predictive modeling should be focused on. Specifically, all of us, those collecting the training data and designing the models, proposing policies and regulations implementing models, and evaluating and critiquing the outcomes of those models, should spend more time asking, What is the cost of an incorrect prediction? And who bears that cost? As with many human rights advocacy efforts, rigorous data analysis can help guide the answers to these crucial questions.
We get excited by rigorous analysis, in part because it keeps us close to the worlds of advocacy and academia. But in the end, it is all done in the service of the people whom the UDHR aims to protect, and today is a very good reminder to keep them at the heart of our research.