Neve Gordon and Nitza Berkovitch recently argued that cross-national quantitative analysis of human rights is based on biased data. They claim that this data ignores significant sub-national variation, hides significant human rights problems in otherwise “highly scored” countries such as the United States, and suffers from methodological nationalism, or the fallacy of treating entire nation states as comparable “units of analysis”.
Ironically, these assertions are based on (1) over generalization from a very small sample of published work, (2) severe selection bias of the large and increasingly sophisticated work in this genre that tackles precisely the issues that they allege are problematic, and (3) ignorance of the work published by these same scholars that does not employ this particular method of analysis.
They assert that, “in sociology and political science most human rights scholars use cross-national datasets for their research”. It is not clear on what basis this assertion is made, since they present no statistical analysis of leading peer review journal or research monograph content. They claim that many of the scales used to measure human rights give scores that are too high for countries such as the United States, which ignore the disproportionate treatment of African Americans by the police and the historical patterns of disenfranchisement of black and ethnic minorities.
They are correct that many of these measures do not capture these kinds of problems in the US; however, they ignore the fact that scholars finding these measures wanting have either (1) sought to improve our understanding of how such coding takes place in an era of raised awareness about human rights problems, or (2) used different kinds of data precisely because errors at that higher level of analysis were glaringly obvious.
For example, the work of Chris Fariss addresses the problem of coding and has produced new and modified scores that actually downgrade the United States in ways that are consistent with the concerns Gordon and Berkovitch raise. Sakiko Fukada Parr has developed a social and economic rights fulfillment index, which she has used to show precisely how the United States is not performing in ways that it ought to, given its high level of economic development In 2000, Joe Foweraker and Roman Krznaric recognized this problem and published a study that looked at how we can “differentiate the democratic performance” of Western democracies, where they criticized available aggregate measures and then provided different sets of data that focused on sentencing, incarceration and the disenfranchisement of African Americans.
.jpg)
Flickr/Sebastian Sikora (Some rights reserved)
Human rights scholars are effectively conducting research at different levels of analysis using a mix of quantitative and qualitative methods.
Cross-national time-series analysis is based on methodological nationalism, but since the method has sufficient “degrees of freedom” (or a large number of observations), it can incorporate additional variables that control for things such as federal-unitary governmental institutional arrangements, presidential or parliamentary regimes, military or civilian rule, levels of domestic conflict, degree of ethnic fractionalization, patterns of land or income inequality, size of the population, per capita GDP, and other differentiating factors. Knowledge claims at this level of analysis focus on empirical generalisations and macro-social relationships, while the method itself can incorporate measures of subnational variation.
Inquiry at one level of analysis leads to curiosity and further research at another level of analysis.
But these same scholars are also conducting research at different levels of analysis using a mix of quantitative and qualitative methods. In my own work, for example, I have used cross-national time-series data analysis to examine the relationship between patterns in human rights treaty ratification and the protection of human rights, in which I am highly critical of the United States (and other countries). I have carried out small-n comparative analysis of social movements and citizenship rights using a combination of quantitative and qualitative approaches in the cases of Brazil, Chile, Mexico, and Spain. I have also worked on the single country cases of Mexico, Peru, Mongolia, Brazil, Chile, and Tunisia again using a mix of qualitative and quantitative methods.
Across this work, the choice of method is related to the research questions that are posed, and that inquiry at one level of analysis leads to curiosity and further research at another level of analysis. My PhD students have produced work that also varies across different methods of analysis or combines different levels of analysis to address challenging research questions. For example, Marco Larizza did a cross-national quantitative analysis of democratic performance and states capacity on all countries in the world, cross-national quantitative analysis of the same relationship across all of Latin America, and then a single-country analysis of Venezuela. Jaime Baeza Freer did a two-country longitudinal comparative analysis of elite dominance in Argentina and Brazil. Edzia Carvalho did a quantitative analysis of subnational variation of democracy and health in the federal system of India.
Indeed, two previous articles I’ve written for openGlobalRights were on (1) inequality and human rights abuse in developed countries like the US and (2) police violence and the need for a Truth Commission in the US. Both were underpinned by the use of quantitative analysis of subnational statistics on race, violence and structural inequalities.
Human rights scholars care deeply about real human rights abuse. They care about its documentation, its measurement and analysis, and the ways in which systematic research—at different levels of analysis using different methods—yields substantive inferences to advance human rights around the world. Gordon and Berkovitch missed the mark, and their argument that over reliance on a particular form of analysis perpetuates the abuse of human rights is simply fallacious.