Measuring Human Rights (33): Measuring Racial Discrimination

The measurement of racial discrimination may seem like a purely technical topic, but in reality it comes with a huge moral dilemma. In order to measure racial discrimination, you have to categorize people into different racial groups (usually in your national census). On the basis of this you can then collect social information about those groups, and compare the average outcomes in order to detect large discrepancies between them. For example, do blacks in the US earn less, achieve less in school etc. Only then can you assume that there may be racism or discrimination and can you design policies that deal with it.

Now, categorizing people into different racial groups is not straightforward. You need to do violence to reality. Racial classifications and categorizations are not simply a reflection of factual reality, of “real group identities”. Instead they are social constructions or even fantasies influenced by centuries of prejudice, stereotypes and power relations. If we want to use racial classifications to measure discrimination, then we give people labels that may have little or nothing to do with who or what they are and how they identify themselves. Instead, these labels perpetuate the stereotypes and power relations that were the basis of the racial classifications when they were first conceived centuries ago. For example, “black” or “African-American” is not a simple descriptive label of a well-defined and existing group of people; instead it’s an ideological construction that was once used to segregate certain groups of very different people and subordinate them to a lower station in life. (Evidence for the claim that race is a social construct rather than a natural fact can be found in biology and in the fact that racial classifications differ wildly from one country to another).

In other words, the “statistical representation of diversity is a complex process which reveals the foundations of societies and their political choices” (source). In this particular case, the foundation of society was racism and the political choices were segregation and discrimination. If today we use the same racial and ethnic classifications that were once used to justify segregation and discrimination, then we run the risk of perpetuating racist social constructions. As a result, we may also help to perpetuate stereotypes and discrimination, even as we try to go in the opposite direction. It’s a form of path dependence.

Statistics are not just a reflection of social reality, but also affect this reality. Statistical categories are supposed to describe social groups, but at the same time they may influence people’s attitudes towards those groups because they contain memories of older judgments that were once attached to those groups. The dilemma is the following: the use of racial classifications to measure discrimination means giving people labels that have little or nothing to do with who they are or what they are; but they have something to do with how others treat them. It’s this treatment that we want to measure, and we can’t do so without the use of classifications. Using such classifications, however, can help to perpetuate the treatment we want to measure and avoid.

More posts in this series are here.

Advertisements

Measuring Human Rights (32): Assessing Advocacy and Policy by Way of Counterfactual Thinking

Human rights measurement is ultimately about levels of respect for human rights, but it can also be useful to try to measure the impact of human rights advocacy and policy on these levels. Both advocacy and policy (the difference being that the former is non-governmental) aim at improving levels of respect for human rights. Obviously, those levels don’t depend solely on advocacy or policy, but it’s reasonably to assume that they are to some extent dependent on those types of action. It’s hard – although not impossible – to imagine that millions of people and dozens of governments and international institutions would engage in pointless activity.

The question is then: to what extent exactly? How much do advocacy and policy help? The problem in answering this question is that we won’t necessarily learn a lot by simply looking at the levels and how they evolve. Not only is there the difficulty of comparing different possible causes; a flat trend line – or even a declining trend line – may cover up how much more awful things would have been without advocacy and policy. Levels of respect may very well stay as they are or even worsen while advocacy and policy are relatively successful because the levels without advocacy and policy would have been even lower.

Of course, it’s very hard to quantify this. If there’s improvement, you can at least try to sort out the relative contribution of different causes. If things don’t improve or even worsen, then the only way to measure the effect of advocacy and policy is the use of counterfactual thinking. And that’s a problem. How bad (or good?) would things have been without advocacy and policy? We can’t redo a part of a country’s history to test what would have happened with other choices. We can speculate about the answer to “what if” questions but since we can’t experiment we’re left with a lot of uncertainty. What if Hitler had won the war? Or had been admitted to art school? Fun questions to try and answer, but the answers won’t tell us much about the real world, unfortunately. If they did, we would know what to do.

More posts in this series are here.

Measuring Human Rights (31): Which Changes in the Spatial Pattern of Human Rights Are Most Likely?

One result of human rights measurement is a spatial pattern of human rights, a pattern that of course changes over time: countries with lower or higher levels of respect for human rights show up on a world map and this world map shows a certain spatial pattern.

The current spatial pattern of human rights is, somewhat simplistically, like this: wealthy and developed “Western” countries, although by no means free from human rights violations, show on average higher levels of respect for human rights than most developing nations. This is no reason to distribute praise or blame: developed countries share responsibility for human rights violations in developing countries, and high levels of respect for certain human rights in developed countries may be partly a matter of luck or perhaps even the direct consequence of the exploitation of developing regions. It’s also the case that rights cost money, hence wealthier countries can be expected to show higher levels of respect for rights.

Just take it as a fact rather than a judgment, admittedly a stylized fact (one can argue that human rights are better protected in Italy than in the US even though the latter is much wealthier; the same is true if you compare Botswana en China). Here‘s an example of one human rights index that confirms this spatial pattern.

Given this current spatial pattern, what’s our best guess about the future? The dynamics of human rights are poorly understood: unfortunately, we don’t really know which actions or events are most likely to change levels of respect for human rights, at least not in the positive sense. We know that war, genocide, authoritarian rule and poverty bring levels down, but we don’t know quite as well how to bring levels up. We assume that different types of forces may play a role:

  • bottom-up forces such as popular revolts, changes in cultural practice etc.;
  • top-down forces such as coups d’états, government policies, national legislation, international law, international institutions etc.;
  • horizontal forces such as peer pressure among states, conditional bilateral development aid, pay-offs, military intervention, naming-and-shaming etc.

Incentives also play a role, and maybe even forces beyond human control such as climate, geography etc. However, the exact result and impact of these forces is unclear and controversial, so we don’t really know what to do and kinda grope in the dark hoping something is successful.

Given the fact that many people and many institutions actually try to do something in order to raise levels of respect for human rights, it’s indeed likely that some actions will be somewhat effective. Hence the spatial pattern of human rights may change in the future. Here are my guesses as to how it may change:

  1. Those areas of the world where respect for rights is already relatively high are most likely to see additional improvements. I agree that low hanging fruit is easiest to pick, and that is why we may see spectacular progress in some countries where respect is currently low: the removal of an oppressive regime can, in theory, bring rapid and large improvements in levels of respect, but in practice there are very few cases (often the overthrow of an oppressive regime is followed by civil war or a successor regime that is only slightly better or even worse). Conversely, sometimes high hanging fruit is, paradoxically, easier to pick. Countries with a reasonably high level of respect often have a history of struggle for rights as well as a culture of rights resulting from that struggle. Rights are part of the ethos of the common man. Remaining rights violations will therefore be more jarring, and existing institutions necessary to tackle them are in place. Another reason to believe that improvements in human rights will first take place in those countries that are already relatively good is the dynamic of bilateral aid: aid donors are likely to give more to countries that already have a certain level of respect, not just because donors like aid conditionality but also because of things such as the “bottomless pit syndrome”. Badly governed countries just take the aid and spend it for the rulers’ personal profit. Donors understandably don’t like this and therefore tend to give to countries that are better governed.
  2. Those areas of the world adjacent to areas where respect for rights is already relatively high are likely to see additional improvements. Countries tend to see rights violations in neighboring countries as more urgent than rights violations far away. The former violations can have spillover effects: a civil war in the country next door can cause refugee flows into your own country or other types of spillovers, hence you have an incentive to do something about the war. The same is true for other types of rights violations. Rights violations in a country far away don’t create the same incentives to act. Additionally, the EU and other regional organizations insist that candidate member countries – almost always adjacent countries – first respect human rights before they can become members. These candidate countries therefore have a powerful incentive to raise levels of respect, since membership is often profitable. And there are also other, non-spatial types of proximity among adjacent countries: they may share a language – or their languages may belong to the same family – or a religion. This kind of cultural proximity makes bilateral intervention more likely and more acceptable. If one of two adjacent countries has a high level of respect for human rights, it may find it easier to intervene in the other country in order to foster human rights. It may offer effective institutional assistance for instance, assistance that is more effective – because more acceptable and easier – than assistance from a country far away, “far away” both spatially and culturally. Another reason to believe that proximity plays a role: a country that exists in the proximity of other countries that perform better in the field of human rights is in direct competition with those other countries; competition for workers, international investment etc. Both workers and companies will prefer to invest in countries that are free. Hence the underperformers in a certain region will have the incentive to do better.

If these two claims are correct, then we’ll see increasing polarization among two groups of countries. Not the optimal outcome, but perhaps the most likely one. Time will tell.

More posts in this series are here.

Measuring Human Rights (30): Distortions Caused by the Exclusion of Prisoners

I’ve already cited one example of human rights measurement gone wrong because of the exclusion of the prison inmate population: violent crime rates seem to go down in many countries, but a lot of the decrease only happens because surveys and databases exclude the crimes that take place inside of prisons. Crime may not have gone down at all; perhaps a lot of it has just been moved to the prisons.

I’ll now add a few other examples of distortions in human rights measurement caused by the exclusion of the prisoner population. The cases I’ll cite result in distortions because the exclusion of the prison population is the exclusion of a non-representative sample of the total population. For example, it’s well-known that African-Americans make up a disproportionate share of the inmate population in the U.S. Becky Pettit, a University of Washington sociologist, argues in her book “Invisible Men” that we shouldn’t take for granted some of the indicators of black progress in the U.S.:

For example, without adjusting for prisoners, the high-school completion gap between white and black men has fallen by more than 50% since 1980 … After adjusting … the gap has barely closed and has been constant since the late 1980s. (source)

We see similar results when counting or better recounting voter turnout numbers, employment rates etc.

It should be rather easy to include prisoners in most of these measurements – certainly compared to the homeless, illegal immigrants and citizens of dictatorships. The fact that we almost systematically exclude them is testimony to our attitude towards prisoners: they are excluded from society, and they literally don’t count.

More posts in this series are here.

Measuring Human Rights (29): When More Means Less, and Vice Versa, Ctd.

Take the example of rape measurement: better statistical and reporting methods used by the police, combined with less social stigma and other factors result in statistics showing a rising number of rapes, but this increase is due to the measurement methods and other effects, not to what happened in real life. The actual number of rapes may have gone down.

This is a general problem in human rights measurement: more often means less, and vice versa. The nature of the thing we’re trying to measure – human rights violations – means that the more there is, the more difficult it is to measure; and the more difficult, the more likely that we wrongly conclude that there is less. (See here). When levels of rights violations approach totalitarianism, people won’t report, won’t dare to speak, or won’t be able to speak. It’s not social stigma or shame that prevents them from speaking, as in the case of rape, but fear. Furthermore, totalitarian governments won’t allow monitoring, and will have managed to some extent to indoctrinate their citizens. Finally, the state of the economy won’t allow for easy transport and communication, given the correlation between economic underdevelopment and authoritarian government.

Conversely, higher levels of respect for human rights will yield statistics showing more rights violations, because a certain level of respect for human rights makes monitoring easier.

More on measuring human rights.

Measuring Human Rights (27): Measuring Crime

A number of crimes are also human rights violations, so crime rates can tell us something about the degree of respect for human rights. Unfortunately, as in most cases of rights measurement, crime measurement is difficult. I won’t discuss the usual difficulties here – underreporting by victims or relatives, lack of evidence, corrupt or inefficient police departments etc. Instead, I want to mention one particularly interesting problem that is seldom mentioned but possibly fatal for crime rate statistics: most reductions in crime rates are not really reductions, especially not those reductions that come about as a result of tougher law enforcement and higher incarceration rates. When we imprison criminals, rather than bringing crimes rates down, we just move the crime from society towards the prisons:

the figures that suggest that violence has been disappearing in the United States contain a blind spot so large that to cite them uncritically, as the major papers do, is to collude in an epic con. Uncounted in the official tallies are the hundreds of thousands of crimes that take place in the country’s prison system, a vast and growing residential network whose forsaken tenants increasingly bear the brunt of America’s propensity for anger and violence.

Crime has not fallen in the United States—it’s been shifted. Just as Wall Street connived with regulators to transfer financial risk from spendthrift banks to careless home buyers, so have federal, state, and local legislatures succeeded in rerouting criminal risk away from urban centers and concentrating it in a proliferating web of hyperhells. (source, source)

And there’s no way to correct for this and adjust overall crime rate statistics because quality statistics on crime rates inside prison are even harder to get than statistics on “normal” crime rates – given the quasi lawlessness of prison life.

More on prison violence here and here.

Measuring Human Rights (26): Measuring Murder

Murder should be easy to measure. Unlike many other crimes or rights violations, the evidence is clear and painstakingly recorded: there is a body, at least in most cases; police seldom fail to notice a murder; and relatives or friends of the victim rarely fail to report the crime. So even if we are not always able to find and punish murderers, we should at least know how many murders there are.

And yet, even this most obvious of crimes can be hard to measure. In poorer countries, police departments may not have the means necessary to record homicides correctly and completely. Families may be weary of reporting homicides for fear of corrupt police officers entering their homes and using the occasion to extort bribes. Civil wars make it difficult to collect any data, including crime data. During wartime, homicides may not be distinguishable from casualties of the war.

And there’s more. Police departments in violent places may be under pressure to bring down crime stats and may manipulate the data as a result: moving some dubious murder cases to categories such as “accidents”, “manslaughter”, “suicide” etc.

Homicides usually take place in cities, hence the temptation to rank cities according to homicide rates. But cities differ in the way they determine their borders: suburbs may be included or not, or partially, and this affects homicide rates since suburbs tend to be less violent. Some cities have more visitors than other cities (more commuters, tourists, business trips) and visitors are usually not counted as “population” while they may also be at risk of murder.

In addition, some ideologies may cause distortions in the data. Does abortion count as murder? Honor killings? Euthanasia and  assisted suicide? Laws and opinions about all this vary between jurisdictions and introduce biases in country comparisons.

And, finally, countries with lower murder rates may not be less violent; they may just have better emergency healthcare systems allowing them to save potential murder victims.

So, if even the most obvious of human rights violations is difficult to measure, you can guess the quality of other indicators.

Measuring Human Rights (25): Measuring Hunger

First, and for those in doubt: hunger is a human rights violations (see article 25 of the Universal Declaration). Second, before we discuss ways to measure this violation, we have to know what it is that we want to measure. It’s surprisingly difficult to define hunger.

Definition of hunger

The word “hunger” in this context does not refer to the subjective sensation that we have when lunch is late. We’re talking here about a chronic lack of food or a sudden and catastrophic lack of food (as in the case of a famine). We measure a lack of food by measuring dietary energy deficiency, which in turn is computed based on average daily calorie intake. The FAO estimates that the average minimum energy requirement per person is 1800 kcal per day. The global average per capita daily calorie intake is currently about 2800 kcal. This average obviously masks extreme differences between the obese and the chronically undernourished.

The FAO minimum energy requirement per person of 1800 kcal is also an average. The minimum calorie need depends on many things: age, climate, health, height, occupation etc.

Usually, the concept of “hunger” as it is defined here is different from “malnutrition“. Hunger is a lack of food defined as a lack of calorie intake. Malnutrition is a lack of quality food, of micronutrients such as vitamins and minerals, and of a divers diet. Hence, people may have access to sufficient quantities of food and still be malnourished.

Hunger and famine are also different concepts. Hunger is a chronic and creeping lack of food, while a famine results from the sudden collapse of food stocks. A famine implies widespread starvation during a limited period. It can’t go on forever because it must stop when everyone has died or when food supplies are restored. Chronic hunger on the other hand can go on forever because it doesn’t imply widespread starvation. Of course, people do die of chronic hunger, and on a global level hunger kills more people than famines do. But whereas in the case of famine people die of starvation, the victims of chronic hunger usually don’t starve to death. When we say that hunger kills someone every 3.6 seconds we usually mean that this person dies from an infectious disease brought on by hunger. Hunger increases people’s vulnerability to diseases which are otherwise nonfatal (e.g. diarrhea, pneumonia etc.). In fact, most hunger related deaths do not occur during famines. Chronic hunger is much more deadly – it’s just not as noticeable as a famine. When and where famines occur, they are more deadly and catastrophic. But they occur, thank God, only exceptionally. Hunger on the other hand is a permanent fixture of the lives of millions and ubiquitous in many countries.

Measurement of hunger

Given this definition, how do we go about and measure the extent of chronic hunger? (The measurement of famine is a separate problem, discussed here). There are different possible methods:

  • So-called food intake surveys (FIS) estimate dietary intake and try to relate this to energy needs determined by physical activity. Calorie intake below a minimum level means hunger. The problem here is that minimum calorie intake thresholds are somewhat arbitrary and do not always take people’s different calorie requirements into account. Even for a single individual, this threshold can vary over time (depending on the climate, the individual’s age, occupation and health etc.). Moreover, when trying to measure calorie intake, you’re faced with the problem of hunger due to imperfect absorption: it’s not because someone in a sample buys and consumes x number of calories that he or she actually absorbs those calories. The widespread incidence of diarrhea and other health problems often mean that only a fraction of calories eaten are absorbed by the body.
  • In order to bypass this, some propose a measurement method based on revealed preferences. The greater the share of calories people receive from the cheapest foods available to them, the hungrier they are; and, conversely, the more they buy expensive sources of calories, the less hungry they are. Their choice of foods reveals whether they have enough calories. This method therefore eliminates the threshold and absorption problems.

Our approach derives from the fact that when a person is below their nutrition threshold, there is a large utility penalty due to the physical discomfort associated with the body’s physiological and biochemical reaction to insufficient nutrition. At this stage, the marginal utility of calories is extremely high, so a utility-maximizing consumer will largely choose foods that are the cheapest available source of calories, typically a staple like cassava, rice or wheat. However, once they have passed subsistence, the marginal utility of calories declines significantly and they will begin to substitute towards foods that are more expensive sources of calories but that have higher levels of non-nutritional attributes such as taste. Thus, though any individual’s actual subsistence threshold is unobservable, their choice to switch away from the cheapest source of calories reveals that their marginal utility of calories is low and that they have surpassed subsistence. Accordingly, the percent of calories consumed from the staple food source, or the staple calorie share (SCS), can be used as an indicator for nutritional sufficiency. (source, source)

  • Still another method consists of measuring hunger’s physical effects on growth and thinness. Instead of measuring calorie intake, hunger or revealed preferences, you measure people’s length, their stunted growth and their body mass index. However, this is very approximative since length and weight may be determined by lots of factors, many of them unrelated to hunger.
  • And finally there are subjective approaches. The WFP does surveys asking people how often they ate in the last week and what they ate, how often they skip meals, how far they are away from markets, if their hunger is temporary or chronic etc. Gallup does something similar.

More on hunger here. And more posts in this series here.

Measuring Human Rights (24): Measuring Racism, Ctd.

Measuring racism is a problem, as I’ve argued before. Asking people if they’re racist won’t work because they don’t answer this question correctly, and understandably so. This is due to the social desirability bias. Surveys may minimize this bias if they approach the subject indirectly. For example, rather than simply asking people if they are racist or if they believe blacks are inferior, surveys could ask some of the following questions:

  • Do you believe God has created the races separately?
  • What do you believe are the reasons for higher incarceration rates/lower IQ scores/… among blacks?
  • Etc.

Still, no guarantee that bias won’t falsify the results. Maybe it’s better to dump the survey method altogether and go for something even more indirect. For example, you can measure

  • racism in employment decisions, such as numbers of callbacks received by applicants with black sounding names
  • racism in criminal justice, for example the degree to which black federal lower-court judges are overturned more often than cases authored by similar white judges, or differences in crime rates by race of the perpetrator, or jury behavior
  • racial profiling
  • residential racial segregation
  • racist consumer behavior, e.g. reluctance to buy something from a black seller
  • the numbers of interracial marriages
  • the numbers and membership of hate groups
  • the number of hate crimes
  • etc.

A disadvantage of many of these indirect measurements is that they don’t necessarily reflect the beliefs of the whole population. You can’t just extrapolate the rates you find in these measurements. It’s not because some judges and police officers are racist that the same rate of the total population is racist. Not all people who live in predominantly white neighborhoods do so because they don’t want to live in mixed neighborhoods. Different crime rates by race can be an indicator of racist law enforcement, but can also hide other causes, such as different poverty rates by race (which can themselves be indicators of racism). Higher numbers of hate crimes or hate groups may represent a radicalization of an increasingly small minority. And so on.

Another alternative measurement system is the Implicit Association Test. This is a psychological test that measures implicit attitudes and beliefs that people are either unwilling or unable to report.

Because the IAT requires that users make a series of rapid judgments, researchers believe that IAT scores may reflect attitudes which people are unwilling to reveal publicly. (source)

Participants in an IAT are asked to rapidly decide which words are associated. For example, is “female” or “male” associated with “family” and “career” respectively? This way, you can measure the strength of association between mental constructs such as “female” or “male” on the one hand and attributes such as “family” or “career” on the other. And this allows you to detect prejudice. The same is true for racism. You can read here or here how an IAT is usually performed.

Yet another measurement system uses evidence from Google search data, such as in this example. The advantage of this system is that it avoids the social desirability bias, since Google searches are done alone and online and without prior knowledge of the fact that the search results will be used to measure racism. Hence, people searching on Google are more likely to express social taboos. In this respect, the measurement system is similar to the IAT. Another advantage of the Google method, compared to traditional surveys, is that the Google sample is very large and more or less evenly distributed across all areas of a country. This allows for some fine grained geographical breakdown of racial animus.

More specifically, the purpose of the Google method is to analyze trends in searches that include words like “nigger” or “niggers” (not “nigga” because that’s slang in some Black communities, and not necessarily a disparaging term). In order to avoid searches for the term “nigger” by people who may not be racially motivated – such as researchers (Google can’t tell the difference) – you could refine the method and analyze only searches for phrases like “why are niggers lazy”, “Obama+nigger”, “niggers/blacks+apes” etc. If you find that those searches are more common in some locations than others, or that they become more common in some locations, then you can try to correlate those findings with other, existing indicators of racism such as those cited above, or with historic indicators such as prevalence of slavery or lynchings.

More posts in this series are here.

Measuring Human Rights (23): When “Worse” Doesn’t Necessarily Mean “Worse”, Ctd.

Just because nobody complains does not mean all parachutes are perfect. Benny Hill

A nice illustration of this piece of wisdom:

Using state-level variation in the timing of political reforms, we find that an increase in female representation in local government induces a large and significant rise in documented crimes against women in India. Our evidence suggests that this increase is good news, driven primarily by greater reporting rather than greater incidence of such crimes. (source)

The cited “increase in female representation in local government” resulted from a constitutional amendment requiring Indian states to have women in one-third of local government council positions.

Since then, documented crimes against women have risen by 44 percent, rapes per capita by 23 percent, and kidnapping of women by 13 percent. (source)

This uptick is probably not retaliatory – male “revenge” for female empowerment – but rather the result of the fact that more women in office has led to more crime reporting. Worse is therefore not worse. A timely reminder of the difficulties measuring human rights violations. Measurements often depend on reporting, and reporting can be influenced, for good and for bad. Also, a good lesson about the danger of taking figures at face value.

Similar cases are here and here. More posts in this series are here.

Measuring Human Rights (22): When Can You Call Something a “Famine”?

With yet another famine in the Horn of Africa, perhaps it’s a good time for a few words about famine measurement.

People have a right to adequate nourishment and to be free from chronic hunger (see article 25 of the Universal Declaration). Starvation is an extreme form of violation of this right (and is obviously also a violation of the right to life). So we obviously want to know the existence and extent of cases of starvation. There are individual cases of starvation – a elderly person who has lost her mobility and social network may starve abandoned in her flat – but most cases involve large scale famines. Let’s focus on the latter.

The problem is that death by famine or starvation is difficult to identify. People suffering from extreme malnutrition often don’t die of hunger but of diseases provoked by malnutrition, such as pneumonia or diarrhea. Since those are diseases that can have other causes besides malnutrition, it’s often difficult to count the number of people who have died from malnutrition. Their body weight may tell us something, but you can’t go about weighing corpses on a large scale.

Hence it’s difficult to determine whether or not a famine has occurred or is occurring. When does widespread suffering of hunger become a famine? Not every food crisis or widespread occurrence of malnutrition leads to famine-type starvation. A famine is obviously characterized by mortality caused by malnutrition. So we must look at mortality rates, but given the difficulty of establishing whether deaths are caused by malnutrition or other factors, how do we decide that a certain mortality rate is caused by malnutrition and is therefore the symptom of a famine? It’s difficult.

And yet, it’s common to find newspaper reports about “an outbreak of famine” is this or other part of the world. Ideally, we only want to declare a famine when a famine is actually occurring or about to occur. False alarms are not only silly but they create indifference. Fortunately, people seem to have overcome some of the difficulties and have agreed on a non-arbitrary way to determine that there is a famine going on:

  • when overall mortality rates in a region are extremely high, or high compared to the baseline – which may itself be high already, perhaps because of a war (a mortality rate of at least two people per 10,000 per day is usually considered part of the evidence of famine conditions)
  • when this is combined with survey indicators about low food availability and malnutrition (a rate of malnutrition – ratio of weight to height – among children age six months to five years above an average of 30% is the usual measure here)
  • when there is anecdotal evidence (perhaps also from surveys)
  • and when there are proxy measures such as below average rainfall

then you can build a useful measurement and a more or less scientific way of ascertaining that a food crisis has passed the famine threshold.

None of this should be understood as implying that food crises which don’t reach the famine threshold are unimportant and don’t deserve attention or assistance. It only means that it’s a good thing to distinguish real famines from lesser crises and to avoid crying wolf.

One problem with the measurement system presented above is that it’s no help in preventing a famine. It’s difficult to turn it into a probability index rather than a threshold index. It tells you when a famine has occurred or is ongoing, not when there’s a risk of famine. When mortality rates are high, you’re already late, perhaps too late.

Measuring Human Rights (20): What is More Important, the Number or Percentage of People Suffering Human Rights Violations?

Take just one human right, the right not to suffer poverty: if we want to measure progress for this human right, we get something like the following fact:

[N]ever in the world have there been so many paupers as in the present times. But the reason of this is that there have never been so many people around. Indeed never in the history of the world has been the percentage of poor people been so low. (source)

So, is this good news or bad news? If it’s more important to reduce the share of the world population suffering a particular type of rights violation, then this is good news. On the other hand, there are now more people – in absolute, not in relative numbers – suffering from poverty. If we take individuals and the distinctions between persons seriously, we should conclude that this is bad news and we’re doing worse than before.

Thomas Pogge has argued for the latter view. Take another example: killing a given number of people doesn’t become less troubling if the world’s population increases. If we would discover that the real number of the world’s population at the time of the Holocaust was twice as large as previously assumed, that wouldn’t diminish the importance of the Holocaust. What matters is the absolute number of people suffering.

On the other hand, if we see that policies and interventions lead to a significant lowering of the proportion of people in poverty – or suffering from any other type of rights violation – between times t and t+n, then we would welcome that, and we would certainly want to know it. The fact that the denominator – total world population – has increased in the mean time, is probably something that has happened independently of those policies. In the specific case of poverty, a growing population can even make a decrease in relative numbers of people suffering from poverty all the more admirable. After all, many still believe (erroneously) in the Malthusian trap theory, which states that population growth necessarily leads to increases in poverty in absolute numbers.

More posts in this series are here.

Measuring Human Rights (18): Guerrilla Polling in Dictatorships

Measuring respect for human rights is most important in societies where respect is a rare commodity. The problem is that it’s not only most important in such societies, but also most difficult. You need a certain level of freedom to measure respect for human rights. And regimes that violate rights also have the means to cover up those violations. I’ve called that the catch 22 of rights measurement. One problem is public opinion: a lot of human rights measurement depends on public opinion polls, but such polls are notoriously unreliable in repressive regimes, for obvious reasons: the public in those countries is either misinformed, indoctrinated or afraid to speak out, or all of the above.

Hence, good quality human rights measurement requires some creative polling. Political scientists Angela Hawken and Matt Leighty have come up with a new strategy, called guerrilla polling. Here’s an example:

Kim Eun Ho is a former police officer from North Korea who defected to the South in 2008. … With the aid of a friend and a smuggled cell phone, he is circumventing North Korea’s leadership to solicit opinions from its citizens.

Kim conducts a nightly public-opinion poll of North Korean residents, the first poll of its kind and illegal in North Korea. Here’s how it works: Kim calls his friend in North Korea on a smuggled cell phone. The friend then uses a North Korean land line to call a subject and presses the cell phone against the handset of the landline phone, allowing Kim to conduct a brief interview.

If the interviewee were discovered by the police, they would almost certainly be punished — perhaps severely. To circumvent the North Korean police, Kim has tailored his questions so that they take about 90 seconds to answer. He tapped phones himself as a North Korean police officer, and he estimates that it takes about two to three minutes for the police to trace a call. (source)

More posts about human rights measurement are here.

Measuring Human Rights (17): Human Rights and Progress

We’re all aware of the horrors of recent history. The 20th century doesn’t get a good press. And yet, most of us still think that humanity is, on average, much better off today  than it was some centuries or millennia ago. The holocaust, Rwanda, Hiroshima, AIDS, terrorism etc. don’t seem to have discouraged the idea of human progress in popular imagination. Those have been disasters of biblical proportions, and yet they are seen as temporary lapses, regrettable but exceptional incidents that did not jeopardize the overall positive evolution of mankind. Some go even further and call these events instances of “progressive violence”: disasters so awful that they bring about progress. Hitler was necessary in order to finally make Germany democratic. The Holocaust was necessary to give the Jews their homeland and the world the Universal Declaration. Evil has to become so extreme that it finally convinces humanity that evil should be abolished.

While that is obviously ludicrous, it’s true that there has been progress:

  • we did practically abolish slavery
  • torture seems to be much less common and much more widely condemned, despite the recent uptick
  • poverty is on the retreat
  • equality has come within reach for non-whites, women and minorities of different kinds
  • there’s a real reduction in violence over the centuries
  • war is much less common and much less bloody
  • more and more countries are democracies and freedom is much more widespread
  • there’s more free speech because censorship is much more difficult now thanks to the internet
  • health and labor conditions have improved for large segments of humanity, resulting in booming life expectancy
  • etc.

So, for a number of human rights, things seem to be progressing quite a lot. Of course, there are some areas of regress: the war on terror, gendercide, islamism etc. Still, those things don’t seem to be weighty enough to discourage the idea of progress, which is still quite popular. On the other hand, some human rights violations were caused by elements of human progress. The Holocaust, for example, would have been unimaginable outside of our modern industrial society. Hiroshima and Mutually Assured Destruction are other examples. Both nazism and communism are “progressive” philosophies in the sense that they believe that they are working for a better society.

Whatever the philosophical merits of the general idea of progress, progress in the field of respect for human rights boils down to a problem of measurement. How doe we measure the level of respect for the whole of the set of human rights? It’s difficult enough to measure respect for the present time, let alone for previous periods in human history for which data are incomplete or even totally absent. Hence, general talk about progress in the field of human rights is probably impossible. More specific measurements of parts of the system of human rights are more likely to succeed, but only for relatively recent time frames.

Measuring Human Rights (16): The Right to Healthcare

(There’s a more theoretical post here about the reasons why we should call health care a human right. But even if you think those are bad reasons, you may find the following useful).

The right to health care is one of the most difficult rights to measure. You can either try to measure people’s health directly and assume that good health means good health care, or you can measure the provision of health care and assume that there will be good health with a good health care system. Doing the latter means, for example:

  • measuring the number of health workers per capita for countries
  • measuring the quality of hospitals
  • measuring health care spending by governments
  • measuring the availability and affordability of health care
  • measuring the availability and affordability of health insurance
  • etc.

Doing the former means:

  • measuring life expectancy
  • measuring infant mortality
  • measuring maternal mortality
  • measuring calorie intake
  • measuring the incidence of certain diseases
  • measuring the survival rates for certain diseases
  • etc.

Needless to say that every single one of these measurements is fraught with problems, although some more so than others. Even if you’re able to have a pretty good measurement for a single indicator for a single country, it may be difficult to compare the measurement across countries. For example, health insurance is organized in so many different ways that it may be impossible to compare the level of insurance across different countries.

But let’s focus on another measure. Life expectancy is often used as a proxy for health. And indeed, when people live longer, on average, we can reasonably assume that they are healthier and that their health care system is better. It’s also something that is relatively easy to measure, compared to other indicators, since even developing countries usually have reasonably good data based on birth and death certificates. And yet, I say “relatively” because there are some conceptual and definitional problems:

  • Exceptional events such as a natural disasters or a war can drag down life expectancy numbers, but those events need not influence health in general or the quality of health care.
  • Wealthy countries may have more deaths from car accidents than poorer countries, simply because they have more cars. This will pull their relative life expectancy down somewhat, given that younger people are more likely to die in car accidents. And if you use life expectancy to measure health you’ll get a smaller health gap compared to poorer countries than is the case in reality (at least if life expectancy is not corrected for this and if it’s not supplemented with other health indicators).
  • How are miscarriages counted? If they are counted as child mortality, they drag down life expectancy rates compared to countries where they are not counted.
  • What about countries that have more homicides? Or suicides? Although the latter should arguably count since suicides are often caused by bad mental health. If a country’s life expectancy rate is pulled down by high suicide rates, life expectancy rates are still a good indicator of health and of the quality of health care, assuming that health care can reduce suicide rates and remove, to some extent, the underlying health causes of suicide. However, homicides are different: a country with a very good health care system, a very high level of health and a high murder rate can have its health rating pulled down artificially when only life expectancy is used to measure health.
  • Differences in diet and other types of risky behavior should also be excluded when comparing health and life expectancy across countries. It’s wellknown, for instance, that obesity is more of a problem in the U.S. than in many countries that are otherwise comparable to it. Obesity drags down life expectancy and reduces the average level of health, so life expectancy rates which are not corrected for obesity rates are still a good measure for health, but they are not a good measure for the quality of the U.S. health care system. If you want to use life expectancy rates to compare the quality of health care systems you’ll have to correct for obesity rates and perhaps for other types of risky behavior such as smoking or the absence of exercise. Maybe the U.S. health care system, even though it “produces” somewhat lower life expectancy rates than in comparable countries, is actually better than in other countries, yet still not good enough to offset the detrimental effects of high average obesity.

Hence, uncorrected life expectancy rates may not be such a good indicator of national health and of the quality of a national health care system. If we return to the case of the U.S., some of this may explain the strange fact that this country spends a lot more on health and yet has somewhat lower life expectancy rates than comparable countries.

Or maybe this discrepancy is caused by a combination of some misuse and waste at the spending side – more spending on health doesn’t necessarily result in better health – and some problems or peculiarities with the measurement of life expectancy. Let’s focus on the latter. As stated above, some cultural elements of American society, such as obesity, pull down life expectancy and worsen health outcomes. But there are other peculiarities that also pull down life expectancy, and that have nothing to do with health. I’m thinking of course of the relatively high levels of violence in the U.S. Death by assault is 5 to 10 times higher in the U.S. than in comparable countries (although those numbers tend to go down with the passing of time). This affects younger people more than older people, and when more young people die, life expectancy rates drop sharper than when more old people die.

However, even if you correct U.S. life expectancy rates for this, the rates don’t move up a lot (see here). The reason is that the numbers of deaths caused by homicide pale in comparison to other causes. Obesity levels, for instance, are a more important cause. But correcting life expectancy rates for obesity levels doesn’t seem appropriate, because we want to measure health. If you leave out all reasons for bad health from life expectancy statistics, your life expectancy rates go up, but your average health doesn’t. Obesity isn’t the same as homicide. Correcting life expectancy statistics for non-health related deaths such as homicide makes them a better indicator of health. Removing deaths from obesity doesn’t. If you have life expectancy rates without obesity, they may be a fairer judgment of the health care system but not a fairer judgment of health: a health care system in a country with a lot of obesity may be equally good as the one in another country and yet result in lower life expectancy. The former country does not necessarily have lower life expectancy because of its underperforming health care system – we assumed it’s of the same quality as elsewhere – but because of its culture of obesity.

However, if you really want to judge health care systems, you could argue that countries plagued by obesity should have a better quality system than other countries. They need a better quality system to fight the consequences of obesity and achieve similar life expectancy rates as other countries that don’t need to spend so much to fight obesity. So, life expectancy is then reinstituted as a good measure of health.

A Killer Argument Against the Quantitative Approach to Human Rights?

Joseph Stalin wasn’t a very nice man. Among his lesser sins was his disdain for statistics: “kill a man and it’s a tragedy, kill a million and it’s a statistic”. What he meant of course was not just that it’s a statistic, but also that it’s not very important. Who cares if Stalin or anyone else killed one million, 10 million or 6,321,012? People care about actual persons, not numbers. (Actually, it’s a misquote; he never really said it).

Regular readers of this blog immediately recognize this as a frontal attack on our main project, the quantitative approach to human rights. I believe that it’s very important to have statistics and other quantitative data on human rights violations if we want to measure progress on human rights. In other words, I do care about numbers. We need to know how many people die of hunger, how many live in poverty etc. so that we can assess the quality and impact of our policies.

Now, I have to admit that Stalin was on to something. Numbers don’t carry a lot of meaning and don’t engender empathy. Powerful anecdotes about the fate of individual persons, testimonies and other narratives about concrete cases make it more likely that people start to care. If you tell school children for example that an estimated 850,000 people died during the Rwandan genocide or that less than 20% of China’s citizens now live on less than $1 a day compared to 80% 30 years ago, they will probably register this information, but they will only really start to care about genocide or poverty when they read about the stories of individuals. If you focus on human rights violations as quantities you may end up viewing human beings as quantities as well, and then you lose the motivating power of the individual story. There’s no room for differences between cases if you focus on numbers, and there are no individual and motivating stories without differences between cases.

The same argument against the abstraction and lack of meaning in numbers can be used against human rights talk in general, and not just quantitative talk. Human rights talk, like number talk, is abstract, devoid of specific personal stories. It’s talk about a biological species and the rights that it has, not about persons. The lists of human rights in treaties and declarations are very general and abstract sentences separated from specific circumstances and people, as they have to be. Human rights make differences between people morally irrelevant, and they have to do so otherwise you end up with privileges instead of human rights. However, we may end up not with the desired equality of rights but with sameness and interchangeable specimens of a biological species. And then we lose the motivating power of very specific and personal stories about suffering and oppression.

The answer to this challenge against number talk and rights talk is obvious, however: one approach doesn’t exclude the other. Numbers and abstractions may not be very motivating but they can help to assess the success of people who are otherwise motivated. And some of us may be motivated by numbers after all.

Measuring Human Rights (15): Measuring Segregation Using the Dissimilarity Index

If people tend to live, work, eat or go to school together with other members of their group – race, gender etc. – then we shouldn’t automatically assume that this is caused by discrimination, forced separation, restrictions on movement or choice of residence, or other kinds of human rights violations. It can be their free choice. However, if it’s not, then we usually call it segregation and we believe it’s a moral wrong that should be corrected. People have a right to live where they want, go to school where they want, and move freely about (with some restrictions necessary to protect the property rights and the freedom of association of others). If they are prohibited from doing so, either by law (e.g. Jim Crow) or by social pressure (e.g. discrimination by landlords or employers), then government policy and legislation should step in in order to better protect people’s rights. Forced desegregation is then an option, and this can take various forms, such as anti-discrimination legislation in employment and rent, forced integration of schools, busing, zoning laws, subsidized housing etc.

There’s also some room for intervention when segregation is not the result of conscious, unconscious, legal or social discrimination. For example, poor people tend to be segregated in poor districts, not because other people make it impossible for them to live elsewhere but because their poverty condemns them to certain residential areas. The same is true for schooling. In order to avoid poverty traps or membership poverty, it’s better to do something about that as well.

In all such cases, the solution should not necessarily be found in physical desegregation, i.e. forcibly moving people about. Perhaps the underlying causes of segregation, rather than segregation itself, should be tackled. For example, rather than moving poor children to better schools or poor families to better, subsidized housing, perhaps we should focus on their poverty directly.

However, before deciding what to do about segregation, we have to know its extent. Is it a big problem, or a minor one? How does it evolve? Is it getting better? How segregated are residential areas, schools, workplaces etc.? And to what extent is this segregation involuntary? The latter question is a hard one, but the others can be answered. There are several methods for measuring different kinds of segregation. The most popular measure of residential segregation is undoubtedly the so-called index of dissimilarity. If you have a city, for example, that is divide into N districts (or sections, census tracts or whatever), the dissimilarity index measures the percentage of a group’s population that would have to change districts for each district to have the same percentage of that group as the whole city.

The dissimilarity index is not perfect, mainly because it depends on the sometimes arbitrary way in which cities are divided into districts or sections. Which means that modifying city partitions can influence levels of “segregation”, which is not something we want. Take this extreme example. You can show the same city twice, with two different partitions, A and B situation. No one has moved residency between situations A and B, but the district boundaries have been altered radically. In situation A with the districts drawn in a certain way, there is no segregation (dissimilarity index of 0). But in situation B, with the districts drawn differently, there is complete segregation (index = 1), although no one has physically moved. That’s why other, complementary measures are probably necessary for correct information about levels of segregation. Some of those measures are proposed here and here.

Measuring Human Rights (14): Numbers of Illegal Immigrants

Calculating a reliable number for a segment of the population that generally wants to hide from officials is very difficult, but it’s politically very important to know more or less how many illegal immigrants there are, and whether their number is increasing or decreasing. There’s a whole lot of populist rhetoric floating around, especially regarding jobs and crime, and passions are often inflamed. Knowing how many illegal immigrants there are – more or less – allows us to quantify the real effects on employment and crime, and to deflate some of the rhetoric.

Immigration is a human rights issue in several respects. Immigration is often a way for people to escape human rights violations (such as poverty or persecution). And upon arrival, immigrants – especially illegal immigrants – often face other human rights violations (invasion of privacy, searches, labor exploitation etc.). The native population may also fear – rightly or wrongly – that the presence of large groups of immigrants will lower their standard of living or threaten their physical security. Illegal immigrants especially are often accused of pulling down wages and labor conditions and of creating native unemployment. If we want to disprove such accusations, we need data on the numbers of immigrants.

So how do we count the number of illegal immigrants? Obviously there’s nothing in census data. The Census Bureau doesn’t ask people about their immigration status, in part because such questions may drive down overall response rates. Maybe in some cases the census data of other countries can help. Other countries may ask their residents how many family members have gone abroad to find a job.

Another possible source are the numbers of births included in hospital data. If you assume a certain number of births per resident, and compare that to the total number of births, you may be able to deduce the number of births among illegal immigrants (disparagingly called “anchor babies“), which in turn may give you an idea about the total number of illegal immigrants.

Fluctuations in the amounts of remittances – money sent back home by immigrants – may also indicate trends in illegal immigration, although remittances are of course sent by both legal and illegal immigrants. Furthermore, it’s not because remittances go down that immigrants leave. It might just be a temporary drop following an economic recession, and immigrants decide to sweat it out (possibly supported by reverse remittances for the time of the recession). Conversely, an increase in remittances may simply reflect technological improvements in international payment systems.

Perhaps a better indicator are the numbers of apprehensions by border-patrol units. However, fluctuations in these numbers may not be due to fluctuations in immigration. Better or worse performance by border-patrol officers or tighter border security may be the real reasons.

So, it’s really not easy to count illegal immigrants, and that means that all rhetoric about illegal immigration – both positive and negative – should be taken with a grain of salt.

More posts on this series are here.

Measuring Human Rights (13): When More Means Less and Vice Versa

Human rights violations can make it difficult to measure human rights violations, and can distort international comparisons of the levels of respect for human rights. Country A, which is generally open and accessible and on average respects basic rights such as speech, movement and press fairly well, may be more in the spotlight of human rights groups than country B which is borderline totalitarian. And not just more in the spotlight: attempts to quantify or measure respect for human rights may in fact yield a score that is worse for A than for B, or at least a score that isn’t much better for A than for B. The reason is of course the openness of A:

  • Human rights groups, researchers and statisticians can move and speak relatively freely in A.
  • The citizens of A aren’t scared shitless by their government and will speak to outsiders.
  • Country A may even have fostered a culture of public discourse, to some extent. Perhaps its citizens are also better educated and better able to analyze political conditions.
  • As Tocqueville has famously argued, the more a society liberates itself from inequalities, the harder it becomes to bear the remaining inequalities. Conversely, people in country B may not know better or may have adapted their ambitions to the rule of oppression. So, citizens of A may have better access to human rights groups to voice their complaints, aren’t afraid to do so, can do so because they are relatively well educated, and will do so because their circumstances seem more outrageous to them even if they really aren’t. Another reason to overestimate rights violations in A and underestimate them in B.
  • The government administration of A may also be more developed, which often means better data on living conditions. And better data allow for better human rights measurement. Data in country B may be secret or non-existent.

I called all this the catch 22 of human rights measurement: in order to measure whether countries respect human rights, you already need respect for human rights. Investigators or monitors must have some freedom to control, to engage in fact finding, to enter countries and move around, to investigate “in situ”, to denounce etc., and victims should have the freedom to speak out and to organize themselves in pressure groups. So we assume what we want to establish. (A side-effect of this is that authoritarian leaders may also be unaware of the extent of suffering among their citizens).

You can see the same problem in the common complaints that countries such as the U.S. and Israel get a raw deal from human rights groups:

[W]hy would the watchdogs neglect authoritarians? We asked both Human Rights Watch and Amnesty, and received similar replies. In some cases, staffers said, access to human rights victims in authoritarian countries was impossible, since the country’s borders were sealed or the repression was too harsh (think North Korea or Uzbekistan). In other instances, neglected countries were simply too small, poor, or unnewsworthy to inspire much media interest. With few journalists urgently demanding information about Niger, it made little sense to invest substantial reporting and advocacy resources there. … The watchdogs can and do seek to stimulate demand for information on the forgotten crises, but this is an expensive and high risk endeavor. (source)

So there may also be a problem with the supply and demand curve in media: human rights groups want to influence public opinion, but can only do so with the help of the media. If the media neglect certain countries or problems because they are deemed “unnewsworthy”, then human rights groups will not have an incentive to monitor those countries or problems. They know that what they will be able to tell will fall on deaf ears anyway. So better focus on the things and the countries which will be easier to channel through the media.

Both the catch 22 problem and the problems caused by media supply and demand can be empirically tested by comparing the intensity of attention given by human rights monitoring organizations to certain countries/problems to the intensity of human rights violations (the latter data are assumed to be available, which is a big assumption, but one could use very general measures such as these). It seems that both effects are present but not much:

[W]e subjected the 1986-2000 Amnesty [International] data to a barrage of statistical tests. (Since Human Rights Watch’s early archival procedures seemed spotty, we did not include their data in our models.) Amnesty’s coverage, we found, was driven by multiple factors, but contrary to the dark rumors swirling through the blogosphere, we discovered no master variable at work. Most importantly, we found that the level of actual violations mattered. Statistically speaking, Amnesty reported more heavily on countries with greater levels of abuse. Size also mattered, but not as expected. Although population didn’t impact reporting much, bigger economies did receive more coverage, either because they carried more weight in global politics and economic affairs, or because their abundant social infrastructure produced more accounts of abuse. Finally, we found that countries already covered by the media also received more Amnesty attention. (source)

More posts in this series are here.

Measuring Human Rights (12): Measuring Public Opinion on Torture

Measuring the number and gravity of cases of actual torture is extremely difficult, for apparent reasons. It takes place in secret, and the people subjected to torture are often in prison long afterwards, or don’t survive it. Either way, they can’t tell us.

That’s why people try to find other ways to measure torture. Asking the public when and under which circumstances they think torture is acceptable may give an approximation of the likelihood of torture, at least as long as we assume that in democratic countries governments will only engage in torture if there’s some level of public support for it. This approach won’t work in dictatorships, obviously, since public opinion in a dictatorship is often completely irrelevant.

However, measuring public opinion on torture has proven to be very difficult and misleading:

Many journalists and politicians believe that during the Bush administration, a majority of Americans supported torture if they were assured that it would prevent a terrorist attack. … But this view was a misperception … we show here that a majority of Americans were opposed to torture throughout the Bush presidency…even when respondents were asked about an imminent terrorist attack, even when enhanced interrogation techniques were not called torture, and even when Americans were assured that torture would work to get crucial information. Opposition to torture remained stable and consistent during the entire Bush presidency.

Gronke et al. attribute confusion of beliefs [among many journalists] to the so-called false consensus effect studied by cognitive psychologists, in which people tend to assume that others agree with them. For example: The 30% who say that torture can “sometimes” be justified believe that 62% of Americans do as well. (source)

Measuring Human Rights (9): When “Worse” Doesn’t Necessarily Mean “Worse”

I discussed in this older post some of the problems related to the measurement of human rights violations, and to the assessment of progress or deterioration. One of the problems I mentioned is caused by improvements in measurement methods. Such improvements can in fact result in a statistic showing increasing numbers of rights violations, whereas in reality the numbers may not be increasing, and perhaps even decreasing. Better measurement means that you now compare current data that are more complete and better measured, with older numbers of rights violations that were simply incomplete.

The example I gave was about rape statistics: better statistical and reporting methods used by the police, combined with less social stigma etc. result in statistics showing a rising number of rapes, but this increase was due to the measurement methods (and other effects), not to what happened in real life.

I now came across another example. Collateral damage – or the unintentional killing of civilians during wars – seems to be higher now than a century ago (source). This may also be the result of better monitoring hiding a totally different trend. We all know that civilian deaths are much less acceptable now than they used to be, and that journalism and war reporting are probably much better (given better communication technology). Hence, people may now believe that it’s more important to count civilian deaths, and have better means to do so. As a result, the numbers of civilian deaths showing up in statistics will rise compared to older periods, but perhaps the real numbers don’t rise at all.

Of course, the increase of collateral damage may be the result of something else than better measurement: perhaps the lower level of acceptability of civilian deaths forces the army to classify some of those deaths as unintentional, even if they’re not (and then we have worse rather than better measurement). Or perhaps the relatively recent development of precision-guided munition has made the use of munition more widespread so that there are more victims: more bombs, even more precise bombs, can make more victims than less yet more imprecise bombs. Or perhaps the current form of warfare, with guerilla troops hiding among populations, does indeed produce more civilian deaths.

Still, I think my point stands: better measurement of human rights violations can give the wrong impression. Things may look as if they’re getting worse, but they’re not.

Measuring Human Rights (8): Measurement of the Fairness of Trials and of Expert Witnesses

An important part of the system of human rights are the rules intended to offer those accused of crimes a fair trial in court. We try to treat everyone, even suspected criminals, with fairness, and we have two principal reasons for this:

  • We only want to punish real criminals. A fair trial is one in which everything is done to avoid punishing the wrong persons. We want to avoid miscarriages of justice.
  • We also want to use court proceedings only to punish criminals and deter crime, not for political or personal reasons, as is often the case in dictatorships.

Most of these rules are included in, for example, articles 9, 10, 14 and 15 of the International Covenant on Civil and Political Rights, article 10 of the Universal Declaration, article 6 of the European Convention of Human Rights, and the Sixth Amendment to the United States Constitution.

Respect for many of these rules can be measured statistically. I’ll mention only one here: the rule regarding the intervention of expert witnesses for the defense or the prosecution. Here’s an example of the way in which this aspect of a fair trial can measured:

In the late 1990s, Harris County, Texas, medical examiner [and forensic specialist] Patricia Moore was repeatedly reprimanded by her superiors for pro-prosecution bias. … In 2004, a statistical analysis showed Moore diagnosed shaken baby syndrome (already a controversial diagnosis) in infant deaths at a rate several times higher than the national average. … One woman convicted of killing her own child because of Moore’s testimony was freed in 2005 after serving six years in prison. Another woman was cleared in 2004 after being accused because of Moore’s autopsy results. In 2001, babysitter Trenda Kemmerer was sentenced to 55 years in prison after being convicted of shaking a baby to death based largely on Moore’s testimony. The prosecutor in that case told the Houston Chronicle in 2004 that she had “no concerns” about Moore’s work. Even though Moore’s diagnosis in that case has since been revised to “undetermined,” and Moore was again reprimanded for her lack of objectivity in the case, Kemmerer remains in prison. (source)

Measuring Human Rights (7): Don’t Let Governments Make it Easy on Themselves

In many cases, the task of measuring respect for human rights in a country falls on the government of that country. It’s obvious that this isn’t a good idea in dictatorships: governments there will not present correct statistics on their own misbehavior. But if not the government, who else? Dictatorships aren’t known for their thriving and free civil societies, or for granting access to outside monitors. As a result, human rights protection can’t be measured.

The problem, however, of depending on governments for human rights measurement isn’t limited to dictatorships. I also gave examples of democratic governments not doing a good job in this respect. Governments, also democratic ones, tend to choose indicators they already have. For example, number of people benefiting from government food programs (they have numbers for that), neglecting private food programs for which information isn’t readily available. In this case, but in many other cases as well, governments choose indicators which are easy to measure, rather than indicators which measure what needs to be measured but which require a lot of effort and money.

Human rights measurement also fails to measure what needs to be measured when the people whose rights we want to measure don’t have a say on which indicators are best. And that happens a lot, even in democracies. Citizen participation is a messy thing and governments tend to want to avoid it, but the result may be that we’re measuring the wrong thing. For example, we think we are measuring poverty when we count the number of internet connections for disadvantaged groups, but these groups may consider the lack of cable TV or public transportation a much more serious deprivation. The reason we’re not measuring what we think we are measuring, or what we really need to measure, is not – as in the previous case – complacency, lack of budgets etc. The reason is a lack of consultation. Because there hasn’t been consultation, the definition of “poverty” used by those measuring human rights is completely different from the one used by those whose rights are to be measured. And, as a result, the indicators that have been chosen aren’t the correct ones, or they don’t show the whole picture. Many indicators chosen by governments are also too specific, measuring only part of the human right (e.g. free meals for the elderly instead of poverty levels for the elderly).

However, even if the indicators that are chosen are the correct ones – i.e. indicators that measure what needs to be measured, completely and not partially – it’s still the case that human rights measurement is extremely difficult, not only conceptually, but also and primarily on the level of execution. Not only are there many indicators to measure, but the data sources are scarce and often unreliable, even in developed countries. For example, let’s assume that we want to measure the human right not to suffer poverty, and that we agree that the best and only indicator to measure respect for this right is the level of income.* So we cleared up the conceptual difficulties. The problem now is data sources. Do you use tax data (taxable income)? We all know that there is tax fraud. Low income declared in tax returns may not reflect real poverty. Tax returns also don’t include welfare benefits etc.

Even if you manage to produce neat tables and graphs you always have to stop and think about the messy ways in which they have been produced, about the flaws and lack of completeness of the chosen indicators themselves, and about the problems encountered while gathering the data. Human rights measurement will always be a difficult thing to do, even under the best circumstances.

* This isn’t obvious. Other indicators could be level of consumption, income inequality etc. But let’s assume, for the sake of simplicity, that level of income is the best and only indicator for this right.

Measuring Human Rights (6): Don’t Make Governments Do It

In the case of dictatorial governments or other governments that are widely implicated in the violation of the rights of their citizens, it’s obvious that the task of measuring respect for human rights should be – where possible – carried out by independent non-governmental organizations, possibly even international or foreign ones (if local ones are not allowed to operate). Counting on the criminal to report on his crimes isn’t a good idea. Of course, sometimes there’s no other way. It’s often impossible to estimate census data, for example, or data on mortality, healthcare providers etc. without using official government information.

All this is rather trivial. The more interesting point, I hope, is that the same is true, to some extent, of governments that generally have a positive attitude towards human rights. Obviously, the human rights performance of these governments also has to be measured, because there are rights violations everywhere, and a positive attitude doesn’t guarantee positive results. However, even in such cases, it’s not always wise to trust governments with the task of measuring their own performance in the field of human rights. An example from a paper by Marilyn Strathern (source, gated):

In 1993, new regulations [required] local authorities in the UK … to publish indicators of output, no fewer than 152 of them, covering a variety of issues of local concern. The idea was … to make councils’ performance transparent and thus give them an incentive to improve their services. As a result, however,… even though elderly people might want a deep freeze and microwave rather than food delivered by home helps, the number of home helps [was] the indicator for helping the elderly with their meals and an authority could only improve its recognised performance of help by providing the elderly with the very service they wanted less of, namely, more home helps.

Even benevolent governments can make crucial mistakes like these. This example isn’t even a measurement error; it’s measuring the wrong thing. And the mistake wasn’t caused by the government’s will to manipulate, but by a genuine misunderstanding of what the measurement should be all about.

I think the general point I’m trying to make is that human rights measurement should take place in a free market of competing measurements – and shouldn’t be a (government) monopoly. Measurement errors are more likely to be identified if there is a possibility to compare competing measurements of the same thing.

Measuring Human Rights (5): Some (Insurmountable?) Problems

If you care about human rights, it’s extremely important to measure the level of protection of human rights in different countries, as well as the level of progress or deterioration. Measurement in the social sciences is always tricky; we’re dealing with human behavior and not with sizes, volumes, speeds etc. However, measuring human rights is especially difficult.

Some examples. I talked about the so-called catch 22 of human rights measurement. In order to measure whether countries respect human rights, one already needs respect for human rights. Organizations, whether international organizations or private organizations (NGOs), must have some freedom to control, to engage in fact finding, to enter countries and move around, to investigate “in situ”, to denounce etc. Victims should have the freedom to speak out and to organize themselves in pressure groups. So we assume what we want to establish.

The more violations of human rights, the more difficult it is to monitor respect for human rights. The more oppressive the regime, the harder it is to establish the nature and severity of its crimes; and the harder it is to correct the situation.

So, a country which does a very bad job protecting human rights, may not have a low score because the act of giving the country a correct score is made impossible by its government. On the other hand, a low score for human rights (or certain human rights) may not be as bad as it seems, because at least it was possible to determine a score.

Another example: suppose a country shows a large increase in the number of rapes. At first sight, this is a bad thing, and would mean giving the country a lower score on certain human rights (such as violence against women, gender discrimination etc.). But perhaps the increase in the number of rapes is simply the result of a larger number of rapes being reported to the police. And better reporting of rape may be the result of a more deeply and widely ingrained human rights culture, or, in other words, it may be the reflection of a growing consciousness of women’s rights and gender equality.

So, a deteriorating score may actually hide progress.

The same can be said of corruption or police brutality. A deteriorating score may simply be a matter of perception, a perception created by more freedom of the press.

I don’t know how to solve these problems, but I think it’s worth mentioning them. They are probably the reason why there is so little good measurement in the field of human rights, and so much anecdotal reporting.