Measuring Human Rights (18): Guerrilla Polling in Dictatorships

Measuring respect for human rights is most important in societies where respect is a rare commodity. The problem is that it’s not only most important in such societies, but also most difficult. You need a certain level of freedom to measure respect for human rights. And regimes that violate rights also have the means to cover up those violations. I’ve called that the catch 22 of rights measurement. One problem is public opinion: a lot of human rights measurement depends on public opinion polls, but such polls are notoriously unreliable in repressive regimes, for obvious reasons: the public in those countries is either misinformed, indoctrinated or afraid to speak out, or all of the above.

Hence, good quality human rights measurement requires some creative polling. Political scientists Angela Hawken and Matt Leighty have come up with a new strategy, called guerrilla polling. Here’s an example:

Kim Eun Ho is a former police officer from North Korea who defected to the South in 2008. … With the aid of a friend and a smuggled cell phone, he is circumventing North Korea’s leadership to solicit opinions from its citizens.

Kim conducts a nightly public-opinion poll of North Korean residents, the first poll of its kind and illegal in North Korea. Here’s how it works: Kim calls his friend in North Korea on a smuggled cell phone. The friend then uses a North Korean land line to call a subject and presses the cell phone against the handset of the landline phone, allowing Kim to conduct a brief interview.

If the interviewee were discovered by the police, they would almost certainly be punished — perhaps severely. To circumvent the North Korean police, Kim has tailored his questions so that they take about 90 seconds to answer. He tapped phones himself as a North Korean police officer, and he estimates that it takes about two to three minutes for the police to trace a call. (source)

More posts about human rights measurement are here.

Income Inequality (23): U.S. Public Opinion on Income Inequality

Despite what foreigners usually believe about the U.S., and despite the confused ramblings of a tiny group of anti-“socialist” loudmouths high on tea, U.S. public opinion is actually very egalitarian:

Americans are in broad agreement on the need for a more equal distribution of wealth. … that’s what a forthcoming study by two psychologists, Dan Ariely of Duke University and Michael I. Norton of Harvard Business School, has concluded. First, Ariely and Norton asked thousands of Americans what they thought the nation’s actual wealth distribution looks like: how much is owned by the wealthiest 20 percent of the population, the next-wealthiest 20 percent, and on down. The researchers then asked people what, in an ideal world, they would like the nation’s wealth distribution to be.

Ariely and Norton found that Americans think they live in a far more equal country than they in fact do. On average, those surveyed estimated that the wealthiest 20 percent of Americans own 59 percent of the nation’s wealth; in reality the top quintile owns around 84 percent. The respondents further estimated that the poorest 20 percent own 3.7 percent, when in reality they own 0.1 percent.

And when asked to give their ideal distribution, they described, on average, a nation where the wealth distribution looks not like the U.S. but like Sweden, only more so—the wealthiest quintile would control just 32 percent of the wealth, the poorest just over 10 percent. “People dramatically underestimated the extent of wealth inequality in the U.S.,” says Ariely. “And they wanted it to be even more equal.” (source)

Measuring Human Rights (12): Measuring Public Opinion on Torture

Measuring the number and gravity of cases of actual torture is extremely difficult, for apparent reasons. It takes place in secret, and the people subjected to torture are often in prison long afterwards, or don’t survive it. Either way, they can’t tell us.

That’s why people try to find other ways to measure torture. Asking the public when and under which circumstances they think torture is acceptable may give an approximation of the likelihood of torture, at least as long as we assume that in democratic countries governments will only engage in torture if there’s some level of public support for it. This approach won’t work in dictatorships, obviously, since public opinion in a dictatorship is often completely irrelevant.

However, measuring public opinion on torture has proven to be very difficult and misleading:

Many journalists and politicians believe that during the Bush administration, a majority of Americans supported torture if they were assured that it would prevent a terrorist attack. … But this view was a misperception … we show here that a majority of Americans were opposed to torture throughout the Bush presidency…even when respondents were asked about an imminent terrorist attack, even when enhanced interrogation techniques were not called torture, and even when Americans were assured that torture would work to get crucial information. Opposition to torture remained stable and consistent during the entire Bush presidency.

Gronke et al. attribute confusion of beliefs [among many journalists] to the so-called false consensus effect studied by cognitive psychologists, in which people tend to assume that others agree with them. For example: The 30% who say that torture can “sometimes” be justified believe that 62% of Americans do as well. (source)

Lies, Damned Lies, and Statistics (31): Common Problems in Opinion Polls

Opinion polls or surveys are very useful tools in human rights measurement. We can use them to measure public opinion on certain human rights violations, such as torture or gender discrimination. High levels of public approval of such rights violations may make them more common and more difficult to stop. And surveys can measure what governments don’t want to measure. Since we can’t trust oppressive governments to give accurate data on their own human rights record, surveys may fill in the blanks. Although even that won’t work if the government is so utterly totalitarian that it doesn’t allow private or international polling of its citizens, or if it has scared its citizens to such an extent that they won’t participate honestly in anonymous surveys.

But apart from physical access and respondent honesty in the most dictatorial regimes, polling in general is vulnerable to mistakes and fraud (fraud being a conscious mistake). Here’s an overview of the issues that can mess up public opinion surveys, inadvertently or not.

Wording effect

There’s the well-known problem of question wording, which I’ve discussed in detail before. Pollsters should avoid leading questions, questions that are put in such a way that they pressure people to give a certain answer, questions that are confusing or easily misinterpreted, wordy questions, questions using jargon, abbreviations or difficult terms, double or triple questions etc. Also quite common are “silly questions”, questions that don’t have meaningful or clear answers: for example “is the catholic church a force for good in the world?” What on earth can you answer to that? Depends on what elements of the church you’re talking about, what circumstances, country or even historical period you’re asking about. The answer is most likely “yes and no”, and hence useless.

The importance of wording is illustrated by the often substantial effects of small modifications in survey questions. Even the replacement of a single word by another, related word, can radically change survey results.

Of course, one often claims that biased poll questions corrupt the average survey responses, but that the overall results of the survey can still be used to learn about time trends and difference between groups. As long as you make a mistake consistently, you may still find something useful. That’s true, but no reason not to take care of wording. The same trends and differences can be seen in survey results that have been produced with correctly worded questions.

Order effect or contamination effect

Answers to questions depend on the order they’re asked in, and especially on the questions that preceded. Here’s an example:

Fox News yesterday came out with a poll that suggested that just 33 percent of registered voters favor the Democrats’ health care reform package, versus 55 percent opposed. … The Fox News numbers on health care, however, have consistently been worse for Democrats than those shown by other pollsters. (source)

The problem is not the framing of the question. This was the question: “Based on what you know about the health care reform legislation being considered right now, do you favor or oppose the plan?” Nothing wrong with that.

So how can Fox News ask a seemingly unbiased question of a seemingly unbiased sample and come up with what seems to be a biased result? The answer may have to do with the questions Fox asks before the question on health care. … the health care questions weren’t asked separately. Instead, they were questions #27-35 of their larger, national poll. … And what were some of those questions? Here are a few: … Do you think President Obama apologizes too much to the rest of the world for past U.S. policies? Do you think the Obama administration is proposing more government spending than American taxpayers can afford, or not? Do you think the size of the national debt is so large it is hurting the future of the country? … These questions run the gamut slightly leading to full-frontal Republican talking points. … A respondent who hears these questions, particularly the series of questions on the national debt, is going to be primed to react somewhat unfavorably to the mention of another big Democratic spending program like health care. And evidently, an unusually high number of them do. … when you ask biased questions first, they are infectious, potentially poisoning everything that comes below. (source)

If you want to avoid this mistake – if we can call it that (since in this case it’s quite likely to have been a “conscious mistake” aka fraud) – randomizing the question order for each respondent might help.

Similar to the order effect is the effect created by follow-up questions. It’s well-known that follow-up questions of the type “but what if…” or “would you change your mind if …” change the answers to the initial questions.

Bradley effect

The Bradley effect is a theory proposed to explain observed discrepancies between voter opinion polls and election outcomes in some U.S. government elections where a white candidate and a non-white candidate run against each other.

Contrary to the wording and order effects, this isn’t an effect created – intentionally or not – by the pollster, but by the respondents. The theory proposes that some voters tend to tell pollsters that they are undecided or likely to vote for a black candidate, and yet, on election day, vote for the white opponent. It was named after Los Angeles Mayor Tom Bradley, an African-American who lost the 1982 California governor’s race despite being ahead in voter polls going into the elections.

The probable cause of this effect is the phenomenon of social desirability bias. Some white respondents may give a certain answer for fear that, by stating their true preference, they will open themselves to criticism of racial motivation. They may feel under pressure to provide a politically correct answer. The existence of the effect is, however, disputed. (Some say the election of Obama disproves the effect, thereby making another statistical mistake).

Fatigue effect

Another effect created by the respondents rather than the pollsters is the fatigue effect. As respondents grow increasingly tired over the course of long interviews, the accuracy of their responses could decrease. They may be able to find shortcuts to shorten the interview; they may figure out a pattern (for example that only positive or only negative answers trigger follow-up questions). Or they may just give up halfway, causing incompletion bias.

However, this effect isn’t entirely due to respondents. Survey design can be at fault as well: there may be repetitive questioning (sometimes deliberately for control purposes), the survey may be too long or longer than initially promised, or the pollster may want to make his life easier and group different polls into one (which is what seems to have happened in the Fox poll mentioned above, creating an order effect – but that’s the charitable view of course). Fatigue effect may also be caused by a pollster interviewing people who don’t care much about the topic.

Sampling effect

Ideally, the sample of people who are to be interviewed for a survey should represent a fully random subset of the entire population. That means that every person in the population should have an equal chance of being included in the sample. That means that there shouldn’t be self-selection (a typical flaw in many if not all internet surveys of the “Polldaddy” variety) or self-deselection. That reduces the randomness of the sample, which can be seen from the fact that self-selection leads to polarized results. The size of the sample is also important. Samples that are too small typically produce biased results.

Even the determination of the total population from which the sample is taken, can lead to biased results. And yes, that has to be determined… For example, do we include inmates, illegal immigrants etc. in the population? See here for some examples of the consequences of such choices.

House effect

A house effect occurs when there are systematic differences in the way that a particular pollster’s surveys tend to lean toward one or the other party’s candidates; Rasmussen is known for that.

I probably forgot an effect or two. Fill in the blanks if you care. Go here for other posts in this series.

What is Democracy? (51): Representatives as Actors and Authors

Sorry for this very long post, but I think this is important. During the discussions about healthcare reform in the U.S., opponents frequently mentioned the unpopularity of the proposed Bill (although now, after the Bill has been accepted and turned into law, it seems that its popularity has gone up). I don’t wish to engage in a discussion about the accuracy of the opinion polls that measure the popularity of healthcare reform (it’s obvious that extremely negative political propaganda has played a role, as well as lack of knowledge about the actual proposals).

What I want to do here is look at the deeper discussion about the problems arising from a representative body voting laws that are unpopular (or seem to be). One of the more eloquent dismissals of unpopular legislation, especially the healthcare legislation, comes from Megan McArdle:

Are we now in a world where there is absolutely no recourse to the tyranny of the majority? Republicans and other opponents of the bill did their job on this; they persuaded the country that they didn’t want this bill. And that mattered basically not at all. If you don’t find that terrifying, let me suggest that you are a Democrat who has not yet contemplated what Republicans might do under similar circumstances. Farewell, Social Security! Au revoir, Medicare! … Oh, wait–suddenly it doesn’t seem quite fair that Republicans could just ignore the will of their constituents that way, does it? … What I hope is that the Democrats take a beating at the ballot box and rethink their contempt for those mouth-breathing illiterates in the electorate. (source)

Apart from the fact that we usually mean something else by “tyranny of the majority” (i.e. majority approved and popular decisions violating the rights of minorities), she and others like her seem to have a valid point, but only at first glance. While I don’t believe that they advocate getting rid of the whole notion of elections and just leave decisions up to opinion polls, they certainly want to give opinion polls much greater weight and turn them into some sort of check on parliamentary majorities (however, it’s not clear how that is supposed to work).

I want to argue against this. It’s true that a democracy is all about electing leaders who are supposed to execute the will of the people by way of laws and policies (if we sidestep the important issue of direct participation). The people don’t vote laws and don’t decide and pursue policies themselves. They decide what can and cannot be viewed as the will of the people, but then they give someone else the power to execute this will in their name and to frame the laws and policies necessary for the execution of this will.

That’s because it’s practically very difficult to allow all people to participate in all decisions. In a representative system, the people can influence the laws and the policies of the government only indirectly. They elect those representatives who they think are likely to vote laws and implement policies in accordance with their wishes, and if, afterwards, the people find out that they elected the wrong representatives, they replace them. The desire to hold on to power, forces the representatives to act in accordance with the wishes of the people.

This means that representatives do not necessarily follow their own personal judgment or their own conscience. The people instruct them and tell them, in a general way perhaps, what kinds of laws or policies to implement, or at least they tell them which values should be promoted by laws and policies. In all their actions, the representatives must never forget whence they came, who elected them and for what reason. They are the servants of the people whom they represent and whose wishes they are supposed to realize with the help of laws and policies. If their own wishes and opinions collide with those of the people, then they should either set them aside or resign from their posts.

In other words, representatives are actors and not authors. The people are the authors and the representatives act out the words of the authors instead of their own words (although of course their own words may coincide with those of the people). This guarantees the congruence of power and society. The political actors speak and act the words owned by those whom they represent (the authors) and, if necessary, leave their own personality behind while doing their work. Their official personality must be the sum of the opinions of the electors who, for this reason, recognize themselves in the representatives. The representatives act with authority (a word related to the word “author”) and are likely to remain in office as long as this recognition lasts and as long as the representatives act in the way they were authorized to do. The difference between rulers and ruled is hereby eliminated, notwithstanding the fact that the representatives and the represented are not the same persons. They are not the same persons but they share the same personality (notice also the etymological origin of the word “person”, namely a mask worn by actors). Not only the election results, but also the laws and the government policies must be the reflection of the will of the people. Representatives do not only have authority on the basis of an election result, but also on the basis of their performance in office.

All that would vindicate the position of McArdle and other opponents of the healthcare bill. However, things are not as simple as this. Representation is more than just a convenient tool for self-government in large communities. It has certain other advantages.

[L]imitation to a small and chosen body of citizens … [is] to serve as the great purifier of both interest and opinion, to guard ‘against the confusion of a multitude'”. Hannah Arendt in On Revolution.

It’s not always easy for a representative to know what the people think, if they think something at all. It often happens that a representative guides, purifies or clarifies the thoughts of the people by presenting his own thoughts in a clear and concise way. At the next election, the people are of course free to express acceptance or rejection of these thoughts and to vote for or against the person defending them.

So it’s clear that the definition of the representative as an actor is a simplification. Representatives should be more than mere errand boys faithfully executing the will of their masters and speaking, not with their own voice, but with the voice of the voters. They are more than robots or parrots doing deeds and saying words that are not their own. Of course, a representative of the people “re-presents” someone, makes someone else present in parliament or in an executive function. He plays a part. He represents something that is pre-existent.

However, this is not always the case. What is represented often arises after and through the act of representation. By presenting his ideas in a clear and convincing way, the representative can convince the people to adopt his ideas. He can also try to add a certain clarity, direction, consistency and unity to the opinions of the voters. In the case of contradicting desires for example, he can establish a certain priority and favor one desire while putting another one temporarily aside. He decides an issue in the name of the undecided electorate torn between two conflicting desires (for example employment and limiting the arms trade) and defends this decision by giving clear arguments to the voters.

At the next election, the voters can always disavow the choices of the representatives, but then at least they are forced to decide what is their point of view, to make up their minds, to focus on one of their conflicting views and to end an internal conflict.

Politics should not always focus on every wish or follow every erratic movement in the opinions of the people. It should also try to guide these wishes by offering and forcing a clear choice. This means that it’s quite all right for a representative to follow his own judgment now and then instead of simply saying what his electors instructed him to say. This kind of independence is of course limited. It cannot be applied to fundamental opinions. For example, a representative chosen on a ticket of anti-racism cannot express racist ideas or execute racist policies while in office.

The simple model of democracy—the people making up their minds beforehand and choosing representatives who will faithfully implement their opinions—is sometimes a simplification of reality. The politician is often the midwife of the truth of society, in the words of Guéhenno, and shapes the will of the people. Politicians necessarily take over characteristics of the people and start to resemble the people, otherwise they cannot represent the people and the people will never support the politicians. However, the opposite is also true. The people often start to resemble the politicians because the politicians clarify the sometimes vague and contradictory opinions of the people.

If the representatives were only allowed to follow the instructions of the electorate, then the affairs of parliament would be no more than an exercise in arithmetic, a sum of opinions. Representatives in parliament could not and should not discuss, deliberate and convince each other. If a representative changes his opinions as a consequence of discussion and argumentation in parliament—and this happens very often, because otherwise discussion and argumentation would be useless—then his opinions are no longer those that won him the election and he no longer represents the people who elected him. If representatives must follow the instructions of the electorate in every case, then parliament cannot be a place where different opinions are juxtaposed and discussed and where people try to come to a common opinion based on argumentation rather than the coincidence of identical opinions.

Parliament is not a congress of ambassadors from different and hostile interests, which interests each must maintain, as an agent and advocate, against other agents and advocates; but Parliament is a deliberative assembly of one nation, with one interest, that of the whole—where not local purposes, not local prejudices, ought to guide, but the general good, resulting from the general reason of the whole. Edmund Burke

More posts in this blog series.

Lies, Damned Lies, and Statistics (16): Measuring Public Opinion in Dictatorships

Measuring human rights requires a certain level of respect for human rights (freedom to travel, freedom to speak, to interview etc.). Trying to measure human rights in situations characterized by the absence of freedom is quite difficult, and can even lead to unexpected results: the absence of (access to) good data may give the impression that things aren’t as bad as they really are. Conversely, when a measurement shows a deteriorating situation, the cause of this may simply be better access to better data. And this better access to better data may be the result of more openness in society. Deteriorating measurements may therefore signal an actual improvement. I gave an example of this dynamic here (it’s an example of statistics on violence against women).

Measuring public opinion in authoritarian countries is always difficult, but if you ask the public if they love or hate their government, it’s likely that you’ll have higher rates of “love” in the more authoritarian countries. After all, in those countries it can be pretty dangerous to tell someone in the street that you hate your government. They choose to lie and say that they approve. That’s the safest answer but probably in many cases not the real one. I don’t believe for a second that the percentage of people approving of their government is 19 times higher in Azerbaijan than in Ukraine, when Ukraine is in fact much more liberal than Azerbaijan.

In the words of Robert Coalson:

The Gallup chart is actually an index of fear. What it reflects is not so much attitudes toward the government as a willingness to openly express one’s attitudes toward the government. As one member of RFE/RL’s Azerbaijan Service told me, “If someone walked up to me in Baku and asked me what I thought about the government, I’d say it was great too”.

The Causes of Human Rights Violations (6): The Behavior of Corporations

Private companies, especially companies engaging in international trade and multinational companies, have duties in the field of human rights. They can violate human rights or they can act positively to protect them.

  • They are obliged to respect human rights, just as individuals or states. They can create labor conditions that respect human rights (fair wages, no child labor etc.).
  • Multinational companies should not use the competition between workers (for example workers from different countries with different levels of wages or different labor conditions) to force down wages or loosen labor regulations.
  • Multinational companies can be an example to local companies in the way they treat workers, and they can require that local companies that co-operate with them respect certain rules, such as the rules regarding labor conditions.
  • Also non-multi-national companies that engage in international trade can require that their suppliers respect human rights. They can always threaten to go to another supplier.
  • Companies have the technology, the know-how and the money that some governments need; hence they have the means to convince these governments to respect human rights.
  • It is in the interest of companies that countries respect human rights and the principles of democracy. They need the rule of law (which is a human right), predictability, law enforcement, stability, an effective judiciary etc. It is also in their interest that the local population can afford to buy their products or services. So they may even be able to promote economic rights.

Companies that violate rights, that trade with dictatorships or with other companies that do not respect human rights, become more and more sensitive to the negative consequences of their choices. Their image and profits can suffer. Their good name is economically more interesting than rights violations, certainly in the long term. Public opinion can turn against them and consumers can stop buying their products or services.

Banks and arms producing companies have a particular responsibility. Banks could monitor and possibly even freeze the assets of dictators. Arms companies should avoid delivering arms to dictators.