Lies, Damned Lies, and Statistics (11): Polarized Statistics as a Result of Self-Selection

One of the most important things in the design of an opinion survey – and opinion surveys are a common tool in data gathering in the field of human rights – is the definition of the sample of people who will be interviewed. We can only assume that the answers given by the people in the sample are representative of the opinions of the entire population if the sample is a fully random subset of the population – that means that every person in the population should have an equal chance of being part of the survey group.

Unfortunately, many surveys depend on self-selection – people get to decide themselves if they cooperate – and self-selection distorts the randomness of the sample:

Those individuals who are highly motivated to respond, typically individuals who have strong opinions, are overrepresented, and individuals that are indifferent or apathetic are less likely to respond. This often leads to a polarization of responses with extreme perspectives being given a disproportionate weight in the summary. (source)

Self-selection is almost always a problem in online surveys (of the PollDaddy variety), phone-in surveys for television or radio shows, and so-called “red-button” surveys in which people vote with the remote control of their television set. However, it can also occur in more traditional types of surveys. When you survey the population of a brutal dictatorial state (if you get the chance) and ask the people about their freedoms and rights, many will deselect themselves: they will refuse to cooperate with the survey for fear of the consequences.

When we limit ourselves to the effects of self-selection (or self-deselection) in democratic states, we may find that this has something to do with the often ugly and stupid “us-and-them” character of much of contemporary politics. There seems to be less and less room for middle ground, compromise or nuance.

Freedom of Expression, or a Duty of Expression?

I often have the impression that people transform the right to free expression into a duty to free expression. And I don’t think that’s a good thing. For example, Muslims in western countries are often told that they should distance themselves from the more violent members of their religion. We require them to speak out against Muslim terrorism.

Another example: politicians, especially in the U.S., are required to speak out on a number of subjects, e.g. abortion, same-sex marriage, their faith in God etc. As if it would be a disaster to elect a politician who happens to doubt about abortion. After all, many people do (myself included).

A somewhat exaggerated view on democratic transparency is undoubtedly a small part of the explanation for this. Democracy can’t function without public knowledge of politicians’ opinions, or without some sense of what our fellow citizens believe (part of democracy is group formation, and group formation is based on discussion and persuasion; and you can’t persuade someone if you don’t know what he or she believes).

But the most important cause of this “duty of expression” is, I think, the manichean nature of contemporary politics. Every issue is painted in black and white, good and evil, for or against. We force people to express themselves on issues so that we can see if they are with us or against us. And if someone expresses him or herself in a nuanced way we automatically assume that he or she takes a position opposite from our own. For example, if Muslims reject Islamic terrorism but at the same time point to the situation in Palestine, we assume that they really think terrorism is OK, or justifiable given certain circumstances. We can’t accept muddled or nuanced middle ground positions, or positions which change according to the circumstances. Gray isn’t an option.

Clarity, simplicity and certainty are important human objectives, but often they aren’t appropriate in thinking. Of course, sometimes manicheism is the only possible position: you either believe the holocaust is a fact of history or you don’t; there’s no middle ground, and those who don’t believe in it are either stupid or evil. But when it comes to political or moral opinions (rather than facts), those who really think about them often find themselves occupying a gray, complex and uncertain position.

I suspect that the difficulty to let go of manicheism and to accept uncertainty and nuance has something to do with the nature of democratic politics. It’s hard to vote for nuance, and easy to vote for or against a clear and simple proposition. And simple propositions get more attention, sell better and make it easier to mobilize large constituencies (see the cartoon below). But then again, when we look at political reality, manicheism is much more common in autocratic societies. The public debate on issues which is made possible by democratic societies forces nuance to appear.

The difficulty to let go of manicheism also has something to do with the fear of the other extreme: the paralysis that follows from endless nuancing and thinking. Politics is a realm where decisions have to be taken, contrary to philosophy where thinking is unending in principle.

However, it doesn’t follow from this that decisiveness has to be manicheism. Decisions can be based on nuanced thinking. The risk of paralysis is averted by the realization that our decisions, often taken under the pressure of urgency, are necessary yet provisional, based on the best thinking available at the time, and open to revision when time has improved our thinking.