Measuring Democracy (8): A Multidimensional Measurement

Any attempt to measure the degree of democracy in a country should take into account the fact that democracy is something multidimensional. It won’t suffice to measure elections, not even the different aspects of elections such as frequency, participation, fairness, transparency etc. It takes more than fair and inclusive elections to have a democracy. Of course, the theoretical ideal of democracy is a controversial notion, so we won’t be able to agree on all the necessary dimensions or elements of a true democracy. Still, you can’t escape this problem if you want to build a measurement system: measuring something means deciding which parts of it are worth measuring.

You would also do best to take a maximalist approach: leaving out too many characteristics would allow many or even all countries to qualify as fully democratic and would make it impossible to differentiate between the different levels or the different quality of democracy across countries. A measurement system is useful precisely because it offers distinctions and detailed rankings and because it makes it possible to determine the distance to an ideal, whatever the nature of the ideal. Obviously, a maximalist approach is by definition more controversial than a minimal one. Everyone agrees that you can’t have a democracy without elections (or, better, without voting more generally). Whether strong free speech rights and an independent judiciary are necessary is less clear. And the same is true for other potential attributes of democracy.

Once you’ve determined what you believe are necessary attributes you can start to measure the extent at which they are present in different countries. Hence, your measurement will look like a set of sliding scales. With all the markers on the right side in the case of a non-existing ideal democracy, and all the markers on the left side in the unfortunately very real case of total absence of democracy.

(The aggregation of these scales into a total country score is another matter that I’ve discussed elsewhere).

Some candidates of attributes are:

  • Does a country include more or less people in the right to have a democratic say? How high is the voting age? Are criminals excluded from the vote, even after they have served their sentence? Are immigrants without citizenship excluded? Are there conditions attached to the right to vote (such as property, education, gender etc.)?
  • Does a country include more or less topics in the right to a democratic say? Are voters not allowed to have a say about the affairs of the military, or about policies that have an impact on the rights of minorities? Does the judiciary have a right to judicial review of democratically approved laws?
  • Does a country include more or less positions in the right to a democratic say? Can voters elect the president, judges, prosecutors, mayors, etc., or only parliamentarians? Can they elect local office holders? Does a country have a federalist structure with important powers at the local or state level?
  • Does a country impose qualified majorities for certain topics or positions? Do voters have to approve certain measures with a two-thirds supermajority?
  • Does a country provide more or less ways to express a democratic say? Can voters only elect officials or can they also vote on issues in referenda?
  • Does a country impose more or less restrictions on the formation of a democratic say? Are free speech rights and assembly and association rights respected?
  • Does a country accept more or less imbalances of power in the formation of a democratic say? Are there campaign financing rules?
  • Does a country show more or less respect for the expression of a democratic say? How much corruption is there? Is the judiciary independent?

A “more” score on any of these attributes will push up the total “democracy score” for a country. At least it seems so, if not for the conclusion that all these complications in the measurement system are still not enough. We need to go further and add additional dimensions. For example, one can argue that we shouldn’t define democracy solely on the basis of the right to a democratic say, not even if we render this right as complex as we did above. A democracy should, ideally, also be a stable form of government, and allowing people to decide about the fundamental rights of minorities is an expression of the right to a democratic say but it is not in the long term interest of democracy. Those minorities will ultimately rebel against this tyranny of the majority and cause havoc for everyone.

More posts in this series are here.

Measuring Democracy (7): Some Technical Difficulties

Suppose you want to construct a democracy index measuring the level or lack of democracy in different countries in the world. The normal thing to do is to select some supposedly essential characteristics or attributes of democracy and try to measure the level or presence of those. So, for example, you may select free speech, elections, judicial independence and a number of other characteristics. Some of those are perhaps already measured and you can simply take those measurements. For others, you may have to set up your own measurement (e.g. a survey, analysis of newspapers or official documents etc.), or use a proxy.

In any case, you’ll end up with different datasets on different attributes of democracy, and you’ll have to bring those datasets together somehow in order to make your overall index, you single country-level democracy score. The problem is that the datasets contain different kinds of scales which cannot as such be aggregated into a global index. The scales and the values in the scales have to be normalized, i.e. translated into a common metric.

normalized value = raw value/maximum raw value

First, however, you have to rescale some existing scales so that they start at 0 – in other words, so that the lowest score is 0 (instead of starting at 1 for example, or at -10 such as the Polity IV scale). This way, all scales will have a normalized range from 0 to 1; 0 being the negation or total absence of the attribute; 1 being the complete and perfect protection or presence of the attribute.

What about weighting the different attributes? Some may be more important for a democracy than others. However, introducing weights in this way inevitably means introducing value judgments. While value judgments can’t be avoided (they’ll pop up at the moment of the selection of the attributes as well, for example), they can be minimized. If you choose not to use weighting, you consider all attributes to be equally important, which is a view that can be defended given the often interdependent nature of the attributes of democracy (an independent judiciary for example will likely not survive without a free press).

Once the different data sources are translated into normalized scales and, if necessary, weighted appropriately, they have to be aggregated in order to calculate the global index of quality of democracy. One possible aggregation rule would be this:

global index = source 1 * source 2 * ... * source n.

So a simple multiplication. But that would mean that a value of 0 for one attribute results in labeling the country as a whole as having 0 democratic quality. This is counter-intuitive, even with the assumption of equal importance of all attributes. Hence, a better aggregation rule is the geometric or arithmetic mean (or perhaps the median).

However, there’s also a problem with averages: low scores on one attribute can be compensated by high scores on another. So very different democracies can have the same score. Also, within one country, a high score on suffrage rights but 0 on actual participation would give a medium democracy score, whereas in reality we wouldn’t want to call this country democratic at all (the score should be 0 or close to 0). Perhaps we can’t avoid weights after all.

More posts in this series are here.

Measuring Democracy (6): Three Waves of Democratization According to Polity IV and Google Ngrams

Following Samuel Huntington, many political scientists believe that there have been three waves of democratization in recent history. The first wave of democracy began in the early 19th century when suffrage was gradually extended to disenfranchised groups of citizens. At its peak, however, there were only about 20 democracies in the world during this first wave. After WWI, with the rise of fascism and communism, the wave started to ebb, and this ebb lasted until the end of WWII. The second wave began following the Allied victory in World War II. This wave culminated in the 1960s with around 30 democracies in the world. The third wave started in the 1970s and really took off in the late 1980s, with the democratization of Latin America and the fall of the Berlin Wall. Today there are some 60 democracies in the world.

Maybe recent vents in the Maghreb and the Middle East are the start of a fourth wave, now focused on Arab countries.

Those numbers I cited above come from one of the two major democracy indexes, namely Polity IV. Polity IV gives countries a score ranging from -10 to +10; the numbers above are of countries achieving the rather ambitious score of +8 or higher (in other interpretations of the Polity IV score, +6 is already a democracy). Freedom House, the other index, usually gives a higher number of democracies, but is only available for the most recent decades. I don’t want to discuss the relative merits of either measurement system in the current post. Let’s just assume, arguendo, that Polity IV is a good measure (Freedom House probably measures something a bit different). In the graph below, the green line represents the Polity IV score (number of countries with a score of +8 or more):

vv

The three waves are clearly visible in the green line. Although some have expressed doubts about the quality of Huntington’s work and the reality of the three waves (see here for instance), there does seem to be at least some truth in the metaphor.

I’ve also included in the graph above the results of a search in Google’s Ngram tool. I searched for “democracy” (blue line) and “democratic” (red line) (democratic without a capital D because I don’t want results including mentions of the Democratic Party). As you may know, this tool allows you to calculate the frequency of keywords in the millions of books available in Google’s book collection. Such frequencies can be thought of as approximations of the general use and popularity of a word at a certain time. One can assume that when there’s a wave of democratization there’s also an uptick in the frequency of the use of word such as “democracy”.

I find it interesting that both the first and the third wave of democratization are reflected in a rising popularity of the words “democracy” and “democratic”, but not the second wave. When the number of democracies was at its lowest point in the 30s and 40s, talk about democracy was most common, more common even than today. And the interest in democracy decreased steadily from the 50s until the 80s, while the number of democracies rose during those decades.

More posts in this series here.

Measuring Democracy (3): But What Kind of Democracy?

Those who want to measure whether countries are democratic or not, or want the measure to what degree countries are democratic, necessarily have to answer the question “what is democracy?”. You can’t start to measure democracy until you have answered this question, as in general you can’t start to measure anything until you have decided what it is you want to measure.

Two approaches to measuring democracy

As the concept of democracy is highly contestable – almost everyone has a different view on what it means to call a country a democracy, or to call it more or less democratic than another – it’s not surprising to see that most of the research projects that have attempted to measure democracy – such as Polity IV, Freedom House etc. – have chosen a different definition of democracy, and are, therefore, actually measuring something different. I don’t intend to give an overview of the differences between all these measures here (this is a decent attempt). What I want to do here is highlight the pros and cons of two extremely different approaches: the minimalist and the maximalist one. The former could, for example, view democracy as no more than a system of regular elections, and measure simply the presence or absence of elections in different countries. The latter, on the other hand, could include in its definition of democracy stuff like rights protections, freedom of the press, division of powers etc., and measure the presence or absence of all of these things, and aggregate the different scores in order to decide whether a country is democratic or not, and to what extent.

When measuring the democratic nature of different countries (and of course comparing them), should we use a minimalist or maximalist definition of democracy? Here are some pros and cons of either approach.

Differentiation

A minimalist definition makes it very difficult to differentiate between countries. It would make it possible to distinguish democracies (minimally defined) from non-democracies, but it wouldn’t allow to measure the degree of democracy of a given country. I believe an ordinal scale with different ranks for different levels of quality of democracy in different countries (ranging from extremely poor quality, i.e. non-democracies, to perfect democracies) is more interesting than a binary scale limited to democracy/non-democracy. The use of a maximalist definition of democracy would make it possible to rank all types of regimes on such an ordinal scale. A maximalist definition of democracy would include a relatively large number of necessary attributes of democracy, and the combination of presence/absence/partial development of each attribute would almost make it possible to give each country a unique rank in the ordinal scale. Such a wide-ranging differentiation is an advantage for progress analysis. A binary scale does not give any information on the quality of democracy. Hence, it would be better to speak of measuring democratization rather than measuring democracy. And democratization not only in the sense of a transition from authoritarian to democratic governance, but also in the sense of progress towards a deepening of democratic rule.

A minimalist definition of democracy necessarily focuses on just a few attributes of democracy. As a result, it is impossible to differentiate between degrees of “democraticness” of different countries. Moreover, the chosen attributes may not be typical of or exclusive to democracy (such as good governance or citizen influence), and may not include some necessary attributes. For example, Polity IV, perhaps the most widely used measure of democracy, does not sufficiently incorporate actual citizen participation, as opposed to the mere right of citizens to participate. I think it’s fair to say that a country that gives its citizens the right to vote but doesn’t actually have many citizens voting, can hardly be called a democracy.

Acceptability of the measurement vs controversy

A disadvantage of maximalism is that the measurement will be more open to controversy. The more attributes of democracy are included in the measure, the higher the risk of disagreement on the model of democracy. As said above, people have different ideas about the number and type of necessary attributes of a democracy, even of an ideal democracy. If the only attribute of democracy retained in the analysis is regular elections, then there will be no controversy since few people would reject this attribute.

Balancing

So we have to balance meaning against acceptability: a measurement system that is maximalist offers a lot of information and the possibility to compare countries beyond the simple dichotomy of democracy/non-democracy, but it may be rejected by those who claim that this system is not measuring democracy as they understand the word. A minimalist system, on the other hand, will measure something that is useful for many people – no one will contest that elections are necessary for democracy, for instance – but will also reduce the utility of the measurement results because it doesn’t yield a lot of information about countries.

Measuring Democracy (2): Polity IV, and Some of Its Problems

Polity IV is, like Freedom House and others, a project ranking countries according to their political regime type. It’s extensively used in comparative and causal analysis that require a distinction between democracies and non-democracies, partly because its time series start from the year 1800.

Its

perspective envisions a spectrum of governing authority that spans from fully institutionalized autocracies through mixed, or incoherent, authority regimes (termed “anocracies”) to fully institutionalized democracies. The “Polity Score” captures this regime authority spectrum on a 21-point scale ranging from -10 (hereditary monarchy) to +10 (consolidated democracy). (source)

The Polity Score is the aggregate of 6 component measures that aim to record what are called key qualities of democracies: executive recruitment, constraints on executive authority, and political competition.

However, it seems that Polity IV doesn’t adequately measure what it claims to measure. Its concept of democracy is quite thin, resulting in a fair number of “perfect democracies”, whereas we all know that there is no such thing in the world we live in. And other countries, which are obviously dictatorial, are classified as fairly democratic. A quote from this paper (which is an attempt to improve Polity IV):

Polity’s 21-point democracy/autocracy scale, illustrated by the dashed line [in the figure below], tracks the major changes in British political history, but only roughly. The Reform Bill of 1832 revised a complicated system of determining the franchise by increasing the number of voters from 500,000 to 813,000. Despite the modesty of this expansion, changes in the Polity Score for Britain give a sense of greatly expanded democracy, moving from a -2 (democracy=4, autocracy=6) to a +3 (democracy=6, autocracy=3).

However, … only six percent of the adult population voted even after the reform.

While the male franchise had broadened considerably by 1884, suffrage still excluded agricultural workers and servants. Actual voter turnout reached 12% of the population only in the election of 1885 before falling, and didn’t return to that level again until 1918. All the while, Polity scores for executive recruitment and competition increased while institutionalized autocracy decreased. In 1880 the Polity democracy score stood at 7 (autocracy=0). By 1901 the democracy score rose to 8 and by 1922 Polity suggests that Britain was a “perfect 10” democracy, even though full male suffrage was not achieved until 1918 and full female suffrage until 1928.

Britain has received the highest democracy rating ever since, even though the voting rate has never exceeded 60% of the adult population.

The high scores that Britain receives from 1880 on are misleading and, with respect to changes in participation, mistimed. As Figure 1 illustrates, participation doubled during a period Polity records as unchanged and doubled again during a modest 2 point move in Polity.

The racial exclusion in South Africa also demonstrates the danger of conceiving democracy without taking account of the breadth of citizen participation. According to Polity, South Africa was a relatively stable democracy from 1910 until 1989. It was coded a 7 out of 10 on democracy and a 3 of 10 on autocracy, bringing its score to +4. A positive score is surprising because it ignores the exclusion of the 90 percent of the population that did not – most could not – vote.

Switzerland, our final example, has scored a perfect 10 out of 10 on democracy in the Polity dataset since 1848, even though women – roughly half the population – were not granted the right to vote until 1971, 123 years later. Furthermore, electoral turnout has hovered around 30% recently, despite virtually universal suffrage. One reason is that Switzerland’s collective executive is an organizational form that diminishes voter motivation by minimizing the significance of election outcomes. Surely such a system should be regarded as less democratic than one in which most citizens participate in elections that actually make a difference in the leadership and policies of the nation.