Measuring Human Rights (32): Assessing Advocacy and Policy by Way of Counterfactual Thinking

Human rights measurement is ultimately about levels of respect for human rights, but it can also be useful to try to measure the impact of human rights advocacy and policy on these levels. Both advocacy and policy (the difference being that the former is non-governmental) aim at improving levels of respect for human rights. Obviously, those levels don’t depend solely on advocacy or policy, but it’s reasonably to assume that they are to some extent dependent on those types of action. It’s hard – although not impossible – to imagine that millions of people and dozens of governments and international institutions would engage in pointless activity.

The question is then: to what extent exactly? How much do advocacy and policy help? The problem in answering this question is that we won’t necessarily learn a lot by simply looking at the levels and how they evolve. Not only is there the difficulty of comparing different possible causes; a flat trend line – or even a declining trend line – may cover up how much more awful things would have been without advocacy and policy. Levels of respect may very well stay as they are or even worsen while advocacy and policy are relatively successful because the levels without advocacy and policy would have been even lower.

Of course, it’s very hard to quantify this. If there’s improvement, you can at least try to sort out the relative contribution of different causes. If things don’t improve or even worsen, then the only way to measure the effect of advocacy and policy is the use of counterfactual thinking. And that’s a problem. How bad (or good?) would things have been without advocacy and policy? We can’t redo a part of a country’s history to test what would have happened with other choices. We can speculate about the answer to “what if” questions but since we can’t experiment we’re left with a lot of uncertainty. What if Hitler had won the war? Or had been admitted to art school? Fun questions to try and answer, but the answers won’t tell us much about the real world, unfortunately. If they did, we would know what to do.

More posts in this series are here.