(In)Consistency between international corruption indicators

Note: Results from this post are presented more systematically in the paper “Marketplace of indicators: Inconsistencies between country trends of measures of governance” co-authored with Paul Bürkner and available on SocArXiv: https://osf.io/preprints/socarxiv/u8gsc/.


Measuring corruption is hard, especially if one is interested in having corruption indicators that are comparable across countries and over time. Arguably the most famous corruption ranking is the Corruption Perceptions Index published annually by Transparency International, but it can’t be used for over-time comparisons (cf. Transparency International 2020, p. 26).

Other corruption indicators include:

  1. Bayesian Corruption Indicator (BCI) published in the Quality of Government Dataset, QOG

  2. Political Corruption Index by Varieties of Democracy, V-Dem

  3. Control of Corruption (CC) in the World Bank’s Worldwide Governance Indicators, WGI

If these indicators measure the same concept, they should be correlated and change similarly over time, even if they are based on slightly different conceptualizations and definitions, rely on different source data, or use different calibration methods.

There exist reports that compare corruption indicators, but they describe the different methodologies and data sources, and at most calculate overall correlations, which are always very strong:

World Bank’s Policy Resesarch Paper Can We Measure the Power of the Grabbing Hand? A Comparative Analysis of Different Indicators of Corruption by Alexander Hamilton and Craig Hammer.

Measuring Corruption: A Comparison Between the Transparency International’s Corruption Perceptions Index and the World Bank’s Worldwide Governance Indicators by Anja Rohwer.

Our World in Data has a report on Corruption by Esteban Ortiz-Ospina and Max Roser.

Only Standaert (2015) notes the correlation between within-country changes in corruption as measured by the Bayesian Corruption Indicator he proposes, as well as the Corruption Perceptions Index and the Worldwide Governance Indicators. While overall correlations exceed 0.9, correlations between deviations from country averages range between 0.2 and 0.35, which means they are rather weak.

Let’s see how the three indicators compare.

Scatter plots

The correlation between the Bayesian Corruption Indicator (QOG) and Control of Corruption (WGI) is the strongest, because they rely on the same data but different estimation / scaling techniques (cf. Standaert 2015).

The association between the Political Corruption Index (V-Dem) and the Control of Corruption indicator (WGI) seems strong as well, but not necessarily linear and less consistent in the lower ranges of corruption.

The association between the Political Corruption Index (V-Dem) and the Bayesian Corruption Indicator (QOG) is surprisingly weak. For medium levels of corruption according to BCI, i.e. around 0.5, the V-Dem indicator ranges between around 0.12 to almost 1, i.e. over almost its entire range.


Let’s look at pairwise correlations to get the numbers.

First, correlations between levels of the three indicators for all country-years (some 4500 observations):

qog_bci wgi_cc vdem_corr
qog_bci 1.000 0.91 0.816
wgi_cc 0.910 1.00 0.900
vdem_corr 0.816 0.90 1.000

Next, pairwise correlations between country means for 173 countries:

qog_bci_mean wgi_cc_mean vdem_corr_mean
qog_bci_mean 1.000 0.925 0.833
wgi_cc_mean 0.925 1.000 0.921
vdem_corr_mean 0.833 0.921 1.000

Now, pairwise correlations between within-country deviations from the means:

qog_bci_diff wgi_cc_diff vdem_corr_diff
qog_bci_diff 1.000 0.341 0.088
wgi_cc_diff 0.341 1.000 0.303
vdem_corr_diff 0.088 0.303 1.000

These last correlarions are very weak. As shown in the scatterplot, there is little consistency between the de-meaned V-Dem Political Corruption Index and the Bayesian Corruption Indicator.

Looking at continents separately, correlations between the Bayesian Corruption Indicator and the V-Dem Corruption Index are negligible in Africa, the Americas, and Europe, and small in Asia and Oceania. In Europe, the correlation is actually negative.

## `summarise()` ungrouping output (override with `.groups` argument)
Pairwise correlations
Continent N country-years N countries QOG - V-Dem WOG - WGI V-Dem - WGI
Africa 1370 54 0.037 0.300 0.293
Americas 750 27 0.034 0.232 0.220
Asia 1220 47 0.214 0.372 0.433
Europe 991 39 -0.023 0.480 0.135
Oceania 150 6 0.234 -0.053 -0.053
comments powered by Disqus