The 2018 Ibrahim Index of African Governance (IIAG), the 12th iteration of this Index, measures the performance of 54 African countries in the four categories, 14 sub-categories and 102 indicators that make up the Overall Governance score over a ten-year period (2008-2017). This blog post aims to provide some useful guidance regarding how to read the results from the 2018 IIAG dataset.
Scores, ranks and trends
There are three main types of IIAG results: scores, ranks and trends. When assessing the performance of a country, region or group, Index users are encouraged to take the three types of results into account, as they provide each other with context.
For instance, despite experiencing the third largest improvement over the decade in Overall Governance in the 2017 IIAG, Zimbabwe still ranked 40th (out of 54) in 2016 (with a score of 45.4 out of 100.0). Similarly, even though Botswana ranked 2nd in the Safety & Rule of Law category in 2016 (with a score of 81.6), its trend over the decade was negative.
Levels and aggregation
The IIAG is a composite Index and this means there are many underlying scores that constitute the composite scores. The benefit of the IIAG structure is that country or group performance can be assessed holistically through the broader measures, but users can also drill down to assess performance in specific governance areas.
It is important to note that while some underlying indicators may mirror a country’s performance in their respective composite measure, others may also be opposing that trajectory. For instance, even though the 2017 IIAG dataset showed that the African average score for Rights improved over the decade, four of its six underlying indicators showed a decline, hampering further progress at the sub-category level.
Trend classifications
The 2018 IIAG dataset also contains annual average trends. Continuing with the analysis approach introduced for the first time in last year’s Index, trend classifications are assigned based on the performance of a country, region or group in the most recent five years (2013-2017) compared to their performance over the past decade (2008-2017).
While a country can either experience an improvement or a deterioration over the last ten years, the five-year trend may show the opposite trajectory. Moreover, a country’s pace of improvement or deterioration over the decade can also accelerate in the latest five years. This approach provides Index users with a more nuanced assessment and sheds light on the non-linear nature of governance performance.
The annual average trends for the ten-year and five-year periods are calculated as follows:
- the annual average trend for the ten-year period is the total change in score between 2008 and 2017, divided by nine (the number of annual time periods experienced)
- the annual average trend for the five-year period is the total change in score between 2013 and 2017, divided by four (the number of annual time periods experienced).
The resulting trend classifications are six: Increasing Improvement, Slowing Improvement, Warning Signs, Bouncing Back, Slowing Deterioration or Increasing Deterioration. These classifications are applied to all measures in the IIAG. Depending on the size and direction of the annual average trends in these periods, countries and groups are assigned these classifications, which are best explained as follows:
Correlations between IIAG measures: the pillars of good governance
The 2018 IIAG publication will also explore the correlations between different measures in the Index dataset. While acknowledging that correlation does not imply causation and only indicates the strength and direction of the relationships between two variables, correlation coefficients can be a valuable tool to know what the best performing countries in any given measure have in common. For instance, knowing which IIAG indicators are the most highly correlated with Overall Governance can provide governments with actionable insights about which areas are fundamental to improve to obtain a higher Overall Governance score.
Quantifying uncertainty
Two assumptions are made about errors in the IIAG. Firstly, each indicator score constitutes a measurement of its true sub-category score plus some measurement error. However, when one indicator misestimates its true sub-category score, there is no reason to believe that another indicator from a different data source will do the same. As errors are assumed to be unrelated, they will cancel each other out when the indicator scores are aggregated. The more indicators are added, the more likely it is that the error gets closer to zero. Secondly, the imputation of missing values also generates some missingness error.
The Foundation publishes standard errors, which are obtained by combining the measurement and missingness errors, as well as confidence intervals alongside the Overall Governance and category scores to quantify the uncertainty of estimates. The standard errors and confidence intervals allow Index users to ascertain which changes in the IIAG performance over time can be confidently treated as actual changes in the state of governance, hence being sufficiently sizeable to be able to ascribe a high likelihood to such change being statistically significant.
Index users are encouraged to consider standard errors and confidence intervals when interpreting changes in country scores over time and differences between country scores. For any time period selected, the 2018 IIAG Excel data portal will contain information on the significant changes at the Overall Governance and category levels with different levels of confidence (80%, 85%, 90% and 95%).
