For Good Measure Read online

  The Rationale of Goal Setting

  There are at least two questions we can ask about the SDGs (as indeed about the MDGs). First, in what sense are they goals of the development process? Second, how, if at all, does goal setting aid the development process? Let us take these questions in turn.

  Are the SDGs truly the goals of development? Following Bourguignon et al. (2010), we can translate their questions on the MDGs to questions for the SDGs: (1) Do the SDGs command (close to) universal agreement? (2) Are the SDGs the final goals of development? Are they inputs, outcomes, or outputs (intermediate variables of interest mainly because of their relationship to some ultimate objective)?1 (3) How are we to weigh the SDGs relative to each other?

  The first issue is perhaps easiest to answer in a formal and substantive sense. In a formal sense, the SDGs have been signed off on by political leaders of almost all of the countries in the world, and are encapsulated in a resolution of the United Nations General Assembly. Agreement does not get much more universal than that in an international setting. In a substantive sense, the SDGs as a package are likely to command consensus precisely because they are so wide ranging, so that many perspectives on development and well-being are brought into the 17 goals and 169 targets. But it is this comprehensiveness that leads to the next question, on what exactly they represent.

  On the second issue, the 17 SDGs (both in the general pronouncement of them, and in their further specification into detailed targets) are a mixture of the causal chain from inputs to outputs to outcomes. Take, for example, Goal 8: “Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all.” This goal, and its associated targets, mixes up inputs, outputs, and outcomes, especially if we think back to the literature that took us away from GDP in the first place. Following Sen (1985) and Stiglitz, Sen, and Fitoussi (2009), GDP is seen as an input, a means to an end rather than an end in itself. Yet target 8.1 is “Sustain per capita economic growth in accordance with national circumstances and, in particular, at least 7% gross domestic product growth per annum in the least developed countries.” Target 8.5 comes much closer to a final outcome variable in specifying employment and pay equality as objectives: “By 2030, achieve full and productive employment and decent work for all women and men, including for young people and persons with disabilities, and equal pay for work of equal value.”

  In the MDGs, Goal 8 on partnerships was often criticized for being a catchall with little structure. In the SDGs, perhaps Goal 17, “Strengthen the means of implementation and revitalize the global partnership for sustainable development,” takes on that role. This goal has no fewer than 19 targets, grouped under the subheadings of Finance, Technology, Capacity Building, Trade, and Systemic Issues. Specific targets include such disparate components as, “Mobilize additional financial resources for developing countries from multiple sources”; “Significantly increase the exports of developing countries, in particular with a view to doubling the least developed countries’ share of global exports by 2020”; and “By 2030, build on existing initiatives to develop measurements of progress on sustainable development that complement gross domestic product, and support statistical capacity-building in developing countries.” The last of these is relevant to our discussion below, in the section “Measurement at the Global Level,” but the sheer complexity of Goal 17 is a testament to how the SDG process has catered to a very wide range of constituencies who have focused on their goal or target (be it inequality reduction, or primary education, or employment generation, or water and sanitation, etc.) and claim some legitimacy from it being present in the list of SDGs, whether it is as input, output, or outcome.

  On the third issue raised by Bourguignon et al. (2010), the large number of goals and targets, spread out along the input-output-outcome chain, raises obvious questions of evaluation and assessment. Supposing even that we were to agree on genuine outcome variables focused on human well-being, how are we to address the inevitable trade-offs? In their discussion of the MDGs, Bourguignon et al. pose the issue as follows: “In a world of limited resources, it is likely that often progress on one MDG will have to be at the expense or postponement of another. Suppose country A rushes ahead on MDGx but falls behind on MDGy, whereas for country B the reverse is true. How is the MDG performance of the two countries to be assessed? Whose trade-off weights are to be used—country A’s, country B’s, or a universal trade-off determined internationally?”

  The same questions can be asked with “SDG” substituted for “MDG.” The issue has become, if anything, even sharper with the broadening of the scope from the 8 MDGs to the 17 SDGs and associated targets. As argued in the next section, the issue is perhaps best resolved at the national level, by selecting which of the SDG targets and goals is most relevant in the specific country context, but this does not, of course, avoid the problem of trade-offs.

  The second major question posed at the start of this section is how, if at all, does goal setting aid the development process? The answers to this question can be given at both the international level and the national level (Bourguignon et al., 2010). At the international level, goal setting can be useful from the technical point of view, quantifying the resources needed to achieve the selected goals. Thus, for example, Sachs (2005) used the MDG targets to estimate that, in order to achieve those goals, development assistance would have needed to increase to around $200 billion annually (compared with its level of around $65 billion in the early 2000s). Detailed sector-by-sector technical calculations underlie this overall figure, and the sector-specific goals and targets again play a role in guiding and focusing these technical calculations.

  UN Secretary General Ban Ki-moon set out a second use of goal setting in the international context when, as quoted earlier, he said: “[The MDGs] generated new and innovative partnerships, galvanized public opinion and showed the immense value of setting ambitious goals.” They do this, for instance, through the normsetting role noted earlier. Given the difficulties of attribution, quantitative assessment of such claims is not easy. The assessments tend to be more qualitative in nature, as in McCarthy (2013):

  The greatest MDG successes undoubtedly concern health. The MDGs have invigorated multilateral institutions, such as the GAVI Alliance (formerly called the Global Alliance for Vaccines and Immunization), which seeks to achieve MDGs “by focusing on performance, outcomes and results.” The goals have also inspired a huge increase in private-sector aid. Ray Chambers, a respected philanthropist and co-founder of a New York private equity firm, first learned of the goals in 2005. Since then, working with Sachs and others, Chambers has co-ordinated a worldwide coalition of policy, business, and NGO leaders in an effort to help the developing world meet the goal for malarial treatment and prevention. Thanks in part to this global effort, malaria-related mortality has dropped by approximately 25 percent since 2000, with most of those gains probably occurring since 2005. Many pharmaceutical companies have also put forth major efforts to make their medicines more widely available in poor countries, and new initiatives are continuing to take shape. The MDG Health Alliance, founded in 2011, is comprised of business and NGO leaders around the world working toward the MDG health targets, including the elimination of mother-to-child HIV transmission.

  Many sectors can no doubt claim successes of this type, which might help explain the dramatic increases in goals and targets by the time the MDGs were transformed into the SDGs.

  Other reasons for the dramatic increase in goals and targets in the SDGs are (1) the inclusive process used to develop the SDGs; and (2) the SDGs’ broadening of perspective to include the environment and human rights agendas. Perhaps the appropriate way to think about the SDGs is indeed in a broad perspective, as a platform that provides global civil society with a base from which to organize around one of the many issues in the SDGs. It also provides national civil society organizations an entry point in the dialogue with their own governments. The fact that the SDGs are sancti
oned, after a fashion, by the community of world leaders gives national civil society a starting point in their national organizing, if organizations care to use them in that way, although one danger is that the discussion focuses so much on measurement that discussion on how to actually achieve the goals gets drowned out. But what this highlights is that, ultimately, the SDGs have to be brought to the national level, and be translated into specific goals and targets as a compact between governments and their populations. Here, technical operationalization and political salience are both needed to go from 17 goals and 169 targets to a dashboard that can capture key national political concerns and can be monitored and communicated easily.

  Implications for National Policy

  A heterogeneous mixture of 17 goals and 169 targets, negotiated between and balancing the interests of a wide range of global groups, cannot provide specific guidance for national policy. That guidance has to come from national concerns and national processes, although the SDGs can provide a useful frame of reference as needed. In the opening statement to the HLEG workshop on “Measurement of Well-Being and Development in Africa,”2 Ebrahim Patel, South Africa’s Minister of Economic Development and one of the co-authors of this chapter, posed two questions as a national policy-maker, recognizing both the centrality of GDP and the depth of its problems:

  • Can we find a single composite index to replace GDP?

  • If not, how big should the dashboard of indicators be and what should be on it (apart from GDP or, as alternatives, measures of household income or consumption)?

  As argued in Stiglitz, Sen, and Fitoussi (2009), GDP has been misused. In his presentation to the HLEG workshop, Lorenzo Fioramonti presented several directions in which GDP could be modified and supplemented, or even supplanted, from an African perspective.3 As he argued, GDP has become a “proxy for everything.” However, this could be because, in effect, GDP has stepped into a vacuum because of its simplicity and its correlation with at least some other dimensions of well-being. Despite its weaknesses, GDP has proved useful as a practical tool to policy-makers. While the critiques of GDP have been sharp, proposals to replace it have been less sharp—as reflected perhaps in the 17 goals and 169 targets that have emerged from the SDG process. This broad a range of goals and targets cannot make for practical policy-making.

  So, should GDP be replaced by an alternative composite index? There are, of course, many possible candidates. Sticking initially to the income sphere, we could consider measures of national poverty, although there are many possible poverty indices that can be presented, ranging from absolute poverty to relative poverty. Or we could, still in the income domain, rely on a measure of per capita national income corrected for income inequality (e.g., we could use, as our composite index, per capita income multiplied by one minus the Gini coefficient); then if income inequality rises holding GDP constant, “corrected” GDP would fall. But even here, Joseph Stiglitz argued, in his presentation to the HLEG workshop, the Gini coefficient may be too simple a measure of inequality, hiding important movements within the income distribution (for example, changes in the income shares at the very top of the income distribution).4

  But all of this is still in the income domain. Various versions of the HDI—starting from the basic one that takes a simple average of per capita income, literacy, and life expectancy—have attempted to move beyond the income space. As noted earlier, the HDI proved quite successful in the international domain in setting up comparisons across countries and giving ammunition to each country’s civil society to spur healthy competition between governments to advance on the components of the HDI. Of course, the components in the basic HDI are national averages and do not take into account the distribution around the average. To address this issue, more sophisticated and distribution-sensitive component values can be developed before averaging across the three dimensions. The “inequality-adjusted HDI” (UNDP, 2016) can give markedly different rankings and, for some purposes, can become a focal point of norm setting. However, the greater the sophistication of each sub-index, and thus of the index as a whole, the more the index is likely to lose its power as a tool of communication.

  An example of an index that combines multi-dimensionality of components with a focus on poverty or deprivation is the Multidimensional Poverty Index (MPI), developed by Alkire and Foster (Alkire et al., 2015) and presented by Sabina Alkire at the same HLEG workshop.5 Here the issues are the selection of dimensions, the specification of the cut-off in each dimension to identify deprivation, and then the normative choice of the number of dimensions in which an individual must be deprived in order for that individual to count as deprived overall. Such reduction of complex multi-dimensionality into a single index has elicited critiques, which can be interpreted more generally as critiques of any composite index and as support for a dashboard of indicators:

  Recognizing that poverty is not just about lack of household command over market goods does not imply that one needs to collapse the multiple dimensions into one (uni-dimensional) index. It is not credible to contend that any single index could capture all that matters in all settings.… But when one faces a trade-off, because a policy spans more than one dimension, those with a stake in the outcomes will almost certainly be in a better position to determine what weights to apply than the analyst calibrating a measure of poverty. (Ravallion, 2011, p. 247)

  In his presentation at the HLEG workshop, Stiglitz also argued that a dashboard was preferable to a composite index. Different numbers are useful for different purposes, and local context is important in selecting which numbers matter for what.

  This then leads to Ebrahim Patel’s second question: How big should the dashboard be, and what should be on it? On the number of goals and targets, the answer is, of course, country specific, but there may be some consensus developing on how many. It is generally agreed, and it was the view expressed by all participants at the HLEG workshop, that the SDGs are good as a platform, but 169 targets is way too large a number of indicators to be useful as a “top of the dashboard” list in a national dialogue. The Genuine Progress Indicator (Talberth, Cobb, and Slattery, 2006), for example, has not really taken off—is it because it has 26 component parts? The Mexican government uses income poverty as well as deprivation on seven other dimensions to monitor national well-being (CONEVAL, 2010). The case for a limited number of indicators is also made in the report of the Atkinson Commission on Global Poverty (Atkinson, 2016). The general point is that the number of top-level entries on a dashboard for measuring and monitoring well-being and development should not be too large, and there is a case to be made for the number to be below 10 and perhaps around 7, depending on country context, although some would argue that even that may be too many.

  What should be on the dashboard is also, of course, a country-specific question. For South Africa, for example, key well-being indicators apart from GDP would have to include the employment dimension. Throughout Africa, the use of unemployment as an indicator is fraught with problems, as argued by Baah-Boateng at the HLEG workshop.6 The high levels of informality mean that the standard International Labour Organization (ILO) measurement of unemployment does not capture the essence of lack of productive work. In South Africa there has been an argument for using employment rather than unemployment as a lead indicator.7 Again in South Africa, the issue of income inequality is also front and center in policy debates and in the national consciousness. But which measure of inequality—the Gini coefficient, or the income share of the top X%? And which metric of household material conditions, e.g., household consumption or income, net or gross of taxes, per capita or equivalized? Access to basic services is yet another leading issue in South Africa, but here we risk getting into a proliferation of dimensions including education (different levels), health care, and housing. Perhaps for these social dimensions one could have a multi-dimensional deprivation index as suggested by Alkire et al. (2015). And none of this touches on longer-term environmental degradation concerns. Further, in So
uth Africa, the metric of disaggregation by race is central to the policy dialogue, as is disaggregation by gender.

  Ultimately the choice of what should figure on a dashboard is a national policy decision with no simple technical methodology to the rescue. But if, following Ebrahim Patel’s question, we were forced to prioritize to, say, five indicators, what would they be? For a country like South Africa, and other countries in Africa, perhaps the following indicators would be prominent:

  • per capita income

  • income inequality and poverty

  • employment

  • Multidimensional Deprivation Index based on access to basic public services

  • long-term environmental degradation

  Throughout, these indicators would need to be disaggregated by race (and often ethnicity) and by gender, and perhaps by other categories such as age; so, because there would be several of these breakdowns by population groups, the sense of a small number of entries on the dashboard may be somewhat illusory. Furthermore, there are sub-indicators behind these key indicators, such as wages, under-employment, or individual dimensions of poverty in the Multidimensional Deprivation Index, or various aspects of long-term environmental degradation. And there may well be disagreements even on the selection of the top 5 key indicators. What is needed is a national-level discourse that takes the SDGs as a platform and then fashions a dashboard that meets national needs and priorities, as well as the statistical capacity of each country to generate the data needed.8

  The Role of Statistics at the National Level

  Statistics have power, and are political. In his presentation to the HLEG workshop, Ravi Kanbur discussed the role that statistics had played in colonial rule, in the struggle for independence, and in post-independence governance in India.9 In the 19th and early 20th centuries, the India Office—the British government department administering Indian affairs—was required to present an annual report to Parliament on the “Moral and Material Progress and Condition of India.” Indeed, John Maynard Keynes, in his first job out of university, served in the India Office and edited the report for 1906–07.