The Politics of Poverty

Ideas and analysis from Oxfam America's policy experts

Bizarre methodology skews World Bank survey on oil, mining “perceptions”

Posted by
Many of the 1,429 households resettled to make way for Vale and Rio Tinto’s international coal mining operations in Tete province, Mozambique received land with poor access to water and limited productivity. The farmland provided to Senolia S. (pictured), upon her resettlement to Cateme, was reclaimed by its original cultivators. She did not receive any replacement land or additional assistance and finally scraped enough money together to rent a plot of rocky, untilled land.
Photo: Samer Muscati / Human Rights Watch Many of the 1,429 households resettled to make way for Vale and Rio Tinto’s international coal mining operations in Tete province, Mozambique received land with poor access to water and limited productivity. The farmland provided to Senolia S. (pictured), upon her resettlement to Cateme, was reclaimed by its original cultivators. She did not receive any replacement land or additional assistance and finally scraped enough money together to rent a plot of rocky, untilled land. Photo: Samer Muscati / Human Rights Watch

Share this story:

Results of a World Bank Institute survey on public perceptions of extractive industries in developing countries are practically useless to understanding what communities directly affected by mining and oil operations actually think.

The results of a new World Bank-sponsored survey say that “a clear majority of responses believe that mining and oil/gas are positively impacting their country.”

It always pays to look at the methodology and underlying data sets.

In January the World Bank Institute (WBI) Governance for Extractive Industries Program commissioned an “Extractive Industries Public Perceptions Survey,” the results of which have just been published. The survey purports to provide “unique insights into people’s perceptions of the contributions of the extractive industries to their country’s development, their views on the availability and reliability of information, and their knowledge of the sector.”

But a huge methodological problem means that those most impacted by these projects are least represented in the survey.

The WBI says that the survey employs an “innovative nano survey approach” described as an Internet-based technology that extends a brief survey to a random sampling of respondents. Of the 14 countries surveyed, seven are in sub-Saharan Africa where Internet access is quite limited, especially in the rural areas affected by mining. The survey report does note that “responses are limited to those with internet access, typically a more urban and general public and not targeted to communities surrounding mining or oil/gas development” but then goes on to report findings as if they were meaningful, despite the admitted digital divide that plagues their results.

In Mozambique, where only 4.3 percent of the total population even has Internet access, 1155 people took the WBI survey. Exactly four people surveyed were from Tete, the province that has been the site of large-scale coal mining operations that have resettled communities! Last year Human Rights Watch report documented the plight of 1,429 households moved to make way for mining projects by Vale and Rio Tinto. Around 59% of the WBI survey respondents were based in Maputo, the capital, and are among those more likely to see benefits and not costs of resource extraction.

Also of note, only 30 percent of the Mozambican respondents were women, even though research has shown that women often disproportionally bear the brunt of the costs of mining. The WBI survey also noted that 47 percent of respondents thought that mining’s impact on the environment was “negative” or “very negative” – a result that certainly would’ve been much higher if more people directly affected by mining had participated.

In Ghana, 98 percent (!) of those responding to the WBI mining survey were from Accra, the capital. The “perceptions” about mining of only 39 people from regions with significant mining operations were included. In the WBI oil survey, exactly three people responding were from the Western Region where the Jubilee oil discovery is! Oxfam partners such as Friends of the Nation, WACAM, the Center for Public Interest Law, and others have been working with local communities affected by mining – from human rights violations to loss and land and livelihoods – and their voices are nowhere to be found in this survey.

One useful bit of information from the WBI survey in Ghana is that when asked “Does your government publish how much money it collects from oil and gas companies?” only 31 percent accurately answered “yes.” The Ghanaian government does, in fact, publish what they receive from oil companies, down to the dollar and barrel on a quarterly basis. This shows that the government and others need to do a better job spreading the word about these important disclosures.

The World Bank may be happy to highlight these so-called positive public perceptions about oil, gas and mining. The WBI says “the results of this survey should inform discussions about ongoing efforts to better govern the extractive industries and make the sector more transparent and accountable.” The results, though, are practically useless to understanding what communities directly affected by mining and oil operations actually think. Actually talking – and listening – to those people must inform the work of donors as they design their programs in the sector.

Join the conversation

  1. Pingback: DevelopmentMail | Analysis of development and governance

  2. Mjarvis@worldbank.org'Michael Jarvis

    Dear Ian,
    Thanks for the blog post. On behalf of the World Bank Insitute team that commissioned the survey, I wish to acknowledge your point on the fact that the survey, by being web based, is not representative of mining affected communities, and we in no way claim that it is. By design, this was a pilot survey testing the nanosurvey approach suitable for sampling a general public that is not always easy to reach or typically sampled in the context of extractive industries. I believe it shows its potential as a tool that can be used to complement other survey approaches, such as household surveys that can be targeted to communities directly affected by mining or oil/gas production.

    As long as we are very aware of the limitations of the survey in terms of reach and the need to balance with other data points, the findings do still have value in giving insight into national level perceptions of the sector. The surprisingly positive perceptions of the industry’s contribution to country development among those sampled need careful interpretation – for example this may link to inflated expectations that will not be born out, and need to be more effectively managed. It may suggest a lack of appreciation in urban centers for the true costs of extraction felt in other parts of the country. Although, taking Canada with high web penetration, you find positive perceptions of mining contributions higher among respondents in mining rich Quebec than Nova Scotia that does not have significant mining.

    Certainly the results in terms of sources of information, suggest possible entry points for engagement with general citizens to build greater understanding of the sector, the stakes at play, and its true impacts. Such engagement is something that Oxfam itself has an admirable track record of supporting, such as in Ghana.

    As web penetration continues to grow, the methodology will have broader application. It allows for truly randomized surveying gathering non-incented responses. While you highlight the gender imbalance for the Mozambique results for the overall survey there is essentially an even gender split of the 16000+ complete responses. As you say we just need to be very much aware of whom this approach can reach and make sure it is integrated with other methodologies that do sample and provide critical perspectives, most obviously those communities directly affected who typically face the highest costs. We would welcome the chance to explore a possible consortium of organizations, including Oxfam, to consider what might make for the most effective combination of methodologies.

    I look forward to others views on the findings (acknowledging its limitations) whether on the views of extractives impacts, whether they suggest a trust deficit in sources re available sources of information, or the demand for more information than currently available.

    Reply
    1. massimilianocali@hotmail.com'Max Cali

      Michael, reading the summary findings of the survey one must agree with Ian that a one line disclaimer of the survey does not do justice to the lack of representativeness of the survey. That is a key issue here and I am not even sure why the WBI has decided to make these results public in the first place. I can understand that there is a value in the data for the World Bank but why publishing them as “public perceptions”? That is misleading and makes one think that it is more a (biased) advocacy piece rather than a way to better understand the perceptions of extraction of a certain section of the population. If you want to publish them it would just be fairer and more useful to call it “Urban middle class perception of extractive industries” or “internet user perception” not certainly “Public perceptions”.

      Reply
  3. mwashome@gmail.com'Andrew Ramsden

    Rather than reflexively ridiculing (bizzare: really?) a report – admittedly with limitations – because it does not align with the ‘mining is bad’ narrative, why don’t we look at what the data might tell us, for instance, about the different perceptions of the industry between local communities and people in capital cities?

    This would certainly be a more constructive and ultimately helpful contribution to the discussion and development outcomes.

    Reply
  4. Ian Gary

    Thanks for the comments. In response to Andrew, yes , it is “bizarre” to label this as an extractive industries perceptions survey for a country when it is completely skewed to a very small slice of the urban elite who use the Internet. To your point of looking at the data – I did. The data doesn’t tell us anything about the “different perceptions between local communities and people in capital cities” because local communities aren’t included. That was precisely my point. If this survey was properly framed and released with a robust survey of local communities living near mines that would be another thing. As WBI’s Felipe Estefan told a press outlet “ ‘we did not do enough thinking about’ clarifying the drawbacks of an online survey for those interpreting the data”. http://www.icij.org/blog/2014/04/no-net-no-say-online-mining-survey-surprising-results
    To Michael’s thoughtful points, I think the framing of the survey and cautionary notes about the methodology need to be improved, in addition to including information on other tools WBI plans to use to gauge perceptions at the local level. Also, I would also question the points about this being useful to gauge the perceptions of the “general public” or the “national level” in countries such as Mozambique or Ghana. Presumably you would agree that these terms should not be equated with the views of those who use the Internet and who are largely based in the capital. An analogy would be asking those living on the Upper East Side in Manhattan whether they think income inequality is a problem in the US and describing this as being the views of the “general public”.
    We would be happy to continue the discussion on ways we can collaborate on developing a combination of methodologies to get a more accurate and representative view. This is especially important in the implementation of policies of Free, Prior and Informed Consent for oil and mining projects – a policy the International Finance Corporation has adopted for indigenous peoples (and which the International Council on Mining and Metals has adopted for all of its members.)
    Will the mining and oil industries “carefully interpret” the findings or trumpet the headline message? I will wait to see.

    Reply
    1. elsa_shumbacher@yahoo.com'SenhorCoisa

      Thank you Mr. Gary for bringing attention to amateurish development research methods employed among the highest international levels. Clearly, where such moneyed interests are involved, as in multinational mining, a very critical eye and reflective discussion is required. Let us hope your observations inform future efforts and encourage more inclusive and thoughtful publishing.

      Reply
  5. Pingback: Bizarre methodology skews World Bank survey on oil, mining “perceptions” – Oxfam America | Kooky Ideas

  6. smills@equitableorigin.com'Soledad Mills

    Many groups are now using mobile phone technology to reach across the digital divide as a tool to survey perceptions, measure impacts, and receive grievances. It’s not a perfect solution but might allow for broader community participation. Ultimately, though, I agree with Ian that there is no substitute for actually talking to and hearing directly from affected communities, which requires commitment, resources and ability to develop relationships on the ground.

    Reply
  7. Pingback: Catherine Coumans of MiningWatch responds to CIIEID’s hosting of survey unveiling | Stop the Institute/CIIEID

Leave a Reply

Your email address will not be published. Required fields are marked *