Balancing Evidence and Compassion-based Approaches to Making a Difference

Monday, January 13, 2014
Chief Economist Commentary

Economics is about the allocation of scarce resources to meet competing demands. Used well, it can shed important light on big questions of public policy and strategy — although it cannot provide a definitive answer to thorny issues of what governments should do when confronted by big choices between, say, military intervention and humanitarian aid. To take one recent example:

Writing in Slate economics commentator Matthew Yglesias recently made a swords versus ploughshares argument about military intervention.

NATO succeeded in Libya. It saved tens of thousands of lives from almost certain destruction. …without a single allied casualty and at a cost — $1.1 billion for the United States and several billion dollars overall — that was a fraction of that spent on previous interventions in the Balkans, Afghanistan, and Iraq.”

On the other hand if we are concerned about saving lives, bombing is not necessarily a good way to do it. Take for a counter example attempts to reduce the incidence of malaria:

…long-lasting insecticide treated bed nets… save… about one life for every $1,865 spent. That’s to say that if the United States was able to spend… $1.1 billion… on long-lasting insecticide treated bed nets we could have saved almost 590,000 lives from almost certain destruction. America’s other allies in Libya spent about $3 billion in total together. That’s something to think about.”

This is a powerful argument – but writing in the Globe and Mail on a different subject, Margaret Wente casts doubts on the effectiveness of bed net programs:

…bed nets treated with insecticide are a great way to tackle the scourge of malaria — theoretically. But there are big logistical problems, including distribution, looting, and costs. And even if you solve those, there’s no guarantee that people will use them for the purposes intended. Sometimes they use them to protect their goats, or to catch fish… Years of social marketing campaigns to promote the use of bed nets have scarcely made a difference.”

An important argument about priorities and resource allocation comes down to a simple question with a complicated answer: do insecticide treated bed nets actually “work”?

On the one hand, evidence from the Cochrane group based on “individual and cluster randomized controlled trials of insecticide-treated bed nets” is clear: ITNs are highly effective in reducing childhood mortality and morbidity from malaria.”

This provides a clinical foundation for effectiveness, but real-world programs, especially in the developing world, are inevitably less tidy and controlled. In a comprehensive but not especially rigorous investigation (involving “thoroughly reviewing the research… researching possible concerns… multi-day site visits to charities’ operations in the field… as well as cost-effectiveness analysis”) Give Well estimates that: “…the cost per child life saved… (is)… just under $2,500 (based on) total cost ($5.54 per LLIN).”

None of these numbers can be considered definitive – for bombing or malaria – data is just too scarce and unreliable. And they raise a key point – in a world where methodology and data will always be imperfect, when is enough evidence enough?

Yglesias and Wente do illustrate important lessons for charities:

  • While a good deal of charitable activity is motivated by compassion or other important but difficult to measure goals, concerns about whether and how well programs work are becoming more important in an increasingly data-driven world.
  • If charities are to be successful in competing for a place in the priorities of governments and donors, they will increasingly be required to demonstrate how effective they are in achieving results – that they are in fact making a difference.
  • In the real world of shaky or non-existent data and frequently intangible outcomes, assembling this sort of evidence will be a significant challenge. However, a careful and comprehensive anecdotal analysis can go a long way to shedding light on the choices that donors and government face.

At the end of the day, in our imperfect world, bed nets look pretty good.

Back to Blog Retour au blogue

Add new comment

About the Author

Chief Economist for Canada’s Charitable and Nonprofit Sector, Brian Emmett is tasked with measuring the impact of the sector and bringing economic issues facing charities and nonprofits to the forefront of public policy decision makers. Mr. Emmett is an economics graduate of the University of Western Ontario and the University of Essex in England, and has enjoyed a long and distinguished public service career. He was Canada’s first Commissioner of the Environment and Sustainable Development in the late 1990s and worked extensively on Canada’s Green Plan. He also served as Vice-President of the Canadian International Development Agency (CIDA) in the early 2000s and has been an Assistant Deputy Minister in a number of federal government departments.

The office of the Chief Economist for Canada’s Charitable and Nonprofit Sector is made possible through funding received by The Muttart Foundation, Ontario Trillium Foundation, Vancouver Foundation, an anonymous donor, and the PricewaterhouseCoopers Canada Foundation.

Our National Partners

  • great west life
  • Lawson Foundation
  • Muttart Foundation
  • RBC Foundation
  • Suncor
  • TD Bank
  • investors group

Learn more about our National Partners and other supporters.

Charitable Registration Number: 119218790 RR0001