Global Fund and how to invest in impact

Standard

Some time after Aidspan first raised its concerns about the different ways the Global Fund treats national and international grant implementers, the Secretariat of the Global Fund has replied.  I don’t want to go into too much detail, so to cut a long story short the issue was that while the Global Fund allows international NGOs and agencies to report some of their overheads as “indirect costs”, national ones are expected to provide detailed reports and receipts on every single expenditure, whether it is related to programmes, procurement or management.  

The Secretariat’s response suggests there won’t be much change to this policy. This is disappointing, but not just because it seems unfair that the smaller, lower capacity organisations should have a more complicated reporting burden.  To me it is disappointing because it suggests that the Global Fund’s attitude to risk and to performance based funding has not changed a great deal.  Risk remains, essentially, the risk that money went astray or was not all spent in the way that was planned.  Having hovered around some of the discussions about the development of the Global Fund’s new model in the past year I had a small amount of hope that there would be a new approach to risk.  One that considers the risk of people not getting treatment, or of people being mistreated by programmes,

I understand why the Global Fund remains jittery about financial management.  But it really is too bad that the focus is still all about monitoring inputs.  If the Fund was truly supportive of performance based funding it would worry less about how money is spent and what it is spent on, and spend more time monitoring the impact of its investments.

The secretariat’s response also suggests to me that there will be no move toward allowing small grass roots sub-grantees to report primarily on their results either.  The risk, which I discussed in this post some time ago, is that community groups will continue to be expected to spend a lot of their time and good will bean-counting and less of it doing the work their communities need them to do. 

The US Supreme court and PEPFAR’s “pledge”

Standard

The Supreme Court of the US is currently deliberating the constitutionality of the so-called “anti-prostitution pledge”, a requirement introduced by the US administration in 2003, according to which organisations involved in programmes to fight AIDS had to adopt a policy explicitly opposing prostitution as a condition of receiving funding from the US government’s PEPFAR programme.  The case against the pledge is that it hampers effective HIV programmes and that it constitutes a violation of first amendment free speech rights (the latter, only in relation to US-based organisations).

For more background, I can recommend the primers and analysis by Serra Sippel, Melissa Gira, Melissa Ditmore and Dan Allman, and Chi Mgbako.  The New York Times also published an editorial calling for the Supreme Court to rule that the Pledge is unconstitutional.

While I also hope that the Pledge is lifted, I think it is also important to recognise that, as important as the influence of US funding is to the global response to HIV and AIDS, the Pledge is neither the only nor the main reason that HIV programmes have failed and are continuing to fail sex workers.  Governments across the world have failed to adopt policies and implement HIV programmes with sex workers that are evidence based and respectful of human rights, irrespective of whether they are using their money, the US government’s money, or money from another source.

Nor is the Pledge the main cause of abuses of the rights of sex workers.  In the majority of countries sex workers remain criminalised or quasi-criminalised, and they face a constant threat of violence and abuse from law enforcement officials and medical authorities (one example here).  Removal of the PEPFAR pledge will, no doubt, enable programmes to work more closely with and support grassroots sex worker groups who want to challenge this situation, but they will still be largely constrained in what they can do by the national and legal context.  If governments want to arbitrarily arrest sex workers, or stop sex workers organising and claiming their rights, they will do so.

We know that it is not beyond the UN system and donor governments to be scandalised by human rights abuses – take, for instance, the justified international reaction to instances of homophobic abuse.  That the mistreatment of sex workers by governments and their agents is not a scandal is a sign of how much further there is to go.  We need more than just the removal of the PEPFAR Pledge: we need a commitment to supporting the rights of the most vulnerable, whoever they are.  At the same time we need much more humility about the impact that decisions taken in western capitals have on human rights on the ground, and much more solidarity and support for grassroots organisations.

Carrying 20 kilos of bananas for 15 miles

Standard

Some people say there are 4 Ps of marketing, some say there are 5, and others still say there are 7 or even 8.  But as far as I know Packaging isn’t on any of the lists.  However, just recently the Design Museum announced that its 2013 award for Best Product was going to Kit Yamoyo, an anti-diarrhoea kit containing essential medicines and products that could play an important role in tackling the second biggest cause of death of children in low-income countries every year.

Why a design award? What first inspired the development of the kit was the insight that companies like Coca Cola manage to get their product to the most remote locations.  And so the kit was designed by the organisation colalife to fit neatly into the crates that are used to transport Coke bottles.  The programme is currently being piloted in Zambia.

And yes, the design looks neat, and innovative (that holy grail of development initiatives) and as a result gets noticed.  But the design, and the tie-in with Coke crates, is largely irrelevant to whether this product will end up saving lives or not.

Someone must want to buy these bananas.

Someone must want to buy these bananas.

Kit Yamoyo has recently been covered by Wired, and although the headline, “Clever Packaging: Essential Medicine Rides Coke’s Distribution Into Remote Villages”, also emphasises the coolness of the packaging and the transportation, read through to the end and you’ll see that, as it turns out, the real insight in this story is not about packaging or even delivery of products.

Don’t take away the impression that over 2 million infants and children are dying of diarrheal disease every year because until now the medicines to treat them have come in the wrong shaped boxes.  Coke itself is not the easiest thing to transport in large volumes. It is heavy, it is bulky, and it is somewhat fragile.  The fact that you can find Coke nearly anywhere has little to do with the design of the crates it comes in.  In the same way, there’s a reason the chap in the picture is carrying 20 kilos of bananas on his back for 15 miles.  He’s not doing it because it is convenient (there are no EU directives on straight bananas in rural Madagascar!), he is doing it because someone wants to buy his bananas.  Similarly, if you’ve ever noticed sellers hanging outside nightclubs with cigarettes, chewing-gum and condoms on their trays, it is because people going into and coming out of nightclubs want cigarettes, chewing-gum and condoms.  If they didn’t, there would be something else on the trays.

The real insight here is to do with how Coke, and any other manufactured product, go about producing something that will find its way onto shop shelves and into people’s daily lives.  It doesn’t matter how nice the distribution system is: if people don’t want to buy the product, shopkeepers aren’t going to buy it in from wholesalers to suck up capital and gather dust on their shelves.  And sure enough, the head of colalife, Simon Berry, tells Wired that little if any use has been made by retailers of the design award-winning compatibility of Kit Yamoyo and Coke crates.  But regardless of this he is confident that they have created a product that people want, that they can afford, and that generates profit for retailers and wholesalers.  In very poor regions affordability may continue to be a problem and in those cases programmes need to find a way of keeping in place the financial incentives that get the product to where it needs to be while making sure that those who need it can get hold of it.  This includes being able to afford it.  Many programmes are experimenting with ways of getting vouchers to those who need them, so that they can get health products for free or at a subsidised price while maintaining the profit margin that retailers need to stock the product.

The result of colalife’s pilot will hopefully tell us more about how well the product has sold, who has used it, and whether some components of the kit are more useful than others.  If the product catches on and people in areas affected by diarrheal disease are willing and able to buy it, then it is good news, whatever the shape of the packaging.  More important still will be improvements in access to clean water and sanitation that will reduce the prevalence of disease in the first place.

Another discussion on sales of health products also caught my eye this week: this World Bank blog post about the evidence on different types of incentives used to encourage hair stylists to sell female condoms, coincidentally also in Zambia.  I agree with the broad conclusions of the author: that we haven’t got a whole lot of evidence about incentives, and that findings of a study on incentives for one intervention should not be uncritically assumed to be valid for other interventions.  But I am even less impressed than he is by the study he is describing. Not the quality of the study but the fact that even the best performing stylists, those who had non-cash rewards, only managed to sell an average 15 packets of female condoms in a whole year – compared to 7 packets a year for those receiving cash rewards.  Either way, that’s a pretty low number of female condoms sold – and the study probably tells us more about peoples’ interest in the product than about incentives.  Just like Kit Yamoyo: if people don’t want the product, and retailers don’t have a good reason to stock it, it’s probably worth having a re-think.

Rubbish evaluations in the aid sector; consultants

Standard

2013-04-10 14.00.37

I missed this, from Rajiv Shah the Administrator of USAID:

In many instances our project evaluations have been commissioned by the same organizations that implement them. Often what passes for evaluation follows a 2-2-2 model: two contractors spending two weeks abroad conducting two-dozen interviews. For about $30,000 they produce a report no one needs and no one reads.

And the results they claim often have little grounding in fact: one of our implementing partners claimed over a quarter-of-a-million people benefitted from $14,000 spent rehabilitating an Iraqi morgue.

This has led to a relationship between implementing partners and evaluators akin to that between investment banks and rating agencies. Just like investors couldn’t tell the difference between triple-A investments and junk, taxpayers can’t tell the difference between a development breakthrough and projects and subprime development.

Well, OUCH.  I think he generalises somewhat.  Some of the jobs I’ve done which Shah would probably categorise as 2-2-2 stemmed, I think, from a genuine desire on the part of the client to get me to have a look as an outsider and to give my views on how things are running and how they can be improved. And this can lead to improvements.  It might be more accurate to call this sort of evaluation a “review”, an incremental learning opportunity if conducted properly. Don’t expect them to demonstrate or prove impact. They won’t, and if they claim they will, someone is lying. But when a programme is putting in place policies or services that are already backed up by good evidence of efficacy and efficiency, assessing whether it has done so effectively is important. Implementation is no walk in the park.

But have I ever ended up doing a 2-2-2 like the one Shah describes (bar the cost element)? Or even a 1-1-1? Yes. More than once.  I’ve even commissioned them.  And despite the very dim view most people in the aid world (and without it) seem to take of independents, I care, as do most of the independents I know and work with.  We don’t want to do meaningless work.  If it looks like an assignment is bullshit many of us will turn it down but it’s not easy to tell up front.  A lot of the time we don’t really know if it is going to go anywhere or not until we are in too deep.

Sometimes what looked like a very exciting, meaningful piece of work, turns out to be nothing more than one bureaucrat’s means of winning an argument with another one down the hallway.

But this is no call for sympathy. Most of us do pretty well and manage to balance out the nonsense with meaningful work.

Anyway, OUCH nonetheless.  Because there’s a lot of truth in what Shah says. There’s a lot of it about. And he is right to want to fix it.

Still, given that 2-2-2s are, in my view, not universally or inevitably poor and meaningless I often wonder about coming up with some sort of checklist or charter to help ensure they are worthwhile.  Here’s a few thoughts. If you read between the lines you might get a sense of some of the challenges I’ve faced.

  • Will the client commit to findings being publicly discussed/shared?
  • Will the client commit to providing all original datasets to the consultant for analysis (rather than summaries)?
  • Will the client accept the consultants’ duty to maintain confidentiality of all informants?
  • Does the client acknowledge that the consultant is responsible for establishing findings and recommendations, and accept that it may not edit or change these but that it may issue a management response?
  • Will the consultant have the right to publish, independently, the methods and findings? (This is standard for university based consultants but not for independents).
  • Was the evaluation designed at the start of the programme rather than at the end when the managers realised an evaluation report had to be produced?
  • What is the mechanism for interpreting and implementing the findings or recommendations? If it is about experiential learning and incremental improvements, will the implementing team have the scope to make any recommended changes? If the review or evaluation is at the end of a major project, who is the “audience” for the findings?
  • Do the terms of reference make any suggestion that the exercise should describe the programme’s “impact”?  If so, question it, and if that doesn’t get you anywhere, run a mile.
  • Has the client pre-empted the findings? HINT: if the terms of reference say something like “demonstrate the impact of…” or “describe best practices developed by…”, they may need to be taken down a peg or two before you start.

I’ll try to add. If you have any thoughts please fire away in the comments

Better participation?

Standard

There is no better way to kill a joke than to over-analyse or explain it. Nonetheless here’s a more earnest reflection on the recent post I wrote for the Stuff Expat Aid Workers Like. “Having better participation than you” is a riff on the combination of principles, guilt, fear of being perceived as a neo-colonialist and obligation to adhere to rigid funding conditions that make “participation” such a unique concept in the world of aid and development workers.

Discussions on the importance of participation in aid and development efforts have been around for a long time; and critiques have been around nearly as long. Some critics have called the participation movement “tyrannical” emphasising, among other things, the absolute nature of the principal of participation. In my SEAWL post, by talking about aid workers always thinking they are more participatory than everyone else, I was hinting at this. The reality is that because it is unlikely that there is any service or project or policy that is going to keep all of the people happy all of the time, it is always going to be possible to find someone whose views and needs are not properly accounted for. It is always going to be possible therefore to chide others for not being participatory enough. Similarly it might be possible to chide others for being too participatory. I sometimes wonder about initiatives that aim to get poor people – who are busy enough just getting by – to identify and resolve their own problems with their own resources, or to provide extra hands when local amenities aren’t working. Sometimes it sounds perilously like poor people being told to get on their bikes by people who take the same amenities for granted back home. The other side of the tyranny of participation is the near obsession with “participatory tools”, maps and pie charts drawn out of sticks and stones in the dirt, and the shunning of anything that looks like technology. I remember colleagues telling me about an aid worker they nicknamed “Mrs Bean” because she went round villages with a bag of dried peas that she insisted the villagers used for voting on priorities (that is, if she could convince them to put their mobile phones on silent for long enough). (The funniest thing about Mrs Bean, by the way, is that in participatory rural appraisal practice, the idea is that you use locally available materials – perhaps including dried peas – not that you bring your own special rustic bits and bobs with you).

Another common criticism of participation is that it is often little more than a cover for manipulation. Or facipulation. “Beneficiaries”, or programme participants, are encouraged to participate and listened to just so long as they give the right answer to the donor or NGO that is actually taking the final decisions. This is all the more problematic when, as is often the case, the “communities” that are being asked are largely imagined communities, defined by their poverty or behaviours or epidemiological status rather than by any common purpose or identity. Demanding consensus or compliance from this sort of community is oppressive.

Both of these critiques point to genuine problems in aid practice. The first suggests that there are limits to the concept, and the second refers to the fact that the widespread capture of the language of participation in aid practice doesn’t necessarily bring with it a solid understanding and application of the principle. Although the problems are genuine, however, it doesn’t mean they can’t be fixed or that efforts to engage people more, and to be more accountable to them, should be abandoned.

I’m not going to discuss these two critiques further. Instead I’m wondering whether the thinking about development might be turning a corner that makes some of the principles of engagement and participation more realistic. Some of the biggest problems with participation, including the ones I was having a go at in the SEAWL post, are that it is most often operationalised within a fairly old-school view of what development is. To simplify the old school, this is how it goes: aid comes from the public or charities or governments in richer countries, and gets given to the public or charities or governments in poor countries, to produce development. And this is done in the form of projects, which generally combine fairly unspecific objectives (in terms of impact) with highly specified financial monitoring and management mechanisms, designed to reassure the donors that the money has been spent on the right sort of things and hasn’t been stolen.

As long as that version of development is the space within which we try to get poor people to engage and participate, with the best of intentions, there is always going to be compromise, and it is always going to be possible to point to a given initiative and say that it is not participatory enough. The rare cases of meaningful, fully compliant participation, even if they do exist, are likely to be very, very small in scale.

This doesn’t mean we should get rid of project based aid, particularly humanitarian aid. It doesn’t mean that aid ventures aimed essentially at financing infrastructures or services shouldn’t make sure they are responding to what people need and what people say will work. On the other hand, sometimes the specific nature of problems is quite hard for communities to identify. Health systems, to be functional, need a multi layered structure that enables effective supplies, procurement, and specialist support. Community knowledge and engagement is crucial to figuring out how to deliver those services at local level. But it is also important to have strategic, synthesised information about the regional or national picture in order to design and resource the systems that make that happen. The “I’ve got better participation than you” response to this might be that it is elitist, or that it limits community engagement to tokenistic consultation on priorities. But consultation is only tokenistic if nothing is done with the results.

Is the “new school” thinking on development any better on these issues? Actually I think so. For starters, the very acknowledgement of the limited role played by aid in development implies an acknowledgement that there is a lot more going on. And that other stuff that is going on, although it is happening alongside remarkable gains that humanitarian aid has helped to produce, is happening independently. And it is coming from people themselves. If anything, participation is proving itself, even though it may not be the version of participation that the aid sector has been designing, redesigning and ritualising for decades.

The growing emphasis on information and transparency, not just within aid-funded projects but within public and social policy in general also helps support this trend.  And if aid funding is to go beyond providing for basic needs and to contribute to development I think these are the dynamics it needs to support, even if the processes are harder to monitor and the outcomes harder to predict.