Monitoring and evaluation...we're in it together

It’s a daunting task to begin measuring the effectiveness of your work, both in relation to knowing what to measure and what you might discover. The key to success is the involvement of funders and service users. Exploring the factors that lead to poor performance in current monitoring and evaluation practice offers some supporting evidence.

In recent years some funders (they will remain unnamed) have designed monitoring and evaluation systems in isolation from the projects and services they fund, and then handed them to service deliverers to complete. As a result, these systems feel more like an imposition, rather than a vital check and balance of how our projects and services are performing. At best we feel demotivated, it feels like a chore, and at worst we feel frustration at being asked to measure the wrong things.

Far from entering into a discussion to create a system that works for funders, service deliverers, and service users alike, we tend to jump through hoops by half heartedly completing the monitoring and evaluation reports we inherit. Of course, this keeps the funders happy, which subsequently keeps funding rolling in. But take a longer term view of the situation. It gives funders the false impression that they are collecting the right information, and it hampers our ability to conduct effective monitoring and evaluation. We fail to learn and apply lessons because we lack the relevant data and motivation to do so. Ultimately, it’s our service users that are affected.

However, we are often guilty of committing the same error when it comes to our relationship with stakeholders. If we develop indicators and outcomes in consultation with our funders, we can be a little too convinced our own wisdom. There’s a temptation to second guess what is important to our service users, rather than just asking them. We’re experts, right? This approach can lead to vital information being overlooked, and it can have a knock-on effect on the services we provide. WRVS’s project to capture what is important illustrates this point perfectly.

Until two years ago WRVS measured the effectiveness of its work largely by measuring numbers, for example, how many meals on wheels were delivered on target. The senior management team took the bold decision to ask the question ‘so what?’ They wanted to know what difference their services actually made to service users’ lives. After commissioning researchers to ask service users what really matters to them, WRVS found that some participants in the meals on wheels programme did not eat the meals they received, yet they continued to participate, because the human contact left them feeling less isolated. WRVS still measure the number of meals delivered, but their monitoring and evaluation is now geared towards measuring softer outcomes, such as reduced isolation and increased confidence.

There are, of course, numerous pressures on both funders and service deliverers, which make for an imperfect relationship when it comes to monitoring and evaluation. However, both parties ultimately strive to achieve the same goal: a better standard of living for the people they support. Rather than entrenching old notions of what counts as success, we need to recognise that effective monitoring and evaluation is dependent on the involvement of both funders and service users.

New Philanthropy Capital will release a new publication this month which will explore what funders can do to help charities conduct monitoring and evaluation and demonstrate impact more effectively. Helping grantees focus on impact will be available to download for free from New Philanthropy Capital's website from 16th March

Clothes collections: needing a makeover?

The fundraising technique of clothes collections may be a traditional mainstay for our sector, but has been recently having some perception issues. To donors, charity clothes collections represent a convenient way of giving away items that are no longer wanted; yet the increase in bogus collections means that members of the public can sometimes find it hard to tell a legitimate ask from an unscrupulous one.

To charity fundraisers the collection of clothes for reuse or resale can be a crucial income generator - yet it’s becoming an increasingly crowded marketplace, with donors often receiving many requests from charities to donate items in this way. Add into the mix recent sensational and negative media reports on charity clothes collections, and the public perception of this fundraising form becomes somewhat blurred.

It is heartening to see these issues being taken up at the highest level, with a Government roundtable debate on bogus collections meaning that a joined up discussion with all relevant stakeholders present and engaged took place at the end of last year. The Institute will soon be bringing its House-to-House Code of Fundraising Practice which clarifies the standards required for the charity sector and donors alike.

What is crucial in all of this is for the sector to work together in presenting a united front. We need to acknowledge that there are a range of models of clothes collections which it might suit a particular charity to employ and that there is no standard rule as to which arrangement is better. It depends on the individual circumstances and make up of the charity in question.

At the end of the day, what is at stake is the giving public’s levels of trust and confidence in charities. If there is uncertainty around techniques, what constitutes a legitimate fundraising collection, how charities use donations of clothes and the profits they keep from such gifts, people’s comprehension of the benefits of this form of giving will naturally decline.

Of course, individuals are more inclined to make judgments about a charity based on what they see of its activity first hand. Some people may have experienced the work of a good cause through being a service user or beneficiary. However, more often than not it’s fundraising that is the mode of engagement with individuals. And it stands to reason that the most ‘visible’ forms of fundraising, such as house-to-house, or face-to-face are shop windows for the rest of the sector’s income generating activity.

This all feeds neatly into the Institute’s accountability and transparency agenda; openness around all issues where fundraising is concerned – costs, salaries and required investment alike - is always encouraged. There are some basics of fundraising which should be covered off as standard by charities. One such example is the premise that money raised needs to be used by charities as intended by donors.

There are no two ways about it. The tense times we are still experiencing as a sector mean an increase in charities’ demand for funding. Crucially, it’s up to all of us to fulfill the missions of our organisations and, in all of the fundraising methods we use, engender trust in service users and donors alike.

Louise Richards, Director of Policy and Campaigns, Institute of Fundraising

Building a great reputation

How do you create a great reputation? Is it simply the case that great reputations follow from great work? Hardly, Bernie Madoff and Icelandic banks had great reputations until a few years ago. Such reputations are a product of effective communication, but they quickly fall apart under scrutiny. To build a reputation on a solid foundation takes more than effective communication. It takes a clear understanding that communicating the difference that your organisation makes generates long-term trust and confidence.

There are three stages to building and maintaining a great reputation. Firstly, you must engage your stakeholders and find out what they want to know about your organisation. Secondly, you must be proactive and transparent about demonstrating the difference your organisation makes. And thirdly, you need to posses an understanding of how people receive information about your organisation. Let’s explore each stage in more detail.

All your stakeholders will have questions about your organisation, but they won’t necessarily ask them. And questions that remain unanswered will be filled by assumptions, colouring the perception of your organisation, and therefore your reputation. A good example is direct mail. Many charities use this fundraising method to generate income, yet only 9% of the public believe that it works. The discrepancy is due our failure to communicate how effective this direct mail is at generating income. It seems that when most people receive direct mail they assume it’s a waste of money, and the logical conclusion is that ‘this charity wastes money, so I won’t give to them’.

We must address stakeholder concerns wherever possible. We tend to get pre-occupied tinkering with the tone and structure of key messages while neglecting to engage stakeholders on issues that, while seemingly minor, can fundamentally effect how they see our organisations. It’s simple: ask them what they want to know and answer them.

Equally important for building a great reputation is to communicate the difference you make, and do so with openness and honesty. New Philanthropy Capital recently released a study, Talking About Results, in which they compared how well charities communicate the difference they make. The paper found that charities still struggle to communicate the most important information.

Slogans, logos and branding can create a great reputation, but relying on these techniques alone leaves your reputation vulnerable. Talking about your impact is the difference between saying your organisation’s great and proving your organisation’s great. Subsequently, it’s the difference between building fragile good will and building trust and confidence. As charities, we possess the former, but we desperately need the latter.

Once you have addressed your stakeholders’ questions and concerns, and communicated the difference your organisation makes with openness and honesty, the final stage of creating a great reputation is to understand how information is delivered.

Information about your organisation will reach people in one or more of four ways: by direct experience of your organisation, through your communications, through the media, or by talking to friends and family. The media and the public (friends and family) are the most powerful drivers of opinion and reputation, and we have the power to harness them by taking a big picture look at the purpose of our communications. Actively targeting the local media and the public with messages about your effectiveness will build your reputation over time, while limiting your communications to service users and funders will also limit the number of potential advocates of your work.

Sadly, when charities begin tightening their purse strings the communications budget is often an early casualty. We need to resist this temptation. Now more than ever we need to shout about our success; we need to demonstrate the difference we make with openness and honesty.

How much should charities care about transparency?

The ImpACT Coalition is all about encouraging charities to become more transparent to their stakeholders. But how seriously should charities take transparency? And if you do get serious about transparency—signing up to the ImpACT Coalition’s Transparency Manifesto, and taking practical steps to being more open about your work—will anyone notice?

In particular, will donors care? At New Philanthropy Capital (NPC), we’ve been working with donors for the past nine years—helping them to focus their giving on creating the greatest impact. But there’s no one-size fits all model for understanding why and how people give, and little research (so far) on how charities’ transparency influences their giving.

So what do we know?

Well, the Charity Commission’s survey of public trust in charities, launched this month, shows that charities are among the most trusted institutions in society, coming closely behind doctors and the police. It also tells us, worryingly, that the most important factor influencing people’s trust is the proportion of donations that ‘gets to the end cause’. Concerns about cost ratios were seen as more important than making an impact—a reversal from the last survey in 2008.

There is also a brilliant piece of research by Hope Consulting in the US, which looks at donors’ motivations for giving. It found that donors can be segmented according to different motivations. The report suggests a segmentation into six types of donor—Repayer, Casual Giver, Faith Based, See The Difference, Personal Ties, and High Impact. It also finds that ‘major donors’ share the same motivations as regular donors within these different segments.,

On a straightforward reading of these findings, charities would be forgiven for thinking three things:

1. that transparency isn’t an urgent priority because donors already trust them;
2. that any efforts to be more transparent should focus on showing how little money is spent on overheads; and
3. that any efforts to be more transparent about impact can focus purely on High Impact donors, rather than regular donors.

I’d warn against drawing these conclusions. Acevo’s survey last year showed that many donors’ perceptions of charities, and therefore of factors contributing to trust, are somewhat divorced from reality. Those that don’t work in the sector often have antiquated visions of charities staffed only by volunteers and funded only by donations, which should all be spent at the front-line and not on wasteful things like offices and chief executives.?

There is a big discrepancy between what donors say they want to know (how the money’s spent) as opposed to what they need to know (what that spending achieves). And charities often don’t make efforts to challenge this. It can be a lot easier to stick to working out your admin costs than it is to work out your impact. As NPC’s forthcoming paper on impact reporting shows, charities are not yet routinely communicating their outcomes or impact in their annual reports, annual reviews, impact reports or websites, tending to talk instead about outputs and internally-focused objectives.

So how much will donors care if charities get more transparent, particularly about what’s important—what they achieve? Some may not care too much, as long as a scandal doesn’t emerge that destroys their trust. But some will, particularly the High Impact segment of donors. And greater transparency may encourage donations from those that aren’t giving now because they don’t trust charities. In time, greater transparency will help all donors to become more informed donors (just like all consumers become more informed consumers), and their trust will become more directly linked to how well charities communicate the difference they make.

If I were a fundraiser, I wouldn’t be encouraging my board to invest in more of the same old fundraising techniques, with ever-decreasing returns. I’d be encouraging them to get ahead of the game and meet the informed donor’s needs. My message to fundraisers is this: Work hard to capture and communicate the difference you make. Use that to start an honest dialogue with donors based on trust and understanding. And don’t throw all your hopes into a major donor campaign—see what happens if you kick off an informed donor campaign instead.