CDE project 3 sec­tion 2: putting the prin­ci­ples and actions into prac­tice — step 3

Written by
The Commission on the Donor Experience
Added
May 03, 2017

Step 3: Decide on what you will measure

This is, perhaps, the hardest part. There are so many things that you can measure and so many emotions that might be critical to how the donor feels.

At the core is the need to learn about your donors – especially why they give to you and what they need and want from you. Really good insight will tell you what the most important needs are for your donors – and measuring whether you are meeting those needs is a great place to start.

There are lots of ways that you can find this out. You can conduct formal research, but the chances are you’ll get pretty close by just trying to put yourselves into their shoes. Monitoring what they are commenting on or complaining about to your supporter care team or on social media will also give you a good indication.

Before we outline the tools available, we have listed some considerations that you might want to bear in mind when deciding which to use.

Considerations: Use publicly available tools or make your own?

There are lots of tools available that you can use, or it is perfectly valid (and sometimes preferable) to create your own measures.

Most of the publicly available tools have been developed in the commercial sector – hence, they use different language (the most obvious one being the reference to ‘customers’ rather than ‘donors’) and often make different assumptions about the motivations for purchase compared to the motivations for giving or supporting. They have, however, been proven to work in growing customer loyalty and retention, have the advantage of being standardised in their approach and provide a simple way to start.

Some of the most successful organisations have learnt from these tools, merged this knowledge with insight about their customers and donors and created their own measures.

The important thing to remember is that there is no one right answer. You should decide what you feel is right for you. Measuring any of these metrics will mean you’re taking a big step forward towards being able to deliver an improved donor experience and measure its impact.

Considerations: Measure overall loyalty or satisfaction with a specific activity

Some of these tools are designed to give you a measure of how your donors feel about you. How do they feel about you as a charity worth supporting? How loyal do they feel about your cause? How committed are they to supporting you? Are they satisfied with the relationship? Do they trust you?

These are all questions that show how the donor feels about you as a charity. There is no one thing that drives the answers to these – the donor’s feelings will arise based on the breadth of their experience of you, on what they read about you in the media and what people they trust say about you.

The way someone feels about you as a charity will influence how they respond to your appeals, how long and how much they give and what they say about you to their friends and family.


Measuring the impact of individual communications

How does every Moment of Truth (or touchpoint – see page 9) make your donor feel? Do they feel inspired and valued? Do they feel pressured and annoyed? Is it easy for them to do what they want to do?

Every time you have any contact with the donor you create an impression: the way you ask for money, the magazine you send them, the way you interrupt them in the street, the way you answer the phone, the content and ease of use of your website and so much more.

Each touchpoint is a point where the relationship can go bad, or good. These are our Moments of Truth. These are the points where you will change the way the donor feels about you.

We believe that the aim is for the donor to feel a little bit better about you after every Moment of Truth than they did before it.

Considerations: Research methodology

Most of the tools rely on quantitative research, proactively asking donors about their experience or feelings towards you. By doing this, you will have the ability to create a benchmark against which you can track performance, as well as monitor performance against different segments, events or communications.

Qualitative research is valid, but its real strength is less about quantifying and measuring success and more about understanding why donors feel and react as they do.

Our recommendation is, if possible, to consider using quantitative and qualitative research together. The quantitative research will allow you to measure, benchmark, track and report. It will also allow you to identify where you have problems. The qualitative research will then allow you to understand why actions are successful or not and to develop plans to improve your scores.

You also need to consider how formal (or robust) the research will be. As a rule of thumb, the larger the organisation, the number of donors or the size of the financial decisions that you will make based on the research, the more robust it should be.

Tools

Through the rest of this section, we will describe some of the most common tools used to measure how donors feel them in more detail.

Table 1: Quick comparison of different Loyalty and Satisfaction measurement tools

Net Promoter Score

Net Promoter Score (commonly NPS) is probably the most commonly used tool in the commercial sector, and increasingly so in the fundraising sector.

The power of NPS is that insight is gained through asking one simple question: 'Using a 0-10 scale: How likely is it that you would recommend [company / product / charity] to a friend or colleague?' Respondents are then classified as follows:

  • Detractors: Anyone who scores 0-6
  • Passives: Anyone who scores 7 or 8
  • Promoters: Anyone who scores 9 or 10

By subtracting the % of people who are Detractors from the % who are Promoters, you will get a score (between -100 and +100).

Pros:

  • This is a very simple tool for measuring the extent to which donors will advocate for you. Behind this decision to become an advocate or not lies a grouping of all the emotions they feel for you. 
  • By using a standard tool, you will be able to share results with other colleagues and charities, thus understanding how you are performing.
  • As a quantitative tool, you are able to set a benchmark and monitor performance over time.

Cons:

  • The most common criticism of NPS is that it is too simplistic. While it gives an overall score for how happy donors are, it doesn’t give any indication of why they feel this way. It is therefore impossible to develop an action plan to address a low score without further analysis.
  • The concept of recommending a charity is not natural for many people, perhaps more traditional donors in particular. Many traditional donors will be very happy with their charity relationships, but believe that giving to charity is a private activity and therefore wouldn’t dream of talking about it to other people. Because this question asks about a behaviour rather than an attitude, a low score may indicate differences in preferred donor behaviours rather than satisfaction levels.

From the conversations we have had in putting together this document, NPS is most commonly used in our sector to measure the views of event participants following fundraising events. The MS Society case study (see Appendix 1) shows both an example of its use and how the organization is beginning to plan for its use in other areas of the fundraising mix.

Customer Satisfaction

Before we start looking at how to measure Satisfaction, we should note that even satisfied donors stop giving. Professor Adrian Sargeant[1] has shown that Satisfaction on its own is not sufficient to retain donors – Commitment is also needed. It is only when a donor becomes very satisfied that they change their behaviours and retention rates increase. This is what you should strive for.

Customer Satisfaction (or C-SAT) can be used in two ways.

Firstly, it can measure of how satisfied the donor is generally with the way you communicate with them.

Asking the right questions is important to understanding whether they are happy with the considerations that are most important to them. Common themes that you should consider include:

  • Feeling valued (and being thanked) by you
  • Communication and tone of voice

You can also use C-SAT to measure satisfaction after each Moment of Truth.

There are common questions which you should ask for all Moments of Truth, which include:

  • Was the tone of voice appropriate for you? (Does it recognise the donor’s motivations, interests, preferences and capabilities to support?)
  • Have we inspired you about what your donation achieves? (Will it leave them feeling really good?)
  • Did you feel unduly pressured into giving?
  • Did this communication surprise you? (Satisfaction is defined as exceeding expectations, so it is helpful to think of what donors expect and then exceeding it.)

Every Moment of Truth is different. Donors have different needs at each one, so you need to measure different factors at each one.

This can get extremely complex. Your job is to keep it simple, so we recommend deciding what the most important Moments of Truth are for your donors (those with the biggest reach or those that have the biggest impact on an individual donor) and finding out how they make the donor feel.

  • After an event, you should ask about their satisfaction with different elements of it, including registration, on-the-day experience, follow-up communications etc.
  • After a call to your Supporter Care team, you should find out if the call was handled professionally and courteously and led to a satisfactory conclusion.
  • Online. you should ask if the donor has found what they were looking for and were able to undertake any action they wanted to easily (see Customer Effort Score below)
  • When you thank someone for a donation, you should find out if they feel valued by you and inspired by what they have done.


[1] Sargeant and Woodliffe: Building Donor Loyalty: The Antecedents and Role of Commitment in the Context of Charity Giving (2008)

Customer Effort Score

The Customer Effort Score (CES) was created by the Corporate Executive Board's Customer Contact Council. Based on the premise that companies become profitable by giving customers solutions to their problems, they found[2] that the best way to retain customers is to solve their problems quickly and easily.

Like NPS, Customer Effort Score is an attempt to measure customer satisfaction with a single question. 

'How much effort did you personally have to put forth to handle your request?' On a 5- or 7-point scale from very low effort (1) to very high effort (5 or 7) 

Applying this in the charity sector is, perhaps, not going to have the impact that it has in the services sector because donors don’t perceive charities as being there to meet their needs in the same way, but there is evidence that it works in some areas:

  • We have all seen results of tests where making it easy for the donor to give (particularly through direct mail or on-line) increases response or conversion.
  • There are many statistics that show that making it easy for a donor to complain if they are unhappy about something (and then resolving the complaint) leads to a more committed and loyal donor in the future.
  • The easier it is for people to sign up for an event or to send in sponsorship money afterwards, the more people will take part and contribute.

[2] See: https://www.cebglobal.com/blogs/unveiling-the-new-and-improved-customer-effort-score/

Loyalty

It is important that we draw a distinction between Behavioural Loyalty (whether a donor gives again) and Emotional Loyalty (how they feel about wanting to continue to support you). In this section, we are talking about Emotional Loyalty, which About Loyalty defines as being the feeling of support or allegiance to a charity or cause.

Of course, we seek to increase a donor’s Emotional Loyalty in the hope and expectation that we will increase their Behavioural Loyalty!

Loyalty is a generic term and is impossible to measure with a simple ‘How loyal do you feel?’ question. We need to understand the drivers of Loyalty and measure how you perform against them.

Considerable research exists, including that by Professor Adrian Sargeant and About Loyalty to which we have already referred, that shows that Emotional Loyalty is created when you grow the following feelings in your donors (see section 3.2):

  • Commitment: Inspiring the donor to feel passionate about the cause and the organisation to want to continue supporting them.
  • Satisfaction: Feeling valued and recognised, not pressures, by the charity’s communications, especially fundraising.
  • Trust: Believing that the charity is effective at spending the donor’s money and that it behaves in the right way.

Therefore, to measure Loyalty you should measure these three factors. Then you can create an overall Loyalty Score for each donor/segment by combining them.

Of course, if one of these factors is more important for your donors and charity, then you can focus in on that one.

In Appendix 3, we have included example questions to help measure these drivers.

Create your own scores

The above measures are all simply suggestions based on what charities generally find useful. They are a good place to start, but may not meet your specific needs.

If you discover or believe that there is another factor that is important to your donors, then we strongly encourage you to be brave and choose to measure that. You will lose the ability to benchmark your performance against other charities, but that is a small price to pay to measure the aspect that is actually important to your donors.

As we mentioned before, some of the most successful charities and companies have created their own measures and use them to measure how they make their donors and customers feel. 

The NSPCC has created a ‘Happiness Index’ and James Walker, an industrial manufacturing company, have also created their own measures, both of which you can read about in Appendix 1

Indicators

All of the above measures involve asking donors about their attitudes or perceptions. This is not always possible, in which case you can also identify and measure indictors of Loyalty, Satisfaction (or Dissatisfaction) and Commitment.

Some of these indicators include:

  • Social media engagement
  • Complaints
  • Email open rates, click-through rates, unsubscribe rates
  • Proactively informing you of changes of address
  • Other digital engagement – use of website, registrations etc.
  • Response to non-financial asks
  • Response to surveys etc.
  • Volunteering

The most extensive use of this methodology we discovered is Make-A-Wish Australia, which we have outlined in Appendix 1.

Click on the image below to see Project 3 in full - PDF format

About the author: The Commission on the Donor Experience

The CDE has one simple ideal – to place donors at the heart of fundraising. The aim of the CDE is to support the transformation of fundraising, to change the culture to a truly consistent donor-based approach to raising money. It is based on evidence drawn from first hand insight of best practice. By identifying best practice and capturing examples, we will enable these to be shared and brought into common use.

Related case studies or articles

CDE project 3 summary: satisfaction and commitment

This project will create a practical guide to measuring other main indices of success than money or RFV and will show where relevant research and examples of best practice can be found.

Read more

CDE project 3 section 1: actions and approach

This section may be called Measurement, but it’s actually about culture, strategy and people. 

Read more

CDE project 3 section 2: putting the principles and actions into practice - step 1 and 2

Step 1: Make a commitment to growing donor satisfaction and loyalty

Read more

CDE project 3 section 3: Making it happen

We have seen that there are lots of things that you can measure, but you don’t have to measure them all. Just choose one! 

Read more

CDE project 3 appendix 1: measurement of satisfaction and loyalty in action

There are several companies and a few charities who measure how their customers and supporters feel, using some of the tools we have outlined in this document or their own methodologies. 

Read more

CDE project 3 appendix 2: frequently asked questions

In our research (see Appendix 4), several people mentioned barriers to measuring how their donors feel. This section lists the most common of these and gives our advice.

Read more

CDE project 3 appendix 3: sample questions

A quick Google search will bring up many sample questionnaires, but for ease we have included a couple here.

Read more

CDE project 3 appendix 4: survey findings

Fundraisers were surveyed about the extent to which the fundraiser measured how their donors feel, how they used the findings (if they did) and what barriers they have come across.

Read more