CDE project 3 appen­dix 1: mea­sure­ment of sat­is­fac­tion and loy­al­ty in action

There are sev­er­al com­pa­nies and a few char­i­ties who mea­sure how their cus­tomers and sup­port­ers feel, using some of the tools we have out­lined in this doc­u­ment or their own methodologies.

Written by
The Commission on the Donor Experience
Added
May 01, 2017

The MS Society – using Net Promoter Score to drive satisfaction

MS Society measures NPS of all events. An NPS question is included in the post-event survey emailed to participants via SurveyMonkey for all events that the MS Society attends. They also plan to expand this practice to ask all community and events fundraisers as part of their stewardship journeys. 

The survey is emailed immediately after an event and followed up in subsequent emails. Currently they achieve a 10-20% response rate, but they are also aiming to improve this by offering incentives to complete the survey and by using printed evaluations given out on the day.

The MS Society has found that:

  • NPS is a strong measure of non-financial success of an event and the supporter experience.
  • Bespoke events have higher NPS than third-party events – their own sponsored walk had a higher NPS score than all other third-party organised events.
  • If the charity has a big presence at the events (e.g. cheer stations, post-race receptions), then this leads to higher NPS scores.

The NPS score is reported in the evaluation for each event. After 2 years, a benchmark score will be created for each event and groups of events by activity. They are testing to quantify how the NPS score relates to repeat participation (short-term impact) and lifetime value (long-term impact).

Because of the success they have had in using measurement of NPS in this area to help them to improve the experience of the participants, the MS Society has recently committed to measuring the NPS of all donors, and fundraising teams and supporter care requests.

Their advice is that the question has to be asked within two weeks of the activity (or experience) that you want to measure to ensure the donor has good recall of it.

NSPCC – measuring Happiness

The NSPCC have long measured satisfaction, but around five years ago they felt that they wanted to measure the elements that make people excited.

They moved to measuring engagement and a year ago created their Happiness Index.

Happiness is measured through a quarterly survey, in order to iron out the influences of any external spikes (e.g. media coverage) and to understand how it varies over the year due to different communications. The survey measures key metrics including:

  • Overall satisfaction with the NSPCC and also satisfaction with how they are treated 
  • Trust
  • Commitment 
  • Planned future giving 
  • Whether NSPCC is one of their favourite charities
  • Whether they’ve encouraged others to support NSPCC (NPS is accepted as a strong indicator of future giving)

The survey also measures the factors that drive happiness, asking how important these are and how the NSPCC is performing in these areas:

  • Satisfaction drivers (e.g. service delivery)
  • Emotional drivers (e.g. being thanked, feeling valued, NSPCC cares about my needs)
  • Engagement drivers (e.g. receiving relevant and interesting comms, being given opportunity to support NSPCC in other ways, feeling they have a good knowledge of NSPCC)

By comparing importance versus performance, the key areas that need to be improved can be identified. The survey also captures likes and dislikes to help provide insight to improve happiness.

They also undertake quarterly brand tracking with the general public, which allows for the two trackers to be compared.

The survey is then followed up with in-depth interviews, which gather more insight, as well as helping to immerse staff into the survey lessons and outcomes.

The key metrics are reported widely, including to the board. Insight Activations Sessions are held to identify practical actions to increase engagement following the survey.

By measuring Happiness, NSPCC has learnt:

  • That supporters like sincere emails about current topics, NSPCC’s TV adverts, NSPCC’s behaviour change campaigns (PANTS / Underwear rule).
  • That some donors dislike upgrade calls – the survey provided insight on how these could be improved, leading to an increase in satisfaction with calls. 
  • Awareness of press stories about staff salaries was not too high, but was highly impactful for those aware of them. Awareness of stories about fundraising was higher, but the negative impact was not as great for those aware of them.
  • That demonstrating local impact is growing more important to donors and that the NSPCC can improve in this area. 
  • That the NSPCC’s donors wanted alternative ways to support the NSPCC.
  • That there was a potential issue with the authenticity of the NSPCC’s communications. 

The survey has informed changes at NSPCC including:

  • Creation of a set of supporter principles to govern communications 
  • Creation of local case for supports to help tell the local story better 
  • Careful management of contact frequency – the supporters’ happiness with the frequency of contact has increased 
  • Investigation into new ways to make supporters feel valued
  • Striving to find original and interesting ways to communicate the impact of donations
  • A focus on how the NSPCC uses phone calls leading to improved satisfaction 
  • An ongoing, audience-led innovation programme to develop new ways to provide support 

Haven’s Hospices – you don’t need big research budgets

Haven’s Hospices (an adult and children’s hospice based in Southend, Essex) employed a Community Engagement Officer three years ago. His role was to look at the entire region they fundraise in and create opportunities for people to engage with the hospices. This went beyond fundraising into creating meaningful experiences, none of which involved the measurement of short-term financial ROI.

For example, he would bring in a volunteer to events to offer free hand massages while they talked about the role of massage and wellness in therapy. Children were encouraged to make butterflies while the volunteer talked to parents about the hospices’ aim to make every day count.

They don’t have formal structured surveying to measure the effectiveness of these efforts, but Vanessa Longley, Haven’s Hospices’ Director of Fundraising, still has to justify this role to her fellow Directors and to Trustees. She uses two measures:

  • Positivity – hearing more positive things about the hospices and monitoring attendance at events. Before the role was created between 30-100 people attended events – this year over 700 people have come, with many people asking for the Community Engagement Officer by name.
  • Awareness – tracking prompted and unprompted awareness and the growth in giving consideration.

Vanessa is open about the fact that they haven’t managed to prove the link between this role and increased income, but she is confident that this is one of the reasons behind their growth and so is keen to continue investing in the role and measuring the impact it has on the way people in her area feel.

James Walker – creating their own measures

James Walker is a manufacturing company, with both internal (other companies within the same group) and external customers.

They have initiated a number of ways of tracking how their customers feel. 

They run a customer satisfaction survey once a year. Customers are called and asked about their experience of the company and about the relative importance of and satisfaction with various factors. Their golden question is: 'Knowing what you know now, if you were to start again from a clean sheet, how likely or unlikely would you be to buy from us?'

They have also created their own version of NPS. They use 20 emotional words [1] that have been shown to be important factors in measuring customer satisfaction. These are grouped into three groups: Promoter words, Passive words and Detractor words. By monitoring the frequency with which each of these words is used, they are able to calculate their own NPS score.

They have a full-time Customer Experience specialist to run this project, working alongside colleagues in the business who are passionate about Customer Experience and making a difference. He estimates that he spends at least 75% of his time representing customers internally.

Ecotricity – focussing on the most important thing.

Ecotricity are the UK’s largest green energy supplier. Their vision and values differentiate them from other energy suppliers because they are driven first and foremost by a passion to change the way the UK generates and consumes energy. A desire for profit follows.

This passion and commitment drives everything in the business, from the offices they use, to the sponsorship of the local football club, to the vegan-only canteen.

But it is not this vision that drives customer satisfaction and retention. Ecotricity closely monitor customer conversations on social media and know that customer service is the number-one issue for their customers.

They measure satisfaction with their service as well as service levels. They report on it throughout the business and obsess about every single detail with the aim of constantly improving their score.

Make-A-Wish Australia – Calculating Commitment based on engagements

Make-A-Wish Australia calculate a ‘Commitment Score’ for each of their donors by allocating a specific score (2, 6, 10 etc.) to selected interactions that the donors has with Make-A-Wish, including:

  • Communicating with Make-A-Wish Australia (e.g. changing details, leaving feedback etc.)
  • Learning about their emotional experience (e.g. connection to the cause, beliefs and values, positive feelings etc.)
  • Giving behaviour (e.g. unprompted upgrades, enquiries)
  • Donor satisfaction 
  • Learning their attitude to the organisation (e.g. mission effectiveness, impact and need, like or dislike)

In addition, they run a donor survey, which is also a key source of information to build this score.

Each donor can achieve a maximum Commitment Score of 100. They have found that donors with a higher score who were selected for more personalised communications (e.g. hand-signing) responded better to subsequent upgrade and reactivation asks, but this wasn’t conducted as a firm, statistically valid test.

They intend to use this score as an additional segmentation tool for all communications and to target those who have demonstrated some level of commitment to the charity.

They also intend to develop a trigger-based stewardship programme, through which they can send targeted, automated communications based on changes to scores or in specific measures.

Click on the image below to see Project 3 in full - PDF format

About the author: The Commission on the Donor Experience

The CDE has one simple ideal – to place donors at the heart of fundraising. The aim of the CDE is to support the transformation of fundraising, to change the culture to a truly consistent donor-based approach to raising money. It is based on evidence drawn from first hand insight of best practice. By identifying best practice and capturing examples, we will enable these to be shared and brought into common use.

Related case studies or articles

CDE project 3 summary: satisfaction and commitment

This project will create a practical guide to measuring other main indices of success than money or RFV and will show where relevant research and examples of best practice can be found.

Read more

CDE project 3 section 1: actions and approach

This section may be called Measurement, but it’s actually about culture, strategy and people. 

Read more

CDE project 3 section 2: putting the principles and actions into practice - step 1 and 2

Step 1: Make a commitment to growing donor satisfaction and loyalty

Read more

CDE project 3 section 3: Making it happen

We have seen that there are lots of things that you can measure, but you don’t have to measure them all. Just choose one! 

Read more

CDE project 3 appendix 2: frequently asked questions

In our research (see Appendix 4), several people mentioned barriers to measuring how their donors feel. This section lists the most common of these and gives our advice.

Read more

CDE project 3 appendix 3: sample questions

A quick Google search will bring up many sample questionnaires, but for ease we have included a couple here.

Read more

CDE project 3 appendix 4: survey findings

Fundraisers were surveyed about the extent to which the fundraiser measured how their donors feel, how they used the findings (if they did) and what barriers they have come across.

Read more