CDE project 3 appendix 1: measurement of satisfaction and loyalty in action
There are several companies and a few charities who measure how their customers and supporters feel, using some of the tools we have outlined in this document or their own methodologies.
- Written by
- The Commission on the Donor Experience
- May 01, 2017
The MS Society – using Net Promoter Score to drive satisfaction
MS Society measures NPS of all events. An NPS question is included in the post-event survey emailed to participants via SurveyMonkey for all events that the MS Society attends. They also plan to expand this practice to ask all community and events fundraisers as part of their stewardship journeys.
The survey is emailed immediately after an event and followed up in subsequent emails. Currently they achieve a 10-20% response rate, but they are also aiming to improve this by offering incentives to complete the survey and by using printed evaluations given out on the day.
The MS Society has found that:
- NPS is a strong measure of non-financial success of an event and the supporter experience.
- Bespoke events have higher NPS than third-party events – their own sponsored walk had a higher NPS score than all other third-party organised events.
- If the charity has a big presence at the events (e.g. cheer stations, post-race receptions), then this leads to higher NPS scores.
The NPS score is reported in the evaluation for each event. After 2 years, a benchmark score will be created for each event and groups of events by activity. They are testing to quantify how the NPS score relates to repeat participation (short-term impact) and lifetime value (long-term impact).
Because of the success they have had in using measurement of NPS in this area to help them to improve the experience of the participants, the MS Society has recently committed to measuring the NPS of all donors, and fundraising teams and supporter care requests.
Their advice is that the question has to be asked within two weeks of the activity (or experience) that you want to measure to ensure the donor has good recall of it.
NSPCC – measuring Happiness
The NSPCC have long measured satisfaction, but around five years ago they felt that they wanted to measure the elements that make people excited.
They moved to measuring engagement and a year ago created their Happiness Index.
Happiness is measured through a quarterly survey, in order to iron out the influences of any external spikes (e.g. media coverage) and to understand how it varies over the year due to different communications. The survey measures key metrics including:
- Overall satisfaction with the NSPCC and also satisfaction with how they are treated
- Planned future giving
- Whether NSPCC is one of their favourite charities
- Whether they’ve encouraged others to support NSPCC (NPS is accepted as a strong indicator of future giving)
The survey also measures the factors that drive happiness, asking how important these are and how the NSPCC is performing in these areas:
- Satisfaction drivers (e.g. service delivery)
- Emotional drivers (e.g. being thanked, feeling valued, NSPCC cares about my needs)
- Engagement drivers (e.g. receiving relevant and interesting comms, being given opportunity to support NSPCC in other ways, feeling they have a good knowledge of NSPCC)
By comparing importance versus performance, the key areas that need to be improved can be identified. The survey also captures likes and dislikes to help provide insight to improve happiness.
They also undertake quarterly brand tracking with the general public, which allows for the two trackers to be compared.
The survey is then followed up with in-depth interviews, which gather more insight, as well as helping to immerse staff into the survey lessons and outcomes.
The key metrics are reported widely, including to the board. Insight Activations Sessions are held to identify practical actions to increase engagement following the survey.
By measuring Happiness, NSPCC has learnt:
- That supporters like sincere emails about current topics, NSPCC’s TV adverts, NSPCC’s behaviour change campaigns (PANTS / Underwear rule).
- That some donors dislike upgrade calls – the survey provided insight on how these could be improved, leading to an increase in satisfaction with calls.
- Awareness of press stories about staff salaries was not too high, but was highly impactful for those aware of them. Awareness of stories about fundraising was higher, but the negative impact was not as great for those aware of them.
- That demonstrating local impact is growing more important to donors and that the NSPCC can improve in this area.
- That the NSPCC’s donors wanted alternative ways to support the NSPCC.
- That there was a potential issue with the authenticity of the NSPCC’s communications.
The survey has informed changes at NSPCC including:
- Creation of a set of supporter principles to govern communications
- Creation of local case for supports to help tell the local story better
- Careful management of contact frequency – the supporters’ happiness with the frequency of contact has increased
- Investigation into new ways to make supporters feel valued
- Striving to find original and interesting ways to communicate the impact of donations
- A focus on how the NSPCC uses phone calls leading to improved satisfaction
- An ongoing, audience-led innovation programme to develop new ways to provide support
Haven’s Hospices – you don’t need big research budgets
Haven’s Hospices (an adult and children’s hospice based in Southend, Essex) employed a Community Engagement Officer three years ago. His role was to look at the entire region they fundraise in and create opportunities for people to engage with the hospices. This went beyond fundraising into creating meaningful experiences, none of which involved the measurement of short-term financial ROI.
For example, he would bring in a volunteer to events to offer free hand massages while they talked about the role of massage and wellness in therapy. Children were encouraged to make butterflies while the volunteer talked to parents about the hospices’ aim to make every day count.
They don’t have formal structured surveying to measure the effectiveness of these efforts, but Vanessa Longley, Haven’s Hospices’ Director of Fundraising, still has to justify this role to her fellow Directors and to Trustees. She uses two measures:
- Positivity – hearing more positive things about the hospices and monitoring attendance at events. Before the role was created between 30-100 people attended events – this year over 700 people have come, with many people asking for the Community Engagement Officer by name.
- Awareness – tracking prompted and unprompted awareness and the growth in giving consideration.
Vanessa is open about the fact that they haven’t managed to prove the link between this role and increased income, but she is confident that this is one of the reasons behind their growth and so is keen to continue investing in the role and measuring the impact it has on the way people in her area feel.
James Walker – creating their own measures
James Walker is a manufacturing company, with both internal (other companies within the same group) and external customers.
They have initiated a number of ways of tracking how their customers feel.
They run a customer satisfaction survey once a year. Customers are called and asked about their experience of the company and about the relative importance of and satisfaction with various factors. Their golden question is: 'Knowing what you know now, if you were to start again from a clean sheet, how likely or unlikely would you be to buy from us?'
They have also created their own version of NPS. They use 20 emotional words  that have been shown to be important factors in measuring customer satisfaction. These are grouped into three groups: Promoter words, Passive words and Detractor words. By monitoring the frequency with which each of these words is used, they are able to calculate their own NPS score.
They have a full-time Customer Experience specialist to run this project, working alongside colleagues in the business who are passionate about Customer Experience and making a difference. He estimates that he spends at least 75% of his time representing customers internally.
Ecotricity – focussing on the most important thing.
Ecotricity are the UK’s largest green energy supplier. Their vision and values differentiate them from other energy suppliers because they are driven first and foremost by a passion to change the way the UK generates and consumes energy. A desire for profit follows.
This passion and commitment drives everything in the business, from the offices they use, to the sponsorship of the local football club, to the vegan-only canteen.
But it is not this vision that drives customer satisfaction and retention. Ecotricity closely monitor customer conversations on social media and know that customer service is the number-one issue for their customers.
They measure satisfaction with their service as well as service levels. They report on it throughout the business and obsess about every single detail with the aim of constantly improving their score.
Make-A-Wish Australia – Calculating Commitment based on engagements
Make-A-Wish Australia calculate a ‘Commitment Score’ for each of their donors by allocating a specific score (2, 6, 10 etc.) to selected interactions that the donors has with Make-A-Wish, including:
- Communicating with Make-A-Wish Australia (e.g. changing details, leaving feedback etc.)
- Learning about their emotional experience (e.g. connection to the cause, beliefs and values, positive feelings etc.)
- Giving behaviour (e.g. unprompted upgrades, enquiries)
- Donor satisfaction
- Learning their attitude to the organisation (e.g. mission effectiveness, impact and need, like or dislike)
In addition, they run a donor survey, which is also a key source of information to build this score.
Each donor can achieve a maximum Commitment Score of 100. They have found that donors with a higher score who were selected for more personalised communications (e.g. hand-signing) responded better to subsequent upgrade and reactivation asks, but this wasn’t conducted as a firm, statistically valid test.
They intend to use this score as an additional segmentation tool for all communications and to target those who have demonstrated some level of commitment to the charity.
They also intend to develop a trigger-based stewardship programme, through which they can send targeted, automated communications based on changes to scores or in specific measures.