Hel­lo! Test­ing! Test­ing! Testing!

The lack of con­sis­ten­cy in the way char­i­ties adjust­ed their asks dur­ing the eco­nom­ic down­turn is quite surprising.

Written by
Andrew Papworth
Added
May 18, 2013

Some simply ploughed on regardless not changing a thing. Some reduced their asks for a couple of years and then returned to their earlier, higher ask – presumably because the change didn’t work.

Others raised their asks for a considerable time and then reduced them again – presumably that didn’t work either. Yet others were simply all over the place.

With a bit of disciplined planning it is actually very easy to discover quite quickly and definitively the combination of asks which works best at any given time. The answer is, of course, the tried and tested A & B split run. Take the Sightsavers examples shown here (no idea if they were split-run tested or not), so long as they can estimate the lifetime value of a donor and know that they get, on average, x per cent of their total response to a newspaper ad within five days of publication, they can make a pretty good stab at which half of a split run has ‘won’ within a week. 

This speed of analysis is vital for three reasons: first it saves you from running too long with an unsuccessful ad; second it means that, once you’ve tested a successful ad, you need waste no time in rolling it out to take advantage of its enhanced performance and third, very importantly, it can give you early warning when your banker ad inevitably hits the ‘wear-out’ wall and allow you to pull it immediately to avoid a disaster.

Sadly, too many charities do their response analysis on a campaign basis without split runs – leaving it to the end of the campaign to see which copy or which ask variant has come out top. This can be disastrously misleading, not only for the reverse of the reasons mentioned above but because the response to individual inserts can be influenced by so many extraneous factors that it renders the comparisons worse than useless – even downright damaging.

As invaluable as A & B split-runs are, they are not without their dangers. For absolutely reliable results it is imperative that only one element of an ad is tested at a time. The temptation to kill two birds with one stone must be resisted because it will only cloud the issue. The Sightsavers ads above, for example, would not make an ideal split-run test because it would be not just testing TEXT v. coupon but one means of response (TEXT) v. two (FREEPOST coupon and 0800 telephone number) and because the TEXT version says £3 will treat three children but the couponed versions says £17.50 will treat 50 children. Apart from the fact that the sums don’t compute, it would be impossible to separate the elements causing one version to perform better than the other.

Because newspapers typically charge 10 per cent extra for a split-run, such testing can seem an expensive luxury but it can pay for itself many times over – and a good buyer may even be able to cut a deal. A very cost-effective solution can be to use loose inserts, which can easily be printed – on a long enough run – with as many as five tests interleaved in order to try five separate elements or copy variants. Loose inserts can be more cost-effective than ads anyway and give you much greater creative scope.

Even small changes can, and should be, tested, such as reversing the order of a range of asks from highest-to-lowest, adding examples of what the money might buy, changing the number of steps in the range of asks or the distances between each step, etc.

I recommend the high assay approach to off-the-page fundraising. This involves putting the bulk of your budget behind your proven banker ad but setting aside, say, around 20 per cent for a carefully planned programme of testing using split-runs – to cover both new creative strategies and the nuts and bolts of ways of improving the performance of the banker. It is a fool-proof way of optimising performance, insuring against unexpected disasters and minimising the deleterious impact of political, economic, competitive and social changes outside your control.

If you’re not using split-runs systematically, you should be. There is no alternative.

About the author: Andrew Papworth

Andrew Papworth

After a long career in advertising agencies, Andrew Papworth has been freelancing as an advertising and communications planner for about two decades.

Recent Articles

Will you join these kind fundraisers at the heart of SOFII?

Who are the SOFII One Hundred? They are fantastic fundraisers from around the world – and you can find them at the heart of SOFII. Read on to discover more about this group of caring people and learn why they have chosen to help fundraisers today, tomorrow and forever.

Read more

New year, new changes in digital fundraising – five tips to help you prepare

Changes in online marketing and data privacy have made it harder than ever to find new donors. And as we embark on a new year in fundraising, more shifts are on the horizon. So what can fundraisers like you do to make sure you’re prepared to communicate with and nurture your donors in an ever-evolving digital landscape?

Read more

Legacy brochures from 90 years ago – what’s the same and what’s different to today?

This is a rare ‘legacy edition’ from our ever-growing fundraising history project. Marina Jones has uncovered some gems in The Salvation Army archive, two legacy brochures from the 1930s. In this article she looks at what’s stayed the same (Marina found ten things!), as well as what’s changed, when it comes to talking to donors about making a gift in their will.

Read more

When fundraising collapsed – are we ready for a new era of responsible fundraising?

As we enter another year in fundraising, Giles Pegram is pausing to look back. In this article, he reflects on some important moments in his time as a fundraiser – from 1983 to the present. Giles also refers to the lessons found in the Commission on the Donor Experience and offers fundraisers ten tips that he believes will help you usher in a new era of responsible fundraising.

Read more

SOFII’s best of the best showcase - introduction and contents

This showcase collates all the greatest fundraising campaigns currently on SOFII. Be prepared to be inspired and amazed by the very best fundraising ever. And you can suggest other appeals or projects that resonated with you as well.

Read more

Also in Categories