Can fundrais­ers tell the dif­fer­ence between AI and human-writ­ten thank you letters?

A recent sur­vey chal­lenged fundrais­ers to exam­ine a selec­tion of thank you let­ters. Some were human-writ­ten and oth­er were gen­er­at­ed with arti­fi­cial intel­li­gence (AI). The results were sur­pris­ing. Per­haps for fundrais­ers, AI is great at thank­ful­ness, but grat­i­tude is still a human endeav­our. Keep read­ing to find out more.

Written by
Craig Linton
Added
December 01, 2008
© Suwaree Tangbovornpichet images/ Canva

The fundraising sector is having a heated debate about AI. On one side, you’ll find enthusiastic adopters excited about efficiency gains. On the other, vocal critics are worried about losing the human touch. 

Thread Fundraising recently conducted research that asks a fundamental question: Can fundraisers actually tell the difference between AI-generated and human-written thank you letters? And perhaps more importantly, does it matter?

The starting hypothesis

The research began with two straightforward hypotheses. First, that most people cannot reliably distinguish AI-generated content from human-written text. Second, that even when people suspect AI involvement, they don’t necessarily rate the quality lower.

The rationale was simple and practical: if AI can draft thank you letters that are at least as good as the sector average, fundraisers should embrace it as a tool to free up time for more meaningful stewardship work. The idea is not to replace the human element, but to enhance it.

How the research worked

Thread Fundraising collected real thank you letters from eight UK charities, including organisations like The Lullaby Trust, St Gemma's Hospice, Jerry Green Dog Rescue, and Day One Trauma Support. It takes a brave fundraiser to allow their human-written thank you letters to be compared to a letter written by AI – so we are very grateful to all the fundraisers and charities who bravely shared their letters for our research purposes!

Thread Fundraising then used AI to generate two alternative versions of a thank you letter. One was created using the more basic ChatGPT and the other came from a custom-trained ‘Gratitude Machine’, built on fundraising and thanking best practices.

Then, the letters were used to form a web-based survey which over 300 fundraising professionals went on to complete.

In the first part of the survey, fundraisers were asked to identify if the two control letters (one human and one AI) and the three randomly selected letters were written by humans or AI. 

In the second part of the survey, they were asked to rate the quality of three random letters. 

It’s worth mentioning that we standardised formats for the letters – i.e. making em-dashes the same size in all letters and didn’t include any pictures of lift materials that fundraisers may ordinarily use in a thank you letter. For the two control letters we believed there were subtle clues that one was human and one was AI. 

The detection results: no better than random

The findings were striking. Fundraisers performed no better than they would have by simply flipping a coin. Overall accuracy in detecting AI-generated content hovered around chance levels. Experience made no difference. Fundraisers with over ten years in the sector fared no better than those with less than a year.

Interestingly, people were slightly better at spotting the basic ChatGPT outputs than the custom ‘Gratitude Machine’ letters, which demonstrates the critical importance of proper prompting. Well-trained AI produces more natural-sounding content that even experienced professionals struggle to identify.

The quality ratings: AI wins

Perhaps even more surprising were the quality ratings. Letters generated by the Gratitude Machine scored highest at 7.31 out of 10. Basic ChatGPT letters came in at 7.21. The human-written letters? They averaged 6.26 – a full point lower.

One AI-generated letter for The Lullaby Trust was thought to be AI by 85 per cent of respondents (they were right), yet it still received the fourth-highest quality rating at 7.53 out of 10. This suggests that even when people suspect AI involvement, they don’t automatically downgrade their assessment of the content’s quality.

Based on previous mystery shopping and supporter experience audits by Thread Fundraising, then all the human written letters were considered to be above average examples. 

Thankfulness vs. gratitude: the critical distinction

‘Thankfulness is the beginning of gratitude. Gratitude is the completion of thankfulness. Thankfulness may consist merely of words. Gratitude is shown in acts.’

This distinction is crucial. AI can absolutely help with thankfulness. The words, the acknowledgment, the timely response. It can draft compelling, heartfelt letters that express appreciation effectively. But gratitude? That requires the human touch.

The Day One Trauma Support case study: where humans excel

The research included a compelling case study from Day One Trauma Support that demonstrates what AI can’t replicate. Their income grew from £120,000 to over £500,000 in three years (a 400 per cent increase) – and this was driven by a culture of genuine gratitude.

This wasn’t about perfect thank you letters. It was about handmade thank you cards, personal touches, following supporters on social media, engaging with their stories, and showing up in ways that mattered. These acts of gratitude, the human connections, created a wave of support that no AI could generate on its own.

It meant great retention, word-of-mouth recommendations and increasing levels of support which helped create the environment for growth.

What this means for fundraisers: the future of supporter care?

The takeaway from this survey and its results isn’t that AI should replace human fundraisers. It's that AI can handle the thankfulness, freeing fundraisers to focus on genuine gratitude.

Consider this workflow: 

  • You scan in a donation form or letter from a supporter.
  • AI extracts the information and then adds the information to your CRM (Customer Relationship Management) system or database.
  • AI then drafts a personalised thank you letter based on the donor’s giving history and the specific appeal.
  • A fundraiser reviews and refines it in minutes rather than starting from scratch. 

And how does a fundraiser use that saved time? It goes toward handwritten notes, phone calls, personal videos, or creative touches that show real gratitude.

The bottom line

Using AI shouldn’t be an either/or question. With proper prompting, it’s capable of producing above-average content for thankfulness. But gratitude, putting relationship fundraising into action, that requires human creativity, empathy, and effort.

The most effective approach combines both: let AI handle the efficiency of thankfulness, then invest your saved time in acts of gratitude that only humans can deliver. That combination could be an unstoppable force for good.

Editor’s note: To see some examples of the letters used in this test, and download a more detailed presentation on AI vs human-written thank you letters, please click here

IMAGES: © All images as credited or via Thread Fundraising.

 

About the author: Craig Linton

Craig Linton

Craig Linton (he/him) co-founded Thread Fundraising – an agency that puts supporters at the heart of fundraising – backed with creative AI solutions to ‘superpower’ fundraisers.

He has over 25 years of fundraising experience, including senior roles at organisations like Sue Ryder and Amnesty International, and has a track record of growing fundraising income. Since becoming a consultant, over 80 per cent of clients have extended their contract or commissioned a second piece of work with Craig.

Craig is a member of the Critical Fundraising Network at Rogare and has previously served as a trustee for SOFII and Thames Hospice. He is also a co-author of the critically acclaimed book Donors for Life: A Practitioner’s Guide to Relationship Fundraising, and a long-time contributor to the fundraising community through blogs, webinars, and thought pieces.

Related case studies or articles

Charity Right: ‘AI vs real people’ image test

There are a lot of potential uses for artificial intelligence (AI) in fundraising. Jamal Abbas shares how one charity set out to discover what worked better in an online fundraising campaign, AI generated images or those taken by a team of skilled photographers. 

Read more

ChatGPT and fundraising – what do you need to know? (part one)

You’ve probably heard about ChatGPT. You might have read about how it could revolutionise (or destroy) everything from internet searches to journalism. But as fundraisers, what do you really need to know? In part one of a two-part piece, we ask three digital fundraisers to give us their opinions on ChatGPT. 

Read more

ChatGPT and fundraising – what do you need to know? (part two)

In the second instalment of our series on ChatGPT and fundraising, SOFII talks to Cherian Koshy – the founder of a platform that democratises access to artificial intelligence and machine learning for nonprofit organisations. Dive in and explore Cherian’s in-depth insights and opinions on ChatGPT for fundraisers. 

Read more

Charity Right: prueba de imágenes “IA vs personas reales”

There are a lot of potential uses for artificial intelligence (AI) in fundraising. Jamal Abbas shares how one charity set En este estudio de caso, Jamal Abbas comparte cómo una organización decidió averiguar qué funciona mejor en una campaña de procuración de fondos en línea: ¿imágenes generadas por IA o fotografías tomadas por su equipo profesional? ¿Y cómo saberlo con certeza? Haciendo, por supuesto, una prueba A/B.

Sigue leyendo para descubrir cómo Jamal analiza el impacto de la IA en la procuración de fondos, la imagen, la ética y la analítica.

Read more