Can fundraisers tell the difference between AI and human-written thank you letters?
A recent survey challenged fundraisers to examine a selection of thank you letters. Some were human-written and other were generated with artificial intelligence (AI). The results were surprising. Perhaps for fundraisers, AI is great at thankfulness, but gratitude is still a human endeavour. Keep reading to find out more.
- Written by
- Craig Linton
- Added
- December 01, 2008
The fundraising sector is having a heated debate about AI. On one side, you’ll find enthusiastic adopters excited about efficiency gains. On the other, vocal critics are worried about losing the human touch.
Thread Fundraising recently conducted research that asks a fundamental question: Can fundraisers actually tell the difference between AI-generated and human-written thank you letters? And perhaps more importantly, does it matter?
The starting hypothesis
The research began with two straightforward hypotheses. First, that most people cannot reliably distinguish AI-generated content from human-written text. Second, that even when people suspect AI involvement, they don’t necessarily rate the quality lower.
The rationale was simple and practical: if AI can draft thank you letters that are at least as good as the sector average, fundraisers should embrace it as a tool to free up time for more meaningful stewardship work. The idea is not to replace the human element, but to enhance it.
How the research worked
Thread Fundraising collected real thank you letters from eight UK charities, including organisations like The Lullaby Trust, St Gemma's Hospice, Jerry Green Dog Rescue, and Day One Trauma Support. It takes a brave fundraiser to allow their human-written thank you letters to be compared to a letter written by AI – so we are very grateful to all the fundraisers and charities who bravely shared their letters for our research purposes!
Thread Fundraising then used AI to generate two alternative versions of a thank you letter. One was created using the more basic ChatGPT and the other came from a custom-trained ‘Gratitude Machine’, built on fundraising and thanking best practices.
Then, the letters were used to form a web-based survey which over 300 fundraising professionals went on to complete.
In the first part of the survey, fundraisers were asked to identify if the two control letters (one human and one AI) and the three randomly selected letters were written by humans or AI.
In the second part of the survey, they were asked to rate the quality of three random letters.
It’s worth mentioning that we standardised formats for the letters – i.e. making em-dashes the same size in all letters and didn’t include any pictures of lift materials that fundraisers may ordinarily use in a thank you letter. For the two control letters we believed there were subtle clues that one was human and one was AI.
The detection results: no better than random
The findings were striking. Fundraisers performed no better than they would have by simply flipping a coin. Overall accuracy in detecting AI-generated content hovered around chance levels. Experience made no difference. Fundraisers with over ten years in the sector fared no better than those with less than a year.
Interestingly, people were slightly better at spotting the basic ChatGPT outputs than the custom ‘Gratitude Machine’ letters, which demonstrates the critical importance of proper prompting. Well-trained AI produces more natural-sounding content that even experienced professionals struggle to identify.
The quality ratings: AI wins
Perhaps even more surprising were the quality ratings. Letters generated by the Gratitude Machine scored highest at 7.31 out of 10. Basic ChatGPT letters came in at 7.21. The human-written letters? They averaged 6.26 – a full point lower.
One AI-generated letter for The Lullaby Trust was thought to be AI by 85 per cent of respondents (they were right), yet it still received the fourth-highest quality rating at 7.53 out of 10. This suggests that even when people suspect AI involvement, they don’t automatically downgrade their assessment of the content’s quality.
Based on previous mystery shopping and supporter experience audits by Thread Fundraising, then all the human written letters were considered to be above average examples.
Thankfulness vs. gratitude: the critical distinction
‘Thankfulness is the beginning of gratitude. Gratitude is the completion of thankfulness. Thankfulness may consist merely of words. Gratitude is shown in acts.’
This distinction is crucial. AI can absolutely help with thankfulness. The words, the acknowledgment, the timely response. It can draft compelling, heartfelt letters that express appreciation effectively. But gratitude? That requires the human touch.
The Day One Trauma Support case study: where humans excel
The research included a compelling case study from Day One Trauma Support that demonstrates what AI can’t replicate. Their income grew from £120,000 to over £500,000 in three years (a 400 per cent increase) – and this was driven by a culture of genuine gratitude.
This wasn’t about perfect thank you letters. It was about handmade thank you cards, personal touches, following supporters on social media, engaging with their stories, and showing up in ways that mattered. These acts of gratitude, the human connections, created a wave of support that no AI could generate on its own.
It meant great retention, word-of-mouth recommendations and increasing levels of support which helped create the environment for growth.
What this means for fundraisers: the future of supporter care?
The takeaway from this survey and its results isn’t that AI should replace human fundraisers. It's that AI can handle the thankfulness, freeing fundraisers to focus on genuine gratitude.
Consider this workflow:
- You scan in a donation form or letter from a supporter.
- AI extracts the information and then adds the information to your CRM (Customer Relationship Management) system or database.
- AI then drafts a personalised thank you letter based on the donor’s giving history and the specific appeal.
- A fundraiser reviews and refines it in minutes rather than starting from scratch.
And how does a fundraiser use that saved time? It goes toward handwritten notes, phone calls, personal videos, or creative touches that show real gratitude.
The bottom line
Using AI shouldn’t be an either/or question. With proper prompting, it’s capable of producing above-average content for thankfulness. But gratitude, putting relationship fundraising into action, that requires human creativity, empathy, and effort.
The most effective approach combines both: let AI handle the efficiency of thankfulness, then invest your saved time in acts of gratitude that only humans can deliver. That combination could be an unstoppable force for good.
Editor’s note: To see some examples of the letters used in this test, and download a more detailed presentation on AI vs human-written thank you letters, please click here.
IMAGES: © All images as credited or via Thread Fundraising.