Claims, concerns and issues

When a new AI tool, such as an AI scribe, is planned for your service, the people affected will have hopes, worries and questions. This activity brings them into the open so the team can act on them before the tool arrives, and so evaluation later focuses on what matters to staff and the people they care for.

It draws on fourth generation evaluation (Guba and Lincoln, 1989), in which the people with a stake in a change set out their claims, concerns and issues. The approach is widely used in practice development.

At a glance

• Time: about 45 minutes
• Who: the team, and where possible service users, families or carers, and the people leading the implementation
• Facilitator: a practice educator, practice development lead or experienced colleague
• You need: three flipchart sheets headed Claims, Concerns and Issues, sticky notes, pens
• Output: an action list and a set of questions to evaluate against

The three headings

Heading	Meaning	Example (AI scribe)
Claims	Positive things people believe or hope the tool will bring	"I'll be able to look at the person instead of the screen."
Concerns	Worries or fears about the tool	"People may not speak freely if they know they are being recorded."
Issues	Questions that need an answer before or during use	"Who checks the note before it goes into the record, and how long will that take?"

Running the session

1. Introduce the tool (5 minutes). Explain what is proposed, plainly and honestly, including what is not yet known. Share your team's shared purpose if you have one.
2. Individual responses (10 minutes). Everyone writes claims, concerns and issues on sticky notes, one idea per note, and places them on the sheets.
3. Cluster the themes (10 minutes). Group similar notes under each heading.
4. Prioritise (10 minutes). Agree which concerns and issues must be resolved before the tool is used, and which can be monitored once it is in use.
5. Agree actions (10 minutes). For each priority, agree who will take it forward, by when, and how the team will know it has been addressed.

Prompts to make sure nothing is missed

• The person: consent, privacy, whether people can say no, and how the tool is explained to them
• Equity: accents, dialects, speech differences, interpreters, learning disability, capacity, and staff who speak English as an additional language
• Safety: errors in the record, checking time, what happens when the tool fails
• Professional practice: accountability, record keeping under the NMC Code, maintaining documentation skills
• Workload: whether time saved goes back into care
• Culture: whether it is safe to say "this output is wrong", and how concerns are reported

Turning the results into action

Priority	Action	Who	By when	How we will know
Example: people may not want to be recorded	Agree a script for explaining the tool and offering an alternative	Ward manager with service user representative	Before go-live	Script in use; opt-outs recorded and respected

Using the results to evaluate

The claims, concerns and issues become your evaluation questions. A few months after the tool is introduced, return to them:

• Have the claims come true? For whom?
• Were the concerns justified? What has been done about them?
• Have the issues been resolved? What new issues have emerged?

Share what you learn with the people who took part, including service users, and with those leading the implementation in your organisation.

Reference

• Guba, E.G. and Lincoln, Y.S. (1989) Fourth Generation Evaluation. Newbury Park, CA: Sage.