When a private company ships an AI feature that gets something wrong, a customer is annoyed. When a public service does, a citizen may lose a benefit, a court date or a housing offer. That asymmetry is why trust has to be designed into public sector AI from the first sketch, and why the Service Standard, the Algorithmic Transparency Recording Standard and the ICO’s guidance exist.
Having taken services through GDS alpha and beta assessments, and designed AI proofs of concept for health and justice, these are the things we have learned to get right early.
Tell people AI is involved, in their words
Users have a right to know when an automated system has shaped a decision about them. The transparency record is the formal version. The design version is a sentence on the page, in plain language, that has been tested with real users. “This summary was generated automatically from the documents you uploaded. A case worker will check it before any decision is made.” If participants cannot explain back what that means, rewrite it.
Keep the human visible and reachable
Every AI touchpoint needs a clearly signposted route to a person, and the person needs the context to help. “Ask to speak to someone” that leads to a generic call centre is not a route. On the blueprint, draw the hand-off: what the human sees, what they can override and how the user is told.
Test fairness with the people most affected
Outcomes must be checked across user groups, and that means recruiting for them: people with access needs, people whose first language is not English, people in vulnerable circumstances. On a benefits discovery we ran for the Department for Work and Pensions, recruiting for accessibility needs changed the service design fundamentally. An AI feature tested only on confident digital users will fail the people a public service exists for.
Design for being wrong
Models are wrong some of the time. The design question is whether users can tell and recover. Show sources. Show confidence where it is meaningful. Make it easy to challenge, correct and escalate. Measure how often that happens, because the challenge rate is one of your best signals of quality after launch.
Treat governance as part of delivery, not a gate
Data protection impact assessments, transparency records and bias testing are faster when they grow with the service. We draft the DPIA during discovery, update it in alpha and finish it in beta, alongside the research. Teams that leave governance to the end meet it as a wall.
Prepare for the assessment early
Assessors will ask why AI was the right tool, how you know it works for all users, what happens when it fails and who is accountable. If those answers are on your blueprint and in your research, the assessment is a conversation. If they are not, no amount of polish on the prototype will help.
Trust is not a disclaimer. It is the sum of a hundred design decisions, most of them small. If you are building AI into a public or regulated service, our responsible AI and public sector work is designed to help you make them well.
