Two years ago, testing an AI idea meant a data science team, a budget and a quarter of elapsed time. Today a service designer with the right tools can put a working prototype in front of users within a week. We have done it for our own product, MyStoryboarder, and for Compass, a proof-of-concept procurement service for the Department of Health and Social Care. These are the lessons.
1. Decide what you are proving
A proof of concept proves one thing. For MyStoryboarder it was: can current image models produce storyboards with consistent characters that a producer would actually use? For Compass it was: will a clinician trust a summary of device evidence enough to act on it? Everything that does not serve that question is left out, however tempting.
2. Use real data, carefully
A prototype on made-up data proves nothing. Use a sample of the real thing, with the permissions to match. For Compass that meant published evidence and anonymised examples. For anything involving personal data, get the legal basis sorted before you build, not after.
3. Prompt design is content design
The quality of an AI feature is mostly the quality of its instructions and the shape of its outputs. Writing them is content design: plain language, clear structure, explicit handling of “I don’t know”. Our best prompts were written by designers, not engineers, and tested like any other copy.
4. Show your sources
Every summary Compass produced linked to where the evidence came from. Users told us this single feature was the difference between “interesting” and “I would use this”. It is also what an assessment panel will ask about first.
5. Build the failure path first
What does the prototype do when the model is unsure, wrong or asked something out of scope? Design that before the happy path. Participants in testing go off-script within minutes, and a prototype that collapses when they do tells you nothing about the idea.
6. Test with the people who will use it
Ten moderated sessions with representative users, including people with accessibility needs, tell you more than any internal review. Watch for trust: do they check the output, do they over-trust it, do they understand that AI is involved? On MyStoryboarder we learned that users wanted to edit, not regenerate, which changed the product.
7. Measure cost per interaction
Model calls cost money and the bill scales with use. Measure it in the prototype so the business case is real. More than one exciting demo has died when someone multiplied the cost by the user base.
8. End with a decision
The output of a proof of concept is not a prototype. It is a decision: proceed, pivot or stop, with evidence. Compass gave the department a blueprint, tested prototypes and a clear view of what a full build would need. MyStoryboarder went to a beta launch with backing from Amazon. Both were decisions we could defend.
If you have an AI idea and want to know within six weeks whether it is worth building, our rapid AI proof of concept service is built on exactly this process.
