AI answer design: sources, uncertainty and user control
Design AI answers people can inspect and correct. A practical guide to sources, uncertainty, approval states, usability testing and agency briefs.
An answer can be accurate and still be hard to use. A support agent needs to know whether a policy applies to this customer. A researcher needs to see where a summary came from. A product manager needs to understand which parts of a recommendation depend on missing information. Fluent text does not settle any of those questions.
AI answer design is the work of making a generated result understandable, inspectable and useful within a real task. It includes sources, uncertainty, editing, approval and what happens after someone acts. These details belong in the product brief alongside the model and retrieval work.
This article follows an illustrative customer-support assistant. It drafts replies from approved product documentation, and a member of staff decides what to send. The example is an editorial exercise, not a client deployment or a claim about measured results. You can use the same review method for internal search, research tools and other document-backed AI products, adjusting it to the consequences of the task.
Keep in mind
Key takeaways
- Design the review decision, source access and correction flow together.
- Describe missing context and conflicting evidence with useful next actions.
- Test realistic mistakes and make approvals refer to a specific version and action.
Start with the decision after the answer
Write down the decision the interface needs to support. In our support example, the user must check whether the proposed product fact is correct, whether it applies to the customer's situation and whether the wording is appropriate to send. Those are three judgments with different information needs.
A generic approval button hides that distinction. Someone may like the tone while disagreeing with the product detail. Another person may accept the fact but need an exception confirmed by a colleague. If both click the same positive-feedback icon, the product team cannot tell what they approved. Give the operational decision its own control and keep product feedback separate.
Work through one actual task with the people responsible for it. Ask them to show the question, the material they consult today and the evidence they need before replying. Watch where they leave the current system, what they copy into notes and which decisions they escalate. These observations help define the review surface more precisely than a request to make a chatbot feel trustworthy.
Turn the findings into a short acceptance statement. For example: a support agent can locate the applicable policy passage, correct a draft and see the recipient before sending. That statement can guide research, interaction design and implementation. It also gives the team something observable to test when the prototype is ready.
Separate the answer from the assumptions behind it
A long paragraph can mix established information, interpretation and guesses so smoothly that the user cannot distinguish them. Consider a draft that says a customer is eligible for an upgrade because their account appears to meet a usage threshold. The eligibility rule may be documented, while the account information is incomplete. The interface needs to expose that difference.
Structure the response around the task rather than around the model's paragraph breaks. A useful support layout might show the proposed reply, the product facts used and any customer details still needed. Keep that structure short enough to scan. If every sentence becomes a card with a warning, the reviewer has another classification job before they can answer the customer.
Choose labels that describe the information. Labels such as “Policy passage,” “Account detail needed” and “Draft reply” say more than “AI insight.” Use the same vocabulary as the surrounding application. An assistant inside a support console should not introduce unfamiliar names for concepts the staff already understand.
Avoid presenting an explanation of how the model might have reasoned as proof that the answer is correct. The reviewer needs relevant evidence and clear assumptions. A plausible explanation can be useful context, but the design should not make it look equivalent to an approved policy, a recorded account fact or a checked calculation.
Put the source passage within reach
A citation helps only if it leads to material that supports the claim. A document title may be enough for a familiar two-page policy. It is less useful when the reviewer must search a long manual to discover which passage the system used. Decide what source context the task requires before choosing a citation style.
For our example, connect the eligibility claim to the relevant paragraph in the approved policy. Show its title and version, and provide enough surrounding text to preserve any exceptions. When the same answer draws from several documents, make the relationship clear. The user should not have to guess which source supports which part of the reply.
Treat source access as part of the interaction. If the reviewer cannot open a document, explain that restriction and provide an allowed next step. Do not expose restricted passages in a preview simply because the full document is protected elsewhere. Ask engineering to demonstrate the source panel with different user roles, not just an administrator account.
Review the experience when the source disappears or changes. A saved answer may refer to a policy that has since been replaced. The interface can preserve the version used for the draft and flag that a fresh check is needed. The exact retention and access rules depend on the product, but the distinction between the old draft and current material should be visible.
Describe what is missing without decorating a guess
Uncertainty becomes useful when it changes the next step. In the support example, the assistant may know the upgrade policy but lack the customer's contract date. A useful response identifies the missing date and asks for it. A vague warning that AI can make mistakes leaves the reviewer to discover the missing condition alone.
Design separate states for different problems. Missing input, conflicting documents, an unavailable integration and no supporting evidence need different recovery paths. Giving all four the same “Try again” button suggests that regenerating the text will solve an information problem. Sometimes the appropriate next step is to select a document, request an account detail or ask a policy owner.
Be careful with numerical confidence labels. If a product shows a percentage, the team should be able to explain what it measures, how it was evaluated and what the user should do differently at different values. A model's self-description is not enough to establish that a number represents the chance that a business answer is correct.
Microsoft's HAX guidance includes clarifying an uncertain goal or reducing the scope of what the system does. In this design exercise, that means offering a useful partial result and asking for the detail required to continue. The source supports the principle; it does not prescribe a particular warning colour, layout or approval rule for your product.
Reference: HAX: Scope services when in doubt ↗
Let people repair one part without losing the rest
When part of a draft is wrong, regeneration is often a blunt response. The support agent may need to change one product name, remove an unsupported promise or adjust the tone. Starting again can replace correct material and force the person to repeat the entire review.
Provide a direct edit where that suits the task. Make it clear whether an edit changes only this draft, updates a saved template or becomes feedback for a later product change. These effects should not be bundled into an unexplained button. People need to know whether they are correcting today's reply or altering a shared system.
Preserve a useful revision history. The reviewer should be able to compare the generated draft with the edited version without studying a dense technical log. If an edit changes a factual claim, make it possible to revisit the supporting passage. Do not imply that the original citation automatically supports any wording the user adds afterward.
HAX's correction guidance supports making partially incorrect output easier to edit or recover from. A practical design test is to insert one deliberate error into an otherwise useful draft. Ask participants to find it, fix it and finish the task. Notice whether the interface makes them discard good work, lose their place or accidentally accept the original wording.
Reference: HAX: Support efficient correction ↗
Specify the states a component library must cover
An answer component needs more than a loading state and a successful result. Document what the reviewer can see and do when the system has incomplete evidence, when a source cannot be reached and when a corrected draft is ready for approval. Include the state after the final action as well.
The following table is a starting point for the illustrative support assistant. It is not a universal design system. Adapt the state names and available actions to the workflow you are building.
Scroll to view all columns →
| State | What the reviewer needs | Useful next action |
|---|---|---|
| Draft ready | Reply, source passage and recipient | Check, edit or approve |
| Context missing | The specific missing customer detail | Supply it or request help |
| Sources conflict | The conflicting passages and versions | Resolve with the policy owner |
| Source unavailable | What could not be checked | Retry retrieval or use another route |
| Approved draft changed | What changed since approval | Review the new version |
Specify how the component behaves with keyboard navigation, screen readers, zoom and long source names. A colour change alone should not carry the state. The label and action need to remain understandable when the page is read in a different order or a person cannot distinguish the chosen colours.
Also include realistic volume. A short answer with one source may fit comfortably beside the customer record, while a longer answer can push the approval action off screen. Test the cases you expect people to use, not only the example that looks best in a presentation.
Make approval refer to a particular version and action
Approval is easy to misunderstand when the draft continues to change. A staff member may review the text, then add a sentence or switch the recipient before sending. Decide which changes require another check and which can be made within the same responsibility. The answer depends on the workflow, not on whether the text was generated.
Show what the final control will do. “Send reply to Jordan” is more informative than “Continue” when the next step sends a message. If the assistant only prepares a draft in another system, say that. If it commits a change, make the destination and effect visible before the person authorizes it.
Separate feedback from authority. A thumbs-up can report that an answer was useful; it should not quietly authorize an external action. Similarly, a note that an answer was reviewed does not establish who approved publication or whether the reviewed version matches the current text. The product needs a clear relationship between the version, the approver and the action.
For an assistant with tools, review these interactions with the engineering team responsible for authorization. The interface should reflect the actual controls, including expired approvals and rejected actions. A polished confirmation screen cannot compensate for a backend that accepts an old approval for a materially different request.
Test review behaviour with realistic mistakes
Ask participants to complete a task rather than rate whether an answer looks trustworthy. Give them the customer's question, an account record and the same source material they would normally use. Include a correct draft, a draft with one unsupported claim and a case where the required information is absent.
Observe what they check before acting. Can they locate the passage that matters? Do they notice an exception? Can they correct the draft without losing its source context? Record misunderstandings about approval, source access and regeneration. These observations reveal design work that a satisfaction score by itself may not explain.
Use mistakes that are plausible within the task. An obviously absurd answer is easy to reject and tells you little about the review interface. A wrong eligibility date or a missing policy condition is closer to the kind of detail a support agent needs to inspect. Label any deliberately altered material clearly in research records so it is not reused as a product fact.
Do not turn a small usability exercise into a claim about production reliability. It can help you understand how people interact with the tested design. It does not establish how often the model will fail, how retrieval will perform across the document collection or what happens under live operating conditions. Those questions need their own evaluation work.
Related reading: Build an AI evaluation dataset
Measure whether review produces a usable result
Choose measures that match the original acceptance statement. For this assistant, the team might observe whether a reviewer finds the policy condition, corrects the planted error and sends the intended version to the intended recipient. Time spent reviewing can provide context, but a fast incorrect approval is not a good outcome.
Keep the observations separate. An answer may be factually supported but poorly phrased. A reviewer may reach the right decision despite a confusing interface. If the team collapses those outcomes into one broad quality score, it becomes harder to decide whether to improve the source material, model behaviour, layout or operating instructions.
Compare results across the tasks and user roles you actually tested. A new support agent and an experienced policy specialist may need different levels of context. Look for where the design requires knowledge that the intended user does not have. Avoid treating every extra click as waste when that step helps someone verify an important condition.
Keep a small record of the interface version, test scenario, source version and observed problem. When a later design changes the source panel or approval flow, repeat the relevant task. That record connects design iteration with evidence instead of allowing visual preference to decide whether the product has improved.
Walk through an incomplete answer from start to finish
Suppose the fictional support assistant drafts a reply saying that an account can use a particular export feature. The approved policy permits that feature for a named account type, but the customer's message does not identify their account type. The draft has turned a missing condition into an eligibility claim.
Start the review at the statement the user might send. The interface should let the support agent reach the applicable passage and notice the qualifying condition. A source panel that highlights only the feature name would leave the important part of the check incomplete. Show enough context to inspect the rule, including the condition that changes its application.
The useful correction is not necessarily another generated answer. The reviewer can change the draft to ask for the missing account detail, save it and see the intended recipient. If the system can look up that information through an authorized integration, the product can offer that route instead. The choice should reflect the available tools and permissions.
Use the same scenario in the design review and the engineering acceptance discussion. Ask what happens when the source cannot be opened, when the account lookup returns no result and when the reviewer edits the recipient after checking the text. Each variation exposes an interaction decision that the original happy-path screen did not settle.
The finished task is a useful, appropriately scoped reply or a clear request for the information needed to continue. That gives the team a concrete definition of success. A more polished unsupported answer would leave the original design problem unresolved.
Turn the findings into a brief an agency can price
An AI product design brief should describe users, review decisions and the operating context. Include representative questions, permitted source material and examples of incomplete or conflicting information. Explain whether the team is designing a concept, testing a prototype or changing an existing product. These stages need different deliverables and access.
Ask for responsibility boundaries. Product research, interaction design, a design system, frontend implementation and model integration may be delivered by different teams. Specify who will implement the states, connect source passages, enforce approvals and maintain the experience when the surrounding systems change. A screen design alone does not establish those responsibilities.
Request the research plan, difficult-state prototypes, findings and handover material alongside the polished interface. Include the questions that remain unresolved so engineering does not mistake an illustrative mockup for an agreed product rule. Where a proposal depends on access to users or internal documents, make that assumption explicit before comparing prices.
Use published agency cases to understand the kind of contribution a team has made, then ask for a conversation about your workflow. The most useful next step is a scoped plan for making your own users' decisions clearer. Attractive examples can start that conversation, but the brief should give both sides a concrete task to design and evaluate.
Related reading: Compare AI product design agencies
Read the original thinking
Sources & further reading
This is original Point of AI analysis. The sources support the statements attributed to them. Hypothetical examples and suggested questions are editorial guidance.
- HAX: Support efficient correction ↗Microsoft · Checked November 13, 2026
Supports the principle that users should be able to edit and recover from partially incorrect AI output. The review exercise is Point of AI's editorial application.
- HAX: Scope services when in doubt ↗Microsoft · Checked November 13, 2026
Supports clarification and reduced service scope when user intent is uncertain; it does not prescribe this article's answer layout.