Guides

How to judge an AI demo call

August 08, 2026 · SayTrue team

A polished vendor script proves that a rehearsed question can produce a rehearsed answer. Your demo should test something harder: whether the system behaves safely when the conversation leaves that path. Bring questions your team actually receives, including the awkward one whose answer changes by season and the one that must reach a person.

Use your questions, not the presentation

Ask about a fact that is present in the approved knowledge base, then ask the same thing in the language and phrasing your customers use. Follow it with a question that is deliberately absent. A useful agent should answer from the approved material, then admit when the material does not support a confirmation.

Listen closely to that refusal. Does it explain the limit without blaming the caller? Does it offer the correct next action: handover during working hours or a recorded request when nobody is available? Confidence on an unsupported question is not a successful demo.

Ask to see the source trail

After a supported answer, open the conversation in the supervisor console. Ask which knowledge entry produced the response and where that link is recorded. A transcript plus its source log lets a reviewer retrace the answer. Then inspect the unsupported question: it should appear as a no-source outcome and, when appropriate, as a knowledge gap.

Ask how a disputed answer would be reviewed later. The answer should involve the saved conversation, source record, and approved knowledge as it existed, not somebody’s recollection of the call.

Call at the boundary

Ask what the caller hears at 2 a.m. Working hours should be configured with a timezone, not inferred from greeting text. Outside those hours, the system can answer covered questions and record contact details, but it should not transfer into an empty office or promise an unscheduled callback.

Test the operating controls

Ask what blocks go-live. The readiness checks should be the same checks used by the live call path, followed by a real test call judged by your team. Ask how retention is enforced, including the scheduled purge; which subprocessors handle conversation data; where each one processes it; what safeguards cover transfers outside the EEA; and what AI providers receive.

Finally, ask to see a real handover, a knowledge gap, and a monthly report that excludes test traffic. A serious demo does not need to hide its limits. It should make the supported answer, the escalation, and the absence of a source equally inspectable.

← All guides