Build a Prelaunch Question Set for Your Website Agent
Choose questions with clear answers
A useful prelaunch question set starts with real visitor needs, not every topic on your site. Look for questions people ask before they book, buy, sign up, or decide whether your service fits. Then confirm that your existing pages answer them clearly.
Examples might include:
- What areas do you serve?
- What does the service include?
- How can I book an appointment?
- What is your return policy?
- When are you open?
These are starting points, not a universal checklist. Choose questions that match what your visitors need to know and that your website content can support. If a page only hints at an answer, or the answer depends on personal circumstances, mark that question for review instead of assuming the agent can fill in the gaps.
A practical AI agent prelaunch checklist begins with a simple test: can you point to a specific sentence, section, or page that answers each question? If not, either improve the source content first or leave the question out of the initial set.
Group questions by topic and source
Organize your website agent questions so you can review related answers together. A small table or spreadsheet is enough:
| Topic | Visitor question | Supporting source page | Notes |
|---|---|---|---|
| Services | What does the package include? | Services page | Check whether every package is covered |
| Availability | What are your opening hours? | Contact page | Confirm holiday hours are addressed |
| Policies | Can I cancel a booking? | Cancellation policy | Note any conditions |
The source page matters because an answer is only as useful as the information behind it. Record the page that supports the answer, and note any conditions a visitor should see. If two pages disagree, resolve the conflict before you evaluate the agent. If an answer is missing, update the relevant page or exclude that question for now.
This inventory also makes review more concrete. Instead of asking whether the agent “knows your business,” you can check whether it answered a specific question using the right website agent source content.
Write prompts visitors might actually use
Visitors rarely phrase the same question in exactly one way. For each question, write a few natural variations, plus a follow-up when it makes sense.
For “What are your opening hours?” you might try:
- “When are you open?”
- “Can I visit on Saturday?”
- “Are you open this weekend?”
For a service question, you could test:
- “What’s included in the standard package?”
- “Does that include setup?”
- “Is setup available with the smaller plan?”
Keep the wording realistic. Include short, informal prompts as well as the more complete questions you might put in a help article. Follow-ups help reveal whether a response stays on topic when a visitor asks for a detail.
You do not need a long list for every page. Start with the questions that matter most to visitors and the variations likely to change how they ask. A focused set is easier to assess and revise before you add the script-tag website widget to your site.
Review responses and supporting source material
Save sample prompts as tests in your agent and inspect the responses. To check AI agent answers against your own material, review them through your configured agent with its own sources. See the Simple Agent plans.
For each response, ask:
- Does it answer the visitor’s question directly?
- Is the answer consistent with the source page you recorded?
- Are any conditions, exclusions, or important details missing?
- When retrieved source material is available, do the chunk-level citations support the claims beside them?
- If the source does not answer the question, does the response make that clear?
A citation is useful only when the cited passage actually supports the nearby claim. Open the source and compare it with the answer. A citation to a relevant page is not enough if the retrieved passage says something different or omits a condition.
Try both the original prompt and its alternate wording. Then ask the follow-up. Note where the answer changes, becomes vague, or introduces details that the source does not support. Those observations give you concrete edits to make in your content or question set. For setup details, see the Simple Agent documentation.
Decide what to handle, revise, or leave out
After reviewing the responses, give each question one of three decisions:
- Handle: The source is clear, the response answers the question, and available citations support its claims.
- Revise: The answer exists, but the source needs clearer wording, the question needs a more specific scope, or the response missed an important condition.
- Leave out: The source does not answer the question, the answer changes too often to publish reliably, or the topic needs a person’s judgment.
For example, if your policy page states the cancellation window but not how customers request a cancellation, the agent may be able to answer “When can I cancel?” but not “How do I cancel?” Treat those as separate questions and evaluate each against its source.
Keep a brief record of the decision and the reason. Then update the source or prompt where needed and review the affected questions again. Avoid treating an unanswered question as a writing problem automatically: sometimes the right decision is to route that topic to your team or keep it outside the agent’s scope.
Once your set is focused and the answers match the source material, you have a practical basis for configuring the agent and embedding its script-tag widget. Review pricing as you plan your setup, then try your question set in Simple Agent before adding the widget to your site.
Ready to build your AI agent?
Guided setup in 90 seconds. One flat price, published monthly volume.
Create my agent