A Maintenance Checklist for Website Agent Source Content
A Maintenance Checklist for Website Agent Source Content
A website agent can only answer from the information available to it. When product details, policies, or instructions change, outdated source pages can lead to answers that no longer match your site. A regular website knowledge base review helps you keep those sources current and makes agent answer accuracy easier to check.
Use this checklist whenever important content changes, and set a recurring review for pages that tend to change over time.
1. Assign an owner and review interval to each source page
Start with a list of the pages your configured agent relies on. For each one, record:
- The page title and URL
- The team or person responsible for its content
- The date it was last reviewed
- When it should be reviewed again
- Whether the page is active, being revised, or no longer authoritative
Clear source page ownership matters because a page can remain online long after the person who wrote it has moved on or its information has changed. Assign the owner to someone who can confirm the content is still correct, not simply someone who can access the CMS.
Choose a review interval that fits the content. A pricing or availability page may need attention whenever those details change. A stable explanation of how a product works may be reviewed less often. Treat these intervals as reminders, not proof that a page is accurate: a material change should trigger a review right away.
If several pages cover the same topic, identify which one is authoritative. Conflicting pages make it harder to know which source should support an answer. Where possible, update the authoritative page and revise or retire outdated copies through your normal content process.
2. Record what changed when product details, policies, or instructions are updated
When a source changes, make a short record of what changed and why. Keep it with your content change log or another place your team already uses. Include:
- The page or section changed
- The previous and current information
- The date of the change
- The person or team that confirmed it
- Any related pages that may also need revision
This record supports AI agent content maintenance because it gives reviewers a concrete starting point. Instead of asking “Is this page still right?”, they can check whether the current wording reflects the approved product detail, policy, or instruction.
Pay close attention to changes that affect visitor decisions: eligibility, prices, included services, process steps, support boundaries, and exceptions. Check that the same information is consistent across relevant pages. If a policy has an exception, make sure the exception is stated clearly where a visitor—or an agent using that page—would need it.
Do not rely on the agent to resolve contradictions between old and new content. Update or remove stale information through your site’s usual process, then review the source material available to the configured agent.
3. Retest representative visitor questions against the revised source pages
After updating a source page, check whether the revised information answers the questions visitors are likely to ask. Keep a small set of representative questions for each important topic. Include questions about the main use case, a relevant limitation, and any exception visitors commonly need to understand.
For example, if a page explains how a service works, test a direct question about the process and another about a stated restriction. Use your real product details and visitor language; the example here is only a pattern, not a claim about a particular service.
Compare each answer with the revised source:
- Does it reflect the current information?
- Does it preserve important conditions or exceptions?
- Does it avoid adding details the source does not establish?
- Is any part of the answer unclear or unsupported?
A useful retest is specific enough to reveal a mismatch. “Tell me about the product” may produce a broad answer that misses a changed term. A question that names the affected feature or policy makes it easier to see whether the revision is reflected.
Keep the question set with the page owner or review record. When the page changes again, rerun the same questions and add new ones if the update introduces a new visitor concern. This gives your team a consistent way to check agent answer accuracy over time.
4. Check whether cited source material supports each answer
When retrieved source material is available, inspect the citations alongside the answer. Check whether each citation supports the specific statement it accompanies, rather than merely mentioning the same general topic.
Ask:
- Does the cited passage establish the answer?
- Does it include the relevant condition or exception?
- Is the answer more definite than the cited source?
- Is another source more current or authoritative?
Chunk-level source citations can help reviewers connect an answer to the passage used. They do not replace checking the passage itself. A citation is useful only when the cited material supports the claim in context.
If source material is missing, ambiguous, or drawn from conflicting pages, correct the source content or its ownership and review the answer again with the configured agent. Do not treat a polished response as source material that the underlying information is correct.
For answers that combine several points, check the support for each point. One relevant citation does not automatically substantiate every sentence. If a claim cannot be supported by the available source material, revise the source or the wording visitors see on the site.
5. Review answers in your configured agent before embedding the widget
Before you publish, save these questions as tests on the agent configured with your own sources and compare each answer with the cited page.
For a meaningful review of your content, use your configured agent and its source material. Test the representative questions you prepared, inspect available citations, and make any needed source updates before deciding to embed the widget. After embedding, continue reviewing the configured agent as your pages change.
If you use WordPress, follow the instructions in the Simple Agent WordPress guide for that platform. For other sites, use the installation instructions provided for the script-tag widget. Do not assume that a platform-specific setup step applies everywhere; check the documentation for the method you are using.
Once your source owners, review intervals, change records, and retest questions are in place, pick a Simple Agent plan and check your configured agent against your own content before embedding it.
Ready to build your AI agent?
Guided setup in 90 seconds. One flat price, published monthly volume.
Create my agent