Keep Your Chatbot Fresh: Re-Crawl & Refresh Content in Alee

A chatbot is only as accurate as the content behind it. Prices change, policies get rewritten, you launch new pages, and old posts retire, and if your bot's knowledge brain doesn't move with you, it starts quoting yesterday's truth. This guide shows you exactly when to re-crawl, how to add new pages and remove stale sources, and a lightweight maintenance routine that keeps your Alee bot trustworthy without becoming a chore.

Why freshness matters more than you think

Your Alee bot answers from a snapshot. When you first add a website URL, sitemap, PDF, YouTube video or pasted text, Alee crawls or reads that content once, splits it into chunks, turns each chunk into a vector embedding and stores it in your pgvector knowledge brain. That snapshot does not auto-update when you edit the underlying page. So if you change a price on your site but never re-crawl, the bot keeps confidently answering with the old number, complete with a source link, which is worse than saying nothing.

Because every answer is grounded only in your stored content and self-checked before it is sent, an outdated source does not produce an obvious error. It produces a confident, well-formatted, wrong answer. That makes keeping content fresh your single highest-leverage maintenance task, and the good news is that refreshing is fast and you only need to touch the sources that actually changed.

When to re-crawl or refresh

You do not need to re-crawl on a fixed timer. Refresh when something that the bot answers from has changed. Concrete triggers:

• You edited a page the bot already learned. Updated pricing, hours, refund policy, course outline, or contact details on a page you previously crawled.
• You published new pages. A new pricing page, a new FAQ, fresh blog posts, a new service or product.
• You retired or merged pages. Deleted a product, killed an old offer, or moved content to a new URL.
• A PDF or document got a new version. A new membership PDF, an updated terms doc, a revised brochure.
• A YouTube video was replaced or its info changed. You re-recorded a walkthrough or updated the description the transcript drew from.
• Your analytics show drift. The Top Questions list and the question-triage inbox keep surfacing things the bot fumbles or gets wrong, a strong signal a source is missing or stale.
• Seasonally or for campaigns. Before a sale, a new batch, an admission cycle, or a festival offer (Diwali, end-of-financial-year in India), refresh so the bot speaks to what is live right now.

If a page changes and the bot answers questions about it, that is your cue. If nothing changed, you can leave the brain alone, it does not decay on its own.

How to re-crawl an existing source

When a page you already trained on has changed, you want Alee to fetch the new version and re-embed it. The exact label varies, but the action is straightforward.

1. Sign in to Alee and open the bot you want to update. If you are just getting started, you can start free and build your first bot in a minute.
2. Open the bot's Sources area, where all your knowledge sources are listed (website URLs, sitemaps, PDFs, YouTube videos, pasted text).
3. Find the specific source that changed, for example the pricing page URL.
4. Trigger a re-crawl or refresh on that source (look for a refresh, re-crawl, re-sync or update action on the source row).
5. Let Alee re-fetch the page and rebuild its chunks and embeddings. This usually takes seconds to a minute depending on page size.
6. Wait for the source to show as ready or indexed again.
7. Test it. Open the bot preview and ask the exact question whose answer changed. Confirm the new price or policy comes back, with the source attached.

A re-crawl replaces that source's old chunks with fresh ones, so the bot stops quoting the previous version. You only need to refresh the sources that actually changed, not your whole brain.

A note on the answer cache

Alee serves repeat or similar questions from a cache so they come back instantly. After you refresh a source, a previously cached answer may briefly reflect the old content until it cycles. If you see a stale reply right after updating, ask the question again or rephrase it slightly to confirm the fresh, re-grounded answer is in place before you call it done.

Adding new pages and sources

Growing the brain is just adding sources, and they all stack into the same knowledge brain alongside what is already there.

• A single new page: open the Sources area, add a new source, choose the website URL option, and paste the new page's address. Alee crawls that page and indexes it.
• Many new pages at once: use the sitemap option to pull in a batch of pages in one go, ideal after you have published a cluster of new posts or product pages. See the sitemap and bulk pages guide for the full walkthrough.
• A new document: add a PDF or document source and upload the file.
• A new video: add a YouTube source so Alee can learn from its transcript.
• Quick facts not written anywhere yet: paste them as a raw text or FAQ source. This is the fastest way to teach the bot something new without editing your website.

You can add sources any time and the brain simply grows. Keep new sources focused, one topic per source where it makes sense, so retrieval stays precise.

Removing stale and conflicting sources

This is the step most people skip, and it is the one that quietly breaks accuracy. If an old source still lives in the brain, the bot can retrieve it, even after you have added a newer one. When two sources disagree (old price vs new price), you get unpredictable answers.

When to remove a source:

• The page or document it came from no longer exists.
• It described an offer, plan, product or policy you have discontinued.
• It has been superseded by a newer version you have already added.
• It contains content you never want the bot to repeat (an outdated promise, a wrong number, internal notes that slipped in).

To remove one: open the Sources area, find the outdated source, and delete or remove it from the bot. Its chunks leave the knowledge brain, and the bot will no longer answer from it.

A clean rule of thumb: when you replace, also remove. If you upload membership-2026.pdf, delete membership-2025.pdf in the same sitting. If you re-crawl a moved page at its new URL, remove the old URL source. One source of truth per fact keeps the bot decisive.

A simple maintenance routine

You do not need a heavy process. Match the rhythm to how often your content actually changes.

After every meaningful content change (event-driven, do it immediately):

1. Edit the page, document or video as usual.
2. Re-crawl the matching source in Alee (or upload the new file and remove the old one).
3. Ask the bot the one question whose answer changed and confirm it is correct.

Monthly tune-up (15 minutes):

1. Open your analytics and read the Top Questions list. Anything popular the bot handles weakly points to a missing or stale source.
2. Work the question-triage inbox: mark questions important, FAQ or answered, and teach a better answer where the bot fell short. This is how you close gaps without guesswork.
3. Skim your Sources list for anything obviously outdated, and refresh or remove it.
4. Re-crawl any high-traffic pages (pricing, policies, contact) even if you think they are unchanged, cheap insurance.

Quarterly deep clean (30 minutes):

1. Re-crawl your sitemap to catch new and edited pages in bulk.
2. Audit every source: is it still accurate, still relevant, still needed? Delete what is dead.
3. Confirm new products, plans or pages all have a corresponding source.
4. Run 8 to 10 real customer questions through the preview as a spot check.

Worked example: a coaching business after a price hike

A career coach raised her flagship programme's price and rewrote the syllabus on her site. Her maintenance pass:

1. She re-crawls the programme page URL so the new price and syllabus replace the old chunks.
2. She deletes a stale text source that still held the previous fee, removing the conflict.
3. She uploads the new syllabus-2026.pdf and removes last year's PDF.
4. She asks the preview "How much is the career programme and what's covered?" and gets the new price and modules, with sources, while the old fee no longer surfaces anywhere.

Total time: under ten minutes, and the bot now quotes the right number to everyone.

How refreshing fits your plan

Re-crawling, adding and removing sources are part of every plan, including Free (1 bot, 200 messages a month), so freshness is never a paywalled feature. Higher tiers, Pro, Agency and Scale, give you more bots and message capacity, and the Agency plan lets you white-label and maintain separate bots for separate clients, each with its own sources to keep current. See pricing for the breakdown, explore everything you can train on in features, or see how the approach compares in Alee vs SiteGPT.

Frequently asked questions

Does Alee automatically re-crawl my site when I update a page?

No, a source is a snapshot from when you added it, so editing the underlying page does not update the bot on its own. Trigger a re-crawl on that source whenever its content changes so the new version replaces the old chunks in your knowledge brain.

If I re-crawl a page, do I lose my other sources or settings?

No. Re-crawling only rebuilds the chunks for that one source. Your other sources, your customization (bot name, color, persona, starter questions) and your analytics all stay exactly as they are.

Why is my bot still giving an old answer right after I refreshed?

Repeat questions can be served from the answer cache for a short while after an update. Ask again or rephrase slightly to pull a freshly grounded answer, and double-check you also removed any older source that still holds the outdated fact.

Keep your chatbot accurate as your business grows, [start free with Alee](/signup) and build a knowledge brain you can refresh in minutes.