# Add docs and your website to knowledge

> Give your agent the help docs, website pages, release notes and FAQs it answers from, by crawling a URL or sitemap, uploading files or pasting text.

Source: https://talkdemo.com/help/knowledge/add-docs-and-website · Updated 2026-11-20

## Key takeaways

- Knowledge sources are what the agent answers questions from. Add a URL, a sitemap, a file or plain text.
- URL and sitemap sources are crawled up to a page limit (50 by default), from public pages only, and pages your robots.txt disallows are skipped.
- A source moves from Pending to Indexing to Ready. Failed sources show the reason.
- Re-index a source after you change the pages so the agent uses the new content.
- Use **Try a question** to see which passages the agent would read for a question, before a prospect asks it.

Knowledge is everything the agent can answer from: your help center, product pages, pricing page, release notes and FAQs. When a prospect asks a question, the agent searches these sources and answers from what it finds.

**Who can do this:** Admin, Member. Viewers can see the list of sources and use **Try a question**. · **Plans:** every plan.

## Add a source

1. Open **Agents**, choose the agent and open **Knowledge**.
2. Click **Add source** and choose a tab:
   - **Web page or site.** A page or a section of your site, such as your help center. TalkDemo follows links on the same site.
   - **Sitemap.** Your sitemap.xml, to pick up every listed page.
   - **File.** PDF, Word (.docx), text, Markdown or HTML files, such as a product FAQ, up to 20 MB each (10 MB on the Free plan). Drop up to 10 files at once or press **Choose files**; remove any from the list before you press **Add sources**. Each file becomes its own source. They upload one after another with a progress bar ("Uploading 2 of 5") and each is scanned for malware before it is read. If one fails, the others are still added and the failed file stays in the list with the reason, so you can try it again.
   - **Paste text.** Short facts, such as plan limits or answers to common objections.
3. Give it a **Title** your team will recognise. The agent cites it with each answer.
4. For a web page or sitemap, set the **Page limit**. The default is 50.
5. Click **Add source**. The status shows **Pending**, then **Indexing**, then **Ready** with the number of pages.

Each page is split into passages that keep their heading, the page address and the source title, so the agent can say where an answer came from.

## How the crawler behaves

- It reads public pages only. Pages behind a login, private network addresses and other sites are never fetched.
- It follows links on the same site, up to the **Page limit**, and reads only the pages a sitemap lists for a **Sitemap** source.
- It respects your robots.txt. Pages it disallows are skipped, and a start page it disallows fails the source with that reason.
- Scripts, menus and footers are left out. Headings become section titles.

## What to add first

| Source | Why |
|---|---|
| Help center | Answers "how do I" questions |
| Pricing page | Answers plan and price questions accurately |
| Product pages | Answers "does it do X" questions |
| Release notes | Keeps answers current |
| Security page | Answers procurement questions |

## Try a question

Check what the agent would find before a prospect asks. Everyone in the workspace can use it, Viewers included.

1. On the agent's **Knowledge** page, scroll to **Try a question**.
2. Type a question the way a prospect would ask it, in English, Hindi or Hinglish, such as "How much does the Growth plan cost?".
3. Click **Search**.

You see the top passages the agent would read for that question, best first, the same search a live demo runs. Each passage shows:

- its rank and **Score** (from 0 to 1; the agent ranks by it),
- the source title, the heading, and a link to the page when the source is a web page,
- **Why**: the words of your question the passage contains. A Hinglish word shows the English word it matched, such as "keemat → price". Where meaning-based search is switched on, it also shows how close the passage is in meaning.

If nothing matches, it says **Your docs don't answer this yet**. In a demo the agent would say it will check with your team. Add a page, file or text that answers the question, wait until the source shows **Ready**, and try again.

Nothing is said, saved or added to analytics, and no AI answer is written: you see the passages only. The agent turns them into a spoken answer in its own words.

## Keep it current

- Click **Re-index** on a source after you change its pages.
- Delete sources that are out of date. Old pages lead to old answers.
- Add a **Text** source for anything sales says that is not written down anywhere.

## If a source fails

The status shows **Failed** with the reason. Common causes: the page needs a login, the host is not public, the file is password-protected, or the file isn't what its name says (for example a renamed file that isn't really a PDF). TalkDemo only fetches public pages, so private docs need to be uploaded as files.

**Privacy:** knowledge sources should describe your product, not people. Do not upload customer lists, contracts or anything with personal data. Search works inside TalkDemo. Where meaning-based search is switched on, the text of your sources is also sent to an embedding provider to make it searchable; no prospect data is included. If AI is turned off for your workspace, nothing is sent to an embedding provider: sources are still indexed and searched by their words inside TalkDemo.

## Frequently asked questions

### Can I see the answer the agent would give?
Not on this page. **Try a question** shows the passages the agent reads, which is what decides whether it can answer. To hear the full answer, ask the question in the test console.

### What happens to knowledge when AI is turned off for our workspace?
Indexing and search keep working. Sources are indexed and searched by the words they contain, inside TalkDemo, and nothing is sent to an embedding provider, for indexing or for questions. Passages that say the same thing in other words rank lower until AI is back on. After AI is turned back on, click **Re-index** on your sources so meaning-based search covers them again. To turn AI off or on for your workspace, contact support.

### Does the agent learn from demos?
No. It answers only from the sources you add. Transcripts are never used to train models.

### Is there a size limit for files?
Yes: 20 MB per file on paid plans and 10 MB on the Free plan. Split a large document into smaller files, or add the pages as a URL source instead. You can add up to 10 files in one go.

### Does search work for Hindi and Hinglish questions?
Yes. Search understands Hindi written in Devanagari, Hinglish and English, ignores filler words such as "kya" and "hai", and links common Hinglish words to English ones, so "Growth plan ki keemat" finds your pricing page. Sources written in Hindi are searched in Hindi.

### How long are sources kept?
Until you delete the source or the agent. See [Data retention](/help/team-and-settings/data-retention).
