Web sources

Have a knowledge base collect pages from a website and keep them up to date.

For: Owners and editors of a knowledge base, and administrators · Last updated

The Web sources card on a knowledge base lists sites it collects page content from. Each page it collects becomes a document named after the page's address. Everyone who can see the knowledge base can see its sources. Owners, editors and administrators can add, change, sync and remove them.

Add a source#

  1. Select Add a source.
  2. Enter the address in the URL field, for example https://docs.example.com/guides.
  3. Choose a Scope:
    • Single page
    • Pages under this path
    • Entire domain, which also shows a Depth field (it starts at 2)
  4. Choose how often it should update under Update: On demand, Daily, Weekly or Monthly.
  5. Select Add source. A message confirms the address was added and that its first sync is running now.

Addresses that cannot be collected, such as ones that lead to private network addresses, are refused with a message that says why.

The sources table#

Column What it shows
Source The address, its scope and, for a domain, its depth
Status Active, Syncing now…, Couldn't be crawled, Sync failed or Partially synced, with the reason when there is one
Last synced When it last synced, or that it has not synced
Update How often it updates. Change it from the list.

While a source is syncing, the table refreshes by itself.

Sync and remove#

  • Select Sync now to collect the pages again straight away. It shows Syncing… while it runs. A source that is already syncing cannot be started again.
  • Sources with a Daily, Weekly or Monthly update sync again on their own once a day, a week or a month has passed since their last sync. A source set to On demand syncs only when you select Sync now.
  • Select Delete and then Remove source to stop future syncs. The dialog notes that content already collected stays until you delete those documents.

When there are no sources the card says no web sources have been added.