Extract

POST /v1/extract loads a page in a real browser and returns its main content as Markdown, HTML or text, with metadata.

POST https://rendersbolt.com/v1/extract
GET  https://rendersbolt.com/v1/extract?url=...&access_key=...

Returns JSON. The page is rendered first, so text added by JavaScript is included. A readability pass keeps the article and drops navigation, sidebars, footers and ads.

Example#

curl https://rendersbolt.com/v1/extract \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://en.wikipedia.org/wiki/Headless_browser","include_images":false}'

const res = await fetch('https://rendersbolt.com/v1/extract', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.API_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({ url: 'https://en.wikipedia.org/wiki/Headless_browser', include_images: false }),
});
const { title, content, metadata } = await res.json();

r = requests.post(
    "https://rendersbolt.com/v1/extract",
    headers={"Authorization": f"Bearer {os.environ['API_KEY']}"},
    json={"url": "https://en.wikipedia.org/wiki/Headless_browser", "include_images": False},
)
page = r.json()

Parameters#

Extract-specific options below. Every common option works too.

Parameter	Default	Description
`url` or `html`		The page to read
`output`	`markdown`	`markdown`, `html` (the cleaned HTML) or `text`
`only_main_content`	`true`	Keep the main article only. `false` returns the whole page body.
`include_links`	`false`	Add a `links` array with every absolute link on the page (up to 1,000)
`include_images`	`true`	Keep images in the content

Response#

{
  "url": "https://en.wikipedia.org/wiki/Headless_browser",
  "title": "Headless browser",
  "output": "markdown",
  "content": "A **headless browser** is a web browser without a graphical user interface...",
  "metadata": {
    "description": "...",
    "language": "en",
    "canonical_url": "https://en.wikipedia.org/wiki/Headless_browser",
    "image": "...",
    "site_name": "...",
    "published_time": "...",
    "byline": "...",
    "excerpt": "...",
    "word_count": 812
  },
  "links": null
}

Metadata fields that a page doesn't provide are left out. If a page has no clear article, the body is returned without nav, header, footer, aside and forms.

Related guide: Markdown for LLMs and RAG.

Stuck? Write to [email protected] with the request you sent and the error code you got back.