Website content extractor

Website Content Extractor Apify — clean docs, pricing, and help-center markdown

Website Content Extractor fetches public docs, pricing, product, policy, and help-center URLs and returns cleaned markdown, text, or HTML. Use it as a bounded source step for content QA, indexability checks, and RAG datasets — not as a ranking tool or a full-site crawler. For news, blog, newsroom, or press URLs, use Article Content Extractor. For head Open Graph, robots, and JSON-LD issue audits, use Meta Tag & OpenGraph Scraper.

Try Input: 1 docs URL ≈ $0.010 total (incl. Start $0.001). From $9.00/1k. Cheapest first paid path is one urls item with reports off.

Try the cheapest first paid Website Content Extractor run on Apify Input

Open taroyamada/website-content-extractor on the Apify Store · Docs and help-center RAG first paid path

Site pages versus articles and metadata audits

Use this page when the URLs are broad site pages: documentation, product, pricing, policy, or help-center. Use Article Content Extractor when the URLs are article-shaped and you need headline, byline, publish date, excerpt, and hero image. Use Meta Tag & OpenGraph Scraper when the job is title, canonical, robots, Open Graph, Twitter Card, hreflang, and JSON-LD issue flags in the HTML head.

Website Content Extractor Article Content Extractor Meta Tag & OpenGraph Scraper
Intent Clean docs, product, pricing, policy, and help-center HTML Parse news, blog, newsroom, and press article pages Technical SEO audit of head metadata, social previews, JSON-LD, and robots
Max URLs 200 per run 300 per run 500 per run
Output format markdown (default), text, or html markdown (default) or text — no HTML emit Metadata and issue fields — no published page-body markdown
Sample row web_content article Head tags plus issues
Optional extras includeMetadata, maxChargeUsd includeImages, generateReport, emitExport Issue flags on the metadata row

None of these Actors is a ranking tool, a full-site crawler, or a legal-advice product. Fetch public HTTP(S) URLs you own or are authorized to audit. Respect site policies, rate limits, and robots guidance where applicable.

Compare: for a side-by-side of docs and site-page markdown versus article-shaped news and blog bodies for RAG, see Article Content Extractor vs Website Content Extractor for RAG.

Use cases

The published README also positions the Actor for AI builders, content ops, SEO teams, and documentation teams that supply public pages they own or are allowed to audit. Route news, blog, and article URLs to Article Content Extractor. This Actor is for broad site pages (max 200 URLs).

Docs and pricing extraction

The default path delivers cleaned source rows. Start with a short urls list. outputFormat defaults to markdown. includeMetadata defaults to true. This Actor returns bounded source rows. It does not emit a Site QA or RAG report; keep extraction and the buyer-facing report as separate runs.

{
  "urls": [
    "https://example.com/docs",
    "https://example.com/pricing"
  ],
  "outputFormat": "markdown",
  "includeMetadata": true,
  "delivery": "dataset",
  "maxChargeUsd": 1,
  "dryRun": false
}

Published output fields: rowType, url, title, markdown, text, wordCount, metadata, sourceUrl, and fetchedAt. Sample rowType is web_content. README sample (placeholder page data):

{
  "meta": {
    "actorName": "website-content-extractor",
    "actorTitle": "Website Content Extractor",
    "bundle": "Site QA & Content Intelligence Pack",
    "fetchedAt": "2026-05-06T00:00:00.000Z",
    "totalRows": 1
  },
  "rows": [
    {
      "actorName": "website-content-extractor",
      "rowType": "web_content",
      "url": "https://example.com/docs",
      "title": "Example Docs",
      "markdown": "# Example Docs\nUseful content.",
      "wordCount": 240,
      "sourceUrl": "https://example.com/docs",
      "fetchedAt": "2026-05-06T00:00:00.000Z"
    }
  ],
  "warnings": []
}

Rows also include warnings when a source is partial, and stable IDs when the workflow supports recurring change detection. The sample is a README illustration, not a live coverage guarantee.

Run a docs extraction on Apify

How is Website Content Extractor different from Article Content Extractor and Meta Tag & OpenGraph Scraper?

Website Content Extractor is the bounded HTML cleaner for public docs, product, pricing, policy, and help-center pages (urls, maximum 200). It returns cleaned markdown, text, or HTML plus title, optional metadata, word count, source URL, and fetch timestamp. Sample rowType is web_content. Article Content Extractor is for article-shaped news, blog, newsroom, and press URLs (maximum 300) when you need headline, byline, publish date, article body, excerpt, and hero image. Meta Tag & OpenGraph Scraper is a technical SEO head-metadata audit (title, canonical, robots, Open Graph, Twitter Card, hreflang, JSON-LD, issue flags) and does not emit cleaned page body. Use this Actor for site-page body cleanup. Use Article Content Extractor for article body. Use Meta Tag & OpenGraph Scraper for crawler and social head tags.

What input is required for a docs, pricing, or help-center extraction?

urls is required: public broad website pages to clean (maximum 200). additionalProperties is false. Live input schema fields:

Field Type Default Notes
urls string[] required Public docs, product, pricing, policy, or knowledge-base pages (max 200). Schema prefill is https://docs.apify.com/platform/actors, https://docs.apify.com/platform/storage/dataset, https://docs.apify.com/platform/storage/key-value-store.
outputFormat string markdown Enum: text, markdown, or html.
includeMetadata boolean true Include description, author, language, and published date when available.
concurrency integer 5 Number of pages to fetch in parallel.
timeoutMs integer 15000 HTTP timeout per page in milliseconds.
delivery string dataset Enum: dataset or webhook.
webhookUrl string — Used when delivery is webhook. Skipped on dryRun.
dryRun boolean false Validate without dataset writes or webhook delivery.
maxChargeUsd number 1 Safety cap. Billable rows beyond this cap return no-charge limit_reached summaries.

Do not send unpublished report-only aliases such as generateReport, emitExport, authorizedUseConfirmed, or emitUnchanged on this extractor. Those belong on a linked Site QA or RAG report Actor.

Cheapest first paid input (one public docs URL). Schema prefill is three docs.apify.com URLs; keep the first paid run at one urls item. Leave generateReport and emitExport off — those are not live fields on this extractor.

{
  "urls": [
    "https://docs.apify.com/platform/actors"
  ],
  "outputFormat": "markdown",
  "delivery": "dataset",
  "dryRun": false
}

If that run charges one Actor Start event and writes one useful content row, documented PPE is $0.001 + $0.009 = $0.010. Failed or no-content rows stay out of the default dataset. Then raise the URL list after the markdown row is useful. For RAG ingestion of those bodies, see website content extractor for docs and help-center RAG.

Run this 1-URL input on Apify

Which fields appear on a web_content extraction row?

The published README field list is rowType, url, title, markdown, text, wordCount, metadata, sourceUrl, and fetchedAt. Sample rowType is web_content. includeMetadata defaults to true and adds page metadata such as description, author, language, and published date when present. Rows also include warnings when a source is partial, and stable IDs when the workflow supports recurring change detection. The published sample is a README illustration, not a live coverage guarantee. There is no published Open Graph issue list, article byline, or Site QA report field on this Actor.

Which public page types belong on Website Content Extractor?

The live schema description is: flagship cleaner for broad HTML pages such as docs, pricing, policy, help-center, and product pages. Pass HTTP(S) URLs you own or are authorized to fetch. Maximum 200 URLs per run. For article, news, or blog URLs, route to Article Content Extractor.

How is Website Content Extractor priced on pay-per-event?

Billing is pay-per-event. The live Store card is from $9.00 / 1,000 results. Live events are Actor Start at $0.001 (charged when the Actor starts; number of events depends on Actor memory, one event per GB, minimum one event) and result at $0.009 (single result in the default dataset). If the cheapest first paid run charges one Actor Start event and writes one useful content row, those two events add to $0.010. Platform usage is listed as Included on the live pricing page. The README pricing section matches those event amounts: $0.001 actor start and $0.009 per useful content row. Failed or no-content rows should stay out of the default dataset. The default dataset is the billable surface.

Try Input: 1 docs URL ≈ $0.010 total (incl. Start $0.001). From $9.00/1k.

See Website Content Extractor pricing on Apify

How does Website Content Extractor deliver dataset, webhook, and dry-run output?

delivery defaults to dataset. A non-dry-run always writes canonical dataset rows first; webhook delivery runs only after dataset and pay-per-event output succeed. webhookUrl is used when delivery is webhook and is skipped on dryRun. dryRun validates the setup and writes only local output: dataset writes and webhook delivery are disabled.

{
  "urls": [
    "https://example.com/help"
  ],
  "outputFormat": "markdown",
  "delivery": "webhook",
  "webhookUrl": "https://example.com/webhook",
  "dryRun": false
}

Start a run on Apify Input

Does Website Content Extractor emit a Site QA or RAG report?

No. This Actor returns bounded source rows. The README positions it as the low-cost extraction step, then you pass the same URLs to a linked Site QA or RAG report Actor in a separate run. Do not treat raw markdown as the buyer-facing deliverable. Linked report actors have separate report and export events; do not infer those charges from a raw content row.

What happens to failed, no-content, dry-run, and unchanged Website Content Extractor rows?

Failed or no-content rows should stay out of the default dataset. Dry runs, validation-only runs, missing-key warnings, and unchanged recurring polls should not write payable default-dataset rows. Result events are charged per default-dataset item ($0.009 each; $9.00 per 1,000). Actor Start is a separate $0.001 event charged when the Actor starts (one event per GB, minimum one event).

Do generateReport and emitExport belong on this extractor input?

No. The README says do not add report-only fields to this extractor input. generateReport, emitExport, authorizedUseConfirmed, and emitUnchanged are shown as a separate input for a linked Site QA or RAG report Actor, not for Website Content Extractor. additionalProperties is false on this Actor, so unpublished report aliases are not accepted.

What is the URL cap, and which unpublished fields are rejected?

Maximum 200 URLs per run. Required field is urls. Live schema names are urls, outputFormat, includeMetadata, concurrency, timeoutMs, delivery, webhookUrl, dryRun, and maxChargeUsd. additionalProperties is false. Unpublished report-only names such as generateReport, emitExport, authorizedUseConfirmed, and emitUnchanged are not working inputs on this extractor.

How does maxChargeUsd cap billable Website Content Extractor rows?

maxChargeUsd defaults to 1 as a safety cap for this run. Billable rows beyond that cap are returned as no-charge limit_reached summaries. It does not replace the live PPE events: Actor Start at $0.001 and result at $0.009 per default-dataset item.

Should news, blog, newsroom, or press URLs go to Website Content Extractor?

No. Route article-shaped news, blog, newsroom, and press URLs to Article Content Extractor when you need headline, byline, publish date, article body, excerpt, and hero image (maximum 300 URLs; markdown or text, no HTML emit). This Actor is for broader site pages that are not article-shaped (maximum 200 URLs; markdown, text, or HTML).

Does Website Content Extractor audit Open Graph, robots meta, or JSON-LD issues?

No. This Actor cleans page body into markdown, text, or HTML. It does not produce a metadata-and-issue audit row. Title, canonical, robots, Open Graph, Twitter Card, hreflang, schema.org JSON-LD, and issue flags belong on Meta Tag & OpenGraph Scraper. There is no published full-page markdown field on that metadata Actor, and no published Open Graph issue list on this one.

What compliance guardrails apply to pages fetched by Website Content Extractor?

Fetch public pages supplied by the user. Pass HTTP(S) URLs you own or are authorized to audit. Respect site policies, rate limits, and robots guidance where applicable. Use output for content operations, QA, and RAG workflows. Do not use provider emblems or wording that implies approval by an upstream data provider. This Actor is not a ranking tool, a full-site crawler, or a legal-advice product.

Try Input: 1 docs URL on Apify

Related actors

Related pages