On this page

Extraction and OCR

Choose an extractor

  • Built-in decoder. Use UTF-8 .txt, .md or a text/* media type. The decoder returns plain text. Invalid UTF-8 and unsupported binary files produce no text.

  • ContextBase Extractor. Run this service for PDF, Office files and scans. Support depends on the installed parsers and OCR engines. Set EXTRACTOR_BACKEND=http, EXTRACTOR_SERVICE_URL and EXTRACTOR_API_KEY for the API and workers. Both must be able to reach the service. Store the key on the server.

If a file has a text layer, the extractor can read it directly. The chunker can use a validated structure with section, page or table ranges. For plain text with no useful passages, check the encoding first.

Built-in decoder · HTTP extraction · Structure assessment

Select an OCR profile

Open Settings → Text extraction and OCR. Select an available engine and test a page in the document’s language and layout. To see the default configuration, installed options, runtime revisions or the reason an engine is unavailable, send GET /api/v1/extraction/capabilities/ with Authorization: Bearer <access_token>. The built-in decoder offers no OCR. Engine IDs are tesseract, xiaomi, paddle, paddle_vl, unlimited_ocr_2 and vision_llm.

Set extraction_config when creating a space. To update it, use the settings screen as the owner or send PATCH /api/v1/spaces/{space_id} with the owner’s session token. This example requires the installed Tesseract fast bundle to contain both languages. See session authentication.

{
  "extraction_config": {
    "engine": "tesseract",
    "tesseract": {
      "languages": ["eng", "rus"],
      "model_set": "fast",
      "page_segmentation": "auto"
    }
  }
}

Tesseract uses the order of the language codes. List each code once. fast and best select installed LSTM bundles. For page_segmentation, use auto for automatic layout analysis, single_block for one text block or sparse for scattered text.

On PATCH, omit extraction_config to keep the profile, send an object to replace it, or send null to inherit installation defaults. Omitted nested settings use runtime defaults. With runtime profiles enabled, jobs keep the resolved revision and settings. Keep that bundle installed for jobs that reference it.

For legacy requests, EXTRACTOR_USE_LLM defaults to false. Set it to true to allow the vision tier on difficult PDF pages. An explicit OCR engine takes precedence over escalation and preserves the text-layer fast path. Configure weights, executable paths and concurrency in the Extractor.

Test a new profile with a fresh upload or replace the original in managed storage. A regular reindex splits the text already accepted and may recover missing structure with the recorded extraction settings. Choose a corpus operation.

OCR options · Runtime resolution · Extraction environment

Optional processing after extraction

For title, section and model-written context in retrieval, see Choose how to split documents. Matching passages still contain the extracted source text.

To extract names and aliases from accepted documents, configure an LLM and set dictionary_extraction.enabled. For documents already in the space, open Space dictionary → Build dictionary. The build runs separately from indexing. To start it through REST, set BASE_URL and SPACE_ID and use an owner API key with manage scope in CONTEXTBASE_API_KEY.

curl --fail-with-body -sS -X POST \
  -H "X-API-Key: ${CONTEXTBASE_API_KEY}" \
  "${BASE_URL}/api/v1/spaces/${SPACE_ID}/dictionary/builds"

Pass the returned id to GET /dictionary/builds/{build_id} under the same space URL. With dictionary_extraction.auto_publish=true (the default), new entries without conflicts are published automatically. Set it to false to review source evidence and publish entries manually for the dictionary query transform.

Dictionary extraction options · Build and list dictionary jobs

ContextBase0.1.0

Search the documentation