On this page
Rerank and diversify candidates¶
Rerank when relevant passages are in the candidate pool but ranked too low. Compare their positions and the added latency on the same questions.
Rerankers run in list order. For scoring nodes, set top_n at or below
candidate_top_k or leave it unset to keep all candidates. Keep extra
candidates if a later step removes near-duplicates. Put mmr last.
Local cross-encoder (cross_encoder)¶
The ContextBase server scores each question–passage pair. The defaults are
BAAI/bge-reranker-v2-m3 and batch_size=8 (1–64). Make the weights
available to the server. Choose batch size by benchmarking typical passages
on your hardware.
profile="library" is the default. There are also ettin,
nemotron and jina_listwise profiles for those model families.
Use the matching model and profile. Depending on the profile, optional
revision, input_mode, input_template, max_length,
truncation and padding override the profile’s settings. Inspect the
effective profile, reviewed revision and licensing details in the editor.
The preflight report includes example rendered inputs. Model precision
is selected by the server.
{"node": "cross_encoder", "params": {"top_n": 10, "batch_size": 4}}
Local ensemble (cross_encoder_ensemble)¶
Use an ensemble to compare a weighted combination on the same candidate
pool. Supply at least two members, each with a local model’s settings
and a nonnegative weight (1 by default). At least one weight must
be positive. A zero-weight member is skipped. Members run sequentially,
with population_zscore normalization and weighted_sum fusion.
A constant vector of scores contributes zero. The default
deduplicate_documents=true keeps one ranked hit per document where
its identity is known. The shared top_n then caps the output.
The default timeout is 120 seconds. The supported
failure_policy="fail_closed" requires the whole enabled ensemble to
finish. A member failure or timeout fails the rerank. Pin compatible
revisions and compare the result against a single-model baseline
before committing the additional compute.
Model judge (llm_judge)¶
Use a language model when ranking needs a criterion you can express in
words. It scores the question against candidates individually. An optional
instruction sets the ranking criterion, and model, api_base
and temperature can override the space’s model settings. Use
judge_concurrency to limit simultaneous requests (the installation’s
limit applies when unset). Concurrency does not reduce the number
of candidates scored. Measure provider latency and cost before
enabling it on a large pool.
For a handbook, an instruction can prefer passages containing the required action and the deadline over passages that merely name a procedure.
External service (remote)¶
Set endpoint to the reranking service’s full POST URL. The service receives
the question and passage texts as {"query": "...", "texts": [...]}
and returns one finite score per input position as
[{"index": 0, "score": 0.8}, ...]. Use a service approved to
receive that content. The optional write-only api_key is sent as a
bearer token. An empty value on update preserves a stored key. The per-call
timeout defaults to 30 seconds.
Diversity and near-duplicates (mmr)¶
MMR compares the candidates’ stored vectors so near-identical passages do not fill the first page. Put it after a scoring reranker if you want to preserve that ranking’s relevance signal while removing redundancy.
relevance="score" (default) uses the previous ranking scores.
relevance="query" uses dense similarity to the original question.
relevance_weight=1 by default prioritizes relevance, while lower
values give diversity more influence. collapse_threshold=0.95
drops near-copies by cosine similarity. Set it to null to turn off
copy collapse. per_document_limit has no limit by default.
The final score is the reciprocal of the new position.
Provide a store that can read back the candidates’ vectors. In an attached store the chunk identity and metadata mapping must support that read. When vectors cannot be read, the node cannot diversify them. If MMR returns too few passages, increase candidate depth before MMR.
{"node": "mmr", "params": {"relevance_weight": 0.8, "per_document_limit": 2}}