Reranker The second pass that puts the best passage first.
Short answer. A reranker is a model that reads a question and a candidate passage together, scores how relevant the passage is, and reorders the results of a first-stage search so the best come first. Retrieval with embeddings is fast but compares the question and each passage separately; a reranker, usually a cross-encoder, reads them as a pair, which is more accurate but too slow to run over a whole archive. So systems retrieve about 100 candidates cheaply and rerank only those. In a 2019 study, BERT re-ranking beat the previous best MRR@10 on MS MARCO by 27%.
01
How it works
The sentence-transformers documentation describes two model types. A bi-encoder turns the query and each document into separate embeddings, so documents can be embedded once and compared quickly. A cross-encoder passes the query and a document into the network together and outputs a single score for how relevant the document is. Scoring thousands or millions of query and document pairs would be slow, so the retriever creates a short list, for example 100 candidates, and the cross-encoder re-ranks it.
Rerankers return a relevance score, not an embedding. BAAI’s bge-reranker-v2-m3 is a multilingual reranker of about 0.6 billion parameters under the Apache 2.0 licence, built on the BGE-M3 base model.
02
A typical pipeline
The counts are examples. At 100 candidates of 300 tokens each, a reranker reads about 30,000 tokens per question, which is why the candidate list is kept short.
Stage
What it does
Candidates
Cost
1. Search
Keyword and embedding search find possible passages
Whole archive in, about 100 out
An index lookup
2. Rerank
A cross-encoder scores each passage against the question
About 100 in, 5 to 10 out
One model pass per candidate
3. Answer
A language model writes the answer and cites the passages
5 to 10 passages in, one answer out
One generation
03
When it is worth adding
The right passage is in the top 20 results but often not in the top 3. Measure this on real questions before adding anything.
Documents are long and similar, such as contracts and policies, so first-stage scores are close together.
Questions mix languages, and a multilingual reranker can sort passages the first stage found.
The model’s context is small, so only a handful of passages can be passed on.
It is not worth it for a small collection where the first-stage search already puts the answer first.
04
Why it matters for an office
The passages shown beside an answer are only as good as the search that picked them. A reranker raises the chance that the passage a person checks is the right one, and lowers the chance that the model writes from a near miss. The price is a second model to host and some added delay for each question. Because rerankers are small, they can run on the same private machine as the rest of the system, so no document needs to leave the building.
05
Questions people ask
Is a reranker the same as an embedding model?
No. An embedding model turns a text into a vector, once. A reranker reads a question and a passage together and returns a relevance score, once per pair.
Do I need a reranker?
Only if measuring shows the right passage is often found but not ranked first. A small collection with a good embedding model may not need one.
Does reranking slow search down?
Yes, by the cost of scoring each candidate. Keep the candidate list short, such as 100 or fewer, and the delay stays small compared with generating the answer.
What is a cross-encoder?
A model that takes two texts as one input, here the question and a passage, and scores them jointly. It is more accurate than comparing separate embeddings, but cannot be precomputed.
06
What is not real yet
No sealed ANVE unit exists; every image of the appliance on this site is a labelled render. The software runs today on a development machine. No customers yet, no certifications, no delivery date, and the final price is not announced.
Reservations are ₹999 in India or US$12 elsewhere, refundable in full until you confirm an order in writing (terms). Offices that cannot use cloud AI can apply to be one of ten design partners.
