How Docusense finds the right material.

Why retrieval decides the quality

A trained model holds a statistical impression of a large amount of public text. It does not hold your quality manual, your certificates, or the answer your colleague gave a customer in March. Asked a question about any of those, it will still produce a fluent answer, and that answer will be an invention.

The only reliable way around this is to find the relevant passage in your own material and give it to the model as part of the request, then require the output to be based on what was supplied. The quality of the final answer is therefore set mostly by the quality of the search, not by the choice of model. If the right paragraph is not retrieved, no model will recover it.

That is why the retrieval strategy below is more elaborate than a single vector lookup.

Four models, four jobs

An instance is configured with four kinds of model. Each one is selected by its purpose rather than by name, so an administrator can change any of them independently and the application asks only for the active model of a given kind.

• Document processing reads uploaded files where the source needs interpretation rather than text extraction.
• Embedding converts text into vectors for search by meaning.
• Reranking scores a question against a candidate passage by reading both together.
• Completion extracts items, writes summaries, drafts suggested answers, and compares items against policy.

Each model has its own sampling settings and prompt templates. In a self-hosted deployment all four run inside your environment. Thetransformer models page explains what distinguishes these kinds of model from each other.

From uploaded file to index

Every uploaded document runs through a pipeline of recorded steps. The step list for each pipeline type is fixed in the application rather than configured per document, and the steps that ran are stored with the document, so an administrator can see exactly where a file is and what happened at each stage.

1. Parse

   A configured parser converts the uploaded file into markdown. Several parsers are supported so an administrator can choose one that suits the material, and pages that need optical character recognition or table extraction are handled by a model configured for that purpose.

2. Clean

   Repeated headers, page furniture, and broken markup are removed. Everything downstream reads this cleaned text, so the same content produces the same result on a re-run.

3. Summarize

   A library document gets a compact summary of the whole file. A questionnaire instead gets its title, language, document type, and summary extracted as metadata.

4. Chunk

   The markdown is split along its own structure rather than at a fixed character count. Sections break at headings, lists and tables are kept whole where they fit, and every chunk carries the breadcrumb of the headings it sits under along with its character offsets in the cleaned text. Those offsets are what let a citation point back at a specific place.

5. Generate retrieval artifacts

   For each chunk, a model writes a short summary and a set of questions that the chunk would answer. These are stored next to the chunk and indexed alongside it.

6. Index

   The chunk text and every generated artifact are embedded in a single batch, so they share one model, and written to the vector store. Indexing removes the document’s existing points before writing new ones, so a re-run cannot leave stale vectors behind, and the deletion happens only after embedding succeeds.

Four ways into the same section

A questionnaire asks “Do you hold a valid ISO 27001 certificate, and who is the responsible officer?”. The paragraph that answers it may never use the word “certificate” in that form and may sit under a heading about governance. Searching the raw text alone finds it only when the wording happens to line up.

Docusense therefore stores four representations of the same material and searches each of them separately, with its own candidate budget.

Chunk text

The section exactly as written. This is what matches when someone uses the same wording as the document.

Chunk summary

A short description of what the section covers. This matches topic-level questions that share no vocabulary with the original text.

Synthetic questions

Questions the section would answer, generated at indexing time. A question in a questionnaire is far more likely to resemble one of these than to resemble a paragraph of policy prose.

Document summary

A summary of the whole file, which catches questions that are about a document in general rather than any one section of it.

The four searches run at the same time and their results are collapsed onto the chunks they point at. A chunk found by more than one route is not counted repeatedly. It takes its best score across the routes, so it ranks by its strongest evidence rather than by how many ways it happened to surface.

Matches at document level are kept separately. They raise the floor for that document's chunks, and if nothing matched at chunk level at all, they select whole documents to fall back on. This is what stops a question that only matches a document summary from returning nothing.

Meaning and wording together

Search by meaning is good at recognising that “How do you vet new suppliers?” and “supplier qualification procedure” are about the same thing. It is unreliable at exact strings. Part numbers, clause references, standard names, and product codes are precisely the things a qualification questionnaire is full of, and an embedding tends to blur them together.

Every indexed item therefore carries both a dense vector and a lexical representation. A search runs both, and the two result lists are combined by reciprocal rank fusion, which merges positions rather than scores. That matters, because a lexical score and a vector distance are measured on different scales and are not comparable. Combining ranks avoids having to calibrate one against the other.

Why the embedding step is asymmetric

A short question and the paragraph that answers it do not look alike. Current embedding models handle this by expecting a different instruction prefix depending on which side of the pair the text belongs to, and using the wrong one measurably reduces what the search finds. Docusense embeds a search input as a query and stored content as a document, applying the template the configured model expects for each.

Each retrieval task also has its own template. Matching a questionnaire question against earlier questionnaire items, matching against maintained reusable answers, and searching the library are separate tasks with separate collections. Every caller has to state which task it is embedding for, because mixing tasks inside one collection makes the distances between vectors meaningless.

Re-scoring the shortlist

Search by vector compares two representations that were produced independently, without either one having seen the other. It is fast enough to run over a whole collection, and it is approximate.

A reranking model works differently. It reads the question and one candidate passage together in a single pass, so its attention can compare them directly and notice that a passage which looks topically close does not actually address what was asked. It is far more accurate and far too slow to run over everything.

Docusense uses both. Hybrid search produces a generous candidate list, the reranking model re-scores that shortlist, and only the top results are used to prepare an answer. How many candidates are retrieved, how many are reranked, and how many reach the answer are all configurable per instance, and reranking can be switched off entirely.

Writing the answer

Only after all of that does a completion model see anything. It receives the question and the selected passages, and it is instructed to answer from those passages alone, to say plainly when the evidence is not sufficient, and to cite only the passages that actually support what it wrote.

The response is returned as structured data validated against a schema rather than as free text, so the answer, its title, and its citations arrive as separate fields. Each citation identifies a stored chunk, which resolves back to a document, a section, and an excerpt that a reviewer can open and read. A citation is a link into your own library, not a footnote the model wrote.

A search is a durable record rather than a request that returns a payload. Creating one retrieves the evidence, queues the model run, and returns immediately. The answer and citations are written to that record when the run completes, which means a long search survives a reload.

Suggestions inside a questionnaire

Answer suggestions use the same machinery against different sources. For a question with no answer yet, Docusense retrieves maintained reusable answers and similar questions your team has already answered in earlier questionnaires, then asks a model to draft a suggestion from those sources.

The instructions for that step are deliberately restrictive, because this is where a helpful model does the most damage. A source may be used only when it asks for the same fact, field, metric, status, or confirmation as the target question. Sharing a category, a business area, or similar wording is not enough. For named roles, contacts, certificates, sites, and dates the match has to be exact. Filling a blank field with the closest available source is prohibited, and so are placeholder answers such as “not available”.

Where no source genuinely supports an answer, the question is left without a suggestion. An empty field is a correct outcome, and it is far more useful to a reviewer than a confident answer to a slightly different question. Each suggestion that is produced carries a confidence level and the identifiers of the sources it used, so the reviewer can open them.

Comparing items against policy

Policy checking is a separate step with a separate instruction set. The items under review are compared against the text of the organization and customer policies selected for that questionnaire, and the model returns only the items it judges to be in conflict.

A conflict means something specific here. The item contradicts the policy, asks for something the policy forbids or excludes, assumes something the policy says is not provided, or omits something the policy explicitly requires. Each finding names the policy involved and explains the conflict. It is information for a reviewer, not a decision, and the reviewer records what to do about it.

What happens when something fails

Model services and vector stores are network dependencies and they will occasionally be unavailable. The behaviour in those cases is deliberate rather than incidental.

• Retrieval degrades instead of failing. An unavailable vector store or embedding model produces an empty result, and the reviewer sees a question with no suggestion rather than an error page.
• If reranking is unavailable, the original ranking is kept and the results carry no relevance score, which is visible rather than silently substituted.
• Model calls are queued with retries and increasing delays between attempts. A retry re-uses byte-identical input, so a repeated attempt is not a different request.
• Partial extraction keeps what was extracted and records a warning, so a reviewer can decide whether to work with it or reprocess the file.
• Processing history keeps the status, model, duration, and errors for each step, and failed work can be queued again.
• Indexing removes a document's old vectors only after new ones have been produced, so a failure leaves the previous index in place rather than emptying the document from search.