The problem
Contract and maintenance teams spend hours finding a single clause across hundreds of PDFs, and a general chat model is not an option: the documents are confidential, the answers must be provably right, and “I think it is three months” is worse than no answer at all.
We set out to build an assistant that only speaks from the documents it was given, shows exactly where an answer comes from, and runs entirely inside the company’s own infrastructure.
What we built
- An ingestion pipeline that parses PDFs, spreadsheets and scanned documents (with OCR), splits them along headings and clauses rather than fixed windows, and keeps page and section references on every chunk.
- Hybrid retrieval: dense multilingual embeddings for meaning, BM25 for exact terms such as clause numbers, followed by a cross-encoder re-ranker that narrows 24 candidates down to the five that matter.
- Generation with an open-weight 8B instruction model, quantised to 4-bit and served on a single GPU in an EU data centre. The prompt enforces numbered citations, and any sentence without a source is dropped before it reaches the user.
- A retrieval trace next to every answer, so a reviewer can see the rewritten query, the chunks that were considered and the latency of each stage.
Results
On an evaluation set of 1,912 questions written by the contract team, the assistant scores 0.94 faithfulness (claims backed by the cited sources) and 0.91 answer relevance, and correctly refuses 96 % of the questions the documents cannot answer. Median response time is 1.5 seconds, and no document ever leaves the company network.
The same pipeline now indexes technical manuals, so a technician on site can ask a question in German and get the relevant paragraph, with page number, instead of a 200-page PDF.