Skip to content
AI retrieval

RAG-Powered Legal Assistant

Production RAG system with 94% accuracy on legal queries using hybrid search

Client
Legal Tech Startup
Duration
8 weeks
Team
2 engineers
Delivered
April 2025
SYSTEM PATHQuestionLegal queryHybrid searchVector + BM25RerankCross-encoderGenerateCited answerValidateClaim check

The outcome

  • 94%accuracy (up from 67%)
  • Sub-3 second response times
  • 10,000+documents indexed
  • Deployed to 50+ legal professionals

The story

The problem

A legal tech startup needed to answer complex legal questions from a large corpus of case law, contracts, and regulations. Generic LLMs produced confident but incorrect answers, often citing non-existent cases or misinterpreting precedents. The firm could not adopt the tool until the answer quality was reliable and every response was traceable to source documents.

The approach

I built a Retrieval-Augmented Generation system that grounds every answer in retrieved source material. The system uses a hybrid search approach and a validation layer to prevent hallucinations and surface confidence levels to the user.

Architecture

Documents are ingested through a semantic chunking pipeline that respects section boundaries, headings, and cross-references. Chunks are stored as dense vectors in Pinecone and also indexed for keyword search. At query time, the system runs both vector and BM25 retrieval, then reranks the combined results with a cross-encoder model. The top-k chunks are passed to a generation model with a strict prompt that requires citations and a confidence statement. A separate validation agent cross-references the generated answer against the retrieved context to flag unsupported claims.

Frontend and user experience

The React interface shows the answer, a confidence score, and clickable citations that open the referenced document at the relevant passage. Users can inspect the source, which builds trust and makes the tool useful for research rather than just quick answers. Search history and feedback are stored to improve the retrieval model over time.

Results

The system achieved a measured accuracy of 94% on a held-out set of legal questions, up from 67% with a naive vector-only approach. Response times stayed under three seconds for typical queries. The firm deployed the tool to over fifty legal professionals, and source citations became the most-used feature.

What made it hard

  1. Accuracy Requirements

    Legal domain requires extremely high accuracy, hallucinations are unacceptable

  2. Document Complexity

    Processing complex legal documents with nested structures and cross-references

  3. Latency Optimization

    Keeping response times under 3 seconds despite complex retrieval pipeline

Next projectSmart Lead Qualification Engine
Open for new work

Have a system that needs to scale?

Bring your architecture problem, automation bottleneck, or product idea. We will work out the right engineering path before you spend more time or money.

Free, 30 minutes, no pitch. You leave with a next step either way.