Hybrid Search
Hybrid search is a retrieval technique that combines multiple search methods, typically semantic search using vector embeddings and traditional keyword-based search using BM25 or similar algorithms, to produce more relevant and accurate search results. Hybrid search leverages the strengths of both approaches: semantic search understands the meaning and context of queries, while keyword search excels at exact matches and handles rare terms, proper names, and specific phrases that semantic search may miss. By combining results from both methods, hybrid search achieves higher recall and precision than either method alone.
In Financial Services
Real-World Example
A financial institution implements a hybrid search system for its regulatory document database containing 50,000 documents. When a compliance officer searches for 'IFRS 9 expected credit loss staging criteria,' the hybrid search system runs both a semantic search using Voyage Finance-2 embeddings and a BM25 keyword search. The semantic search finds documents about credit loss classification even if they use different terminology, while the keyword search ensures exact matches for 'IFRS 9' that might be missed by semantic search alone. The system combines the results using a weighted scoring algorithm, ranking documents that match both approaches highest. The hybrid search achieves 95% recall compared to 85% for semantic search alone and 70% for keyword search alone, significantly improving the compliance team's ability to find relevant regulatory guidance.
Why It Matters for Finance
Hybrid search is essential for building robust document retrieval systems in financial services. Financial documents contain a mix of conceptual content and specific identifiers that require different retrieval approaches. Pure semantic search may miss documents containing specific regulatory references or numerical identifiers, while pure keyword search fails to find conceptually related content. Hybrid search provides the best of both worlds, improving retrieval accuracy for the diverse types of queries that financial professionals need to perform.
Related Terms
Explore in Finatune
Frequently Asked Questions
What is hybrid search in financial AI?
Hybrid search is a retrieval technique that combines semantic search using vector embeddings with keyword search using BM25 to produce more accurate results. It leverages the strengths of both approaches, finding conceptually related documents while also ensuring exact matches for specific terms, regulatory references, and numerical identifiers.
Why is hybrid search better than pure vector search for finance?
Hybrid search is better for finance because financial documents contain specialized terminology, regulatory references, and numerical identifiers that require exact matching. Pure vector search may miss documents with specific regulatory references or standard numbers, while hybrid search ensures both conceptual understanding and precise term matching.
How does hybrid search improve financial document retrieval?
Hybrid search improves financial document retrieval by combining semantic understanding with exact keyword matching. It achieves higher recall and precision than either method alone, finding both conceptually relevant documents and documents containing specific terms, regulations, and identifiers that semantic search might miss.