In-Context Learning
In-context learning is a capability of large language models where they learn to perform new tasks from examples provided in the input prompt, without requiring parameter updates or fine-tuning. The model uses the patterns and relationships demonstrated in the provided examples to infer the desired behavior for new inputs, adapting its responses based on the context established by the examples. This capability is one of the most powerful features of modern LLMs, enabling them to perform a wide range of tasks with minimal task-specific training. In-context learning works through the model's ability to recognize and extend patterns in the input sequence. When a user provides several examples of a task — such as input-output pairs for classification, translation, or analysis — the model identifies the underlying pattern and applies it to new inputs. The number of examples provided is called the shot count, with zero-shot learning using no examples, one-shot learning using a single example, and few-shot learning using multiple examples. The effectiveness of in-context learning depends on several factors: the quality and representativeness of the examples, the clarity of the task description, the similarity between examples and the target input, and the model's pre-training data. In-context learning is particularly effective for tasks that the model has some general knowledge about from pre-training, as the examples serve to focus and direct the model's existing capabilities. The mechanism behind in-context learning is an active area of research. Some theories suggest that models perform implicit gradient descent during the forward pass, effectively learning from the examples without updating weights. Others suggest that in-context learning emerges from the model's ability to retrieve and apply relevant patterns from its pre-training data, with the examples serving as retrieval cues. Regardless of the mechanism, in-context learning has become a fundamental capability for LLM applications, enabling users to adapt models to new tasks quickly without the computational cost of fine-tuning. In-context learning is particularly valuable for tasks that require domain-specific knowledge or specialized output formats, where providing examples in the prompt can significantly improve the model's performance.
In Financial Services
Real-World Example
A large investment bank uses in-context learning for its AI-powered earnings analysis system. The system is provided with 5 examples of earnings report analysis, each demonstrating the desired output format: a summary table with key metrics, a paragraph analyzing revenue trends, a paragraph on profitability, a paragraph on segment performance, and a section on risks and outlook. The examples cover different types of companies and earnings scenarios. When the system receives a new earnings report, it applies the pattern demonstrated in the examples to generate a comprehensive analysis. The bank reports that the in-context learning approach achieves 90% of the accuracy of a fine-tuned model while requiring no training infrastructure or ongoing maintenance. The bank maintains a library of 50 example sets covering different analysis types, document formats, and regulatory requirements. The example sets are updated periodically to reflect changes in financial reporting standards and regulatory requirements. The bank reports that the in-context learning system processes 1,000 earnings reports per quarter, saving 500 analyst hours compared to manual analysis.
Why It Matters for Finance
In-context learning is a transformative capability for financial AI because it enables rapid adaptation to new tasks without the cost and complexity of model fine-tuning. Financial institutions have hundreds of specialized tasks — from analyzing different document types to generating different regulatory reports — and fine-tuning a model for each task would be impractical. In-context learning allows a single model to perform many different tasks, with the task specification provided in the prompt. The quality of in-context learning depends on the examples provided, making example design an important skill for financial AI practitioners. Institutions that invest in creating high-quality example sets can achieve performance that approaches fine-tuned models without the associated infrastructure and maintenance costs. As financial institutions scale their AI adoption, in-context learning will become an increasingly important technique for adapting AI to diverse financial use cases.
Related Terms
Explore in Finatune
Frequently Asked Questions
What is in-context learning in finance AI?
In-context learning is a capability of LLMs where they learn to perform new tasks from examples provided in the input prompt, without requiring parameter updates or fine-tuning. The model uses the patterns in the examples to infer the desired behavior for new inputs.
How does in-context learning help with financial document analysis?
In-context learning helps by allowing the model to adapt to different document types and analysis formats based on examples. A model can be given examples of how to analyze 10-K filings, earnings transcripts, and analyst reports, then apply the appropriate format to each document.
When should I use in-context learning vs fine-tuning for finance?
Use in-context learning when you need to quickly adapt to new tasks, when the number of tasks is large and fine-tuning each would be impractical, or when you don't have the infrastructure for fine-tuning. Use fine-tuning when you need maximum performance on a specific, stable task.