Catastrophic Forgetting
Catastrophic forgetting is a phenomenon in machine learning where a model loses previously learned knowledge when trained on new data. In the context of LLM fine-tuning, catastrophic forgetting occurs when the fine-tuning process overwrites or distorts the knowledge the base model acquired during pre-training, causing the model to perform worse on general tasks while improving on the specific fine-tuning task. This is particularly problematic when fine-tuning on narrow, domain-specific datasets that do not represent the diversity of the pre-training data. The degree of catastrophic forgetting depends on several factors including the size and diversity of the fine-tuning dataset, the number of training steps, the learning rate, and the fine-tuning method used. Full fine-tuning is most susceptible to catastrophic forgetting because it updates all model parameters. Parameter-efficient methods like LoRA and QLoRA are more resistant because they only modify a small subset of parameters, preserving most of the pre-trained knowledge. Mitigation strategies for catastrophic forgetting include using smaller learning rates, mixing general-domain data with the fine-tuning data, using elastic weight consolidation that penalizes large changes to important parameters, and early stopping based on validation performance on general tasks. For financial institutions, catastrophic forgetting is a critical risk because a fine-tuned model that loses general knowledge may produce incorrect outputs for edge cases or tasks outside the fine-tuning domain without the user realizing the model's general capabilities have degraded.
In Financial Services
Real-World Example
A European bank fine-tunes a Llama 3.1 8B model on 2,000 examples of anti-money laundering (AML) transaction analysis using full fine-tuning with a high learning rate. Before fine-tuning, the base model scores 88% on a general financial knowledge benchmark covering banking regulations, accounting principles, and economic concepts. After fine-tuning, the AML-specific task accuracy improves from 72% to 94%, but the general financial knowledge benchmark score drops to 67% — a catastrophic forgetting loss of 21 percentage points. More concerning, the fine-tuned model begins generating incorrect answers to basic questions about European banking regulations that it previously answered correctly. A compliance officer using the model for AML analysis also asks it about transaction reporting thresholds under EU regulations and receives an incorrect answer because the model has forgotten this general regulation knowledge during fine-tuning. The bank retrains using LoRA with rank r=16, achieving 91% AML accuracy (compared to 94% with full fine-tuning) while maintaining 85% on the general financial knowledge benchmark (compared to 67% after full fine-tuning). The bank also implements a multi-task evaluation approach — measuring performance on both the target task and a set of general financial knowledge tasks — to detect future catastrophic forgetting. The LoRA-based approach is approved by the bank's model risk committee, while the full fine-tuning approach would have required additional validation and safeguards.
Why It Matters for Finance
Catastrophic forgetting is the most underappreciated risk in financial AI fine-tuning. Financial institutions evaluating fine-tuned models typically focus on task-specific accuracy — does the model perform well on the credit analysis task? — without verifying that the model retains its general financial knowledge and safety capabilities. A fine-tuned model that appears to perform excellently on the target task may have degraded to the point where it produces unreliable outputs for any input outside the narrow training distribution. For regulated financial institutions, catastrophic forgetting creates a model risk management challenge. Standard model validation approaches that only evaluate task-specific performance will not detect catastrophic forgetting. Validation frameworks must include evaluation of general capabilities — financial knowledge, reasoning, safety guardrails — both before and after fine-tuning. The magnitude of the problem depends on the fine-tuning approach. Full fine-tuning with aggressive hyperparameters can cause severe forgetting, while PEFT methods like LoRA preserve general capabilities much better. Financial regulators are increasingly aware of this risk, and model risk management guidance for fine-tuned AI systems is expected to require evidence that general capabilities are preserved.
Related Terms
Explore in Finatune
Frequently Asked Questions
What is catastrophic forgetting in AI fine-tuning?
Catastrophic forgetting is when a fine-tuned model loses previously learned knowledge — such as general financial concepts or regulatory knowledge — because the fine-tuning process overwrites the base model's weights. The model performs well on the fine-tuning task but degrades on other tasks. Full fine-tuning is most susceptible, while PEFT methods like LoRA are more resistant.
How does catastrophic forgetting affect financial AI models?
A credit analysis model fine-tuned on loan applications may lose its understanding of basic financial regulations, accounting principles, or economic concepts. An analyst relying on the model for a novel scenario might receive incorrect assessments because the model has forgotten relevant general knowledge. This degradation often goes undetected if testing only covers the fine-tuning task.
How does LoRA prevent catastrophic forgetting in financial models?
LoRA prevents catastrophic forgetting by preserving the base model's original weights and only training small adapter matrices. Since the base model's knowledge remains intact and only the adapter is modified, general capabilities including financial knowledge, reasoning ability, and safety guardrails are preserved. The adapter adds task-specific knowledge without overwriting existing knowledge.