Szczegółowy przewodnik wkrótce
Pracujemy nad kompleksowym przewodnikiem edukacyjnym dla VaR Backtesting Calculator. Wróć wkrótce po wyjaśnienia krok po kroku, wzory, przykłady z życia i porady ekspertów.
VaR backtesting is the statistical process of validating a Value at Risk model by comparing predicted VaR estimates to subsequently realized portfolio profits and losses. An exceedance (or 'exception') occurs on any day when the actual portfolio loss exceeds the predicted VaR for that day. For a correctly specified 99% VaR model, we expect exceedances to occur on approximately 1% of trading days — about 2–3 days per year for a 250-trading-day calendar. Backtesting asks: is the observed number of exceedances statistically consistent with what the model predicts? The Basel Committee first formalized VaR backtesting requirements in its Market Risk Amendment (1996) and has refined them through subsequent accords. Under Basel III, banks using internal VaR models for market risk capital must conduct daily backtesting of their 99% 1-day VaR estimates against actual P&L. The regulatory traffic light framework classifies the backtest result based on the number of exceedances in the most recent 250 trading days: 0–4 exceedances (green zone) — model passes; 5–9 exceedances (yellow zone) — capital multiplier increased; 10+ exceedances (red zone) — model fails, requiring revision and potentially moving to standardized approach. The Kupiec (1995) Proportion of Failures (POF) test is the foundational statistical test: it tests whether the observed number of exceedances is consistent with the stated VaR confidence level using a likelihood ratio test. The null hypothesis is that the true exceedance probability equals (1−confidence level). The Christoffersen (1998) conditional coverage test improves on Kupiec by also testing whether exceedances are independent over time — a valid VaR model should not have exceedances clustering together (which would indicate the model is slow to adapt to changing volatility). Backtesting alone is not sufficient to fully validate a VaR model. P&L attribution (explaining each day's P&L using the model's risk factors) is required under Basel FRTB. Hypothetical P&L (using current portfolio positions repriced with yesterday's market changes) must be consistent with theoretical P&L (from the risk model). Models that pass backtesting but fail P&L attribution may still have significant model risk. Model validation extends beyond backtesting to include sensitivity analysis, stress testing, benchmarking against alternative models, and review of modeling assumptions. Exceedance magnitude analysis (are losses on exception days much larger than VaR, suggesting fat tails?) complements the count-based approach.
Expected Exceedances = (1 − Confidence) × N Kupiec LR = −2 × [ln((1−p)^(N−x) × p^x) − ln((1−x/N)^(N−x) × (x/N)^x)] Critical value: χ²(1) at 5% = 3.84 | χ²(2) at 5% = 5.99 (Christoffersen)
- 1Collect daily VaR estimates (at stated confidence level) and actual daily P&L for the backtesting window (minimum 250 trading days).
- 2Count exceedances: days where loss > VaR estimate (i.e., actual P&L is more negative than −VaR).
- 3Calculate expected exceedances: E[x] = (1−confidence) × N. For 99% VaR over 250 days: E[x] = 2.5.
- 4Apply Kupiec POF test: compute LR statistic using the formula; compare to χ²(1) = 3.84 at 5% significance. If LR > 3.84, model fails the test.
- 5Apply Christoffersen conditional coverage test: build the 2×2 transition matrix of consecutive day exceedances; test independence of exceedances.
- 6Apply Basel traffic light framework: 0–4 exceptions = green (no action); 5–9 = yellow (capital add-on); 10+ = red (model failure).
- 7Analyze exception magnitude: are losses on exception days merely slightly above VaR, or dramatically larger? Large exceedances suggest fat-tail underestimation.
Model passes — 3 exceedances consistent with 99% VaR expectation
Expected exceedances = 1% × 250 = 2.5. Observed = 3. LR_POF = −2 × [ln((0.99)^247 × (0.01)^3) − ln((0.988)^247 × (0.012)^3)] ≈ 0.076, which is far below the χ²(1) critical value of 3.84. The model cannot be rejected. 3 exceedances in 250 days is very consistent with a 99% VaR model. Basel traffic light: Green Zone (0–4 exceptions) — no capital multiplier penalty. This is the expected outcome for a well-specified, regularly updated VaR model.
Model fails badly — 12 exceedances means actual risk is 5× VaR model estimate
12 exceedances vs. expected 2.5 — a 4.8× overrun. LR = −2 × [ln(0.99^238 × 0.01^12) − ln(0.952^238 × 0.048^12)] ≈ 24.1, which vastly exceeds χ²(1)=3.84. The model is statistically rejected at any reasonable significance level. Basel Red Zone (10+ exceptions): the bank must explain the model failures to regulators, the capital multiplier is increased, and the bank may be required to switch to the standardized approach for capital. Common causes: underestimated volatility, ignored fat tails, insufficient correlation capture, or model applied outside its calibration range.
Independence failure means model doesn't adapt quickly to volatility regime changes
5 exceedances in 250 days at 99% VaR is in the Basel yellow zone. The Kupiec POF test may marginally pass (borderline). However, 4 of 5 exceptions occurring on consecutive days is a strong violation of the independence assumption. Christoffersen's CC test computes the transition matrix: (day after non-exception being exception) vs. (day after exception being exception). If p(exception | previous exception) >> p(exception | previous non-exception), independence fails. Clustering indicates the VaR model is slow to respond to volatility spikes — likely using a long historical window or rolling volatility rather than an adaptive estimator.
One catastrophic day ($2.1M = 4.2×VaR) suggests fat tails underestimation
Four of five exceptions are only slightly above VaR ($480K–$550K) — these are expected and consistent with a well-calibrated model. But one day saw a loss of $2.1M — 4.2× the VaR estimate. This outlier is extremely unlikely under a normal distribution (probability of a loss > 4.2σ ≈ 0.001%). This suggests the return distribution has fat tails that the VaR model is not capturing. Average exception = $832K is 1.66× VaR — expected to be approximately 1.14× for a 99% normal VaR — again suggesting fat tails. The exception magnitude analysis triggers a model review even if the count (5) is in the yellow zone.
Bank regulatory capital reporting under Basel III FRTB, representing an important application area for the Var Back Testing in professional and analytical contexts where accurate var back testing calculations directly support informed decision-making, strategic planning, and performance optimization
Internal model validation and model risk management, representing an important application area for the Var Back Testing in professional and analytical contexts where accurate var back testing calculations directly support informed decision-making, strategic planning, and performance optimization
Hedge fund risk model performance monitoring, representing an important application area for the Var Back Testing in professional and analytical contexts where accurate var back testing calculations directly support informed decision-making, strategic planning, and performance optimization
Insurance company risk model validation under Solvency II, representing an important application area for the Var Back Testing in professional and analytical contexts where accurate var back testing calculations directly support informed decision-making, strategic planning, and performance optimization
Trading desk performance attribution — distinguishing P&L from risk factor exposure, representing an important application area for the Var Back Testing in professional and analytical contexts where accurate var back testing calculations directly support informed decision-making, strategic planning, and performance optimization
In the Var Back Testing, this scenario requires additional caution when interpreting var back testing results. The standard formula may not fully account for all factors present in this edge case, and supplementary analysis or expert consultation may be warranted. Professional best practice involves documenting assumptions, running sensitivity analyses, and cross-referencing results with alternative methods when var back testing calculations fall into non-standard territory.
Extremely large or small input values in the Var Back Testing may push var back
Extremely large or small input values in the Var Back Testing may push var back testing calculations beyond typical operating ranges. While mathematically valid, results from extreme inputs may not reflect realistic var back testing scenarios and should be interpreted cautiously. In professional var back testing settings, extreme values often indicate measurement errors, unusual conditions, or edge cases meriting additional analysis. Use sensitivity analysis to understand how results change across plausible input ranges rather than relying on single extreme-case calculations.
In the Var Back Testing, this scenario requires additional caution when interpreting var back testing results. The standard formula may not fully account for all factors present in this edge case, and supplementary analysis or expert consultation may be warranted. Professional best practice involves documenting assumptions, running sensitivity analyses, and cross-referencing results with alternative methods when var back testing calculations fall into non-standard territory.
| Exceptions (x) | Probability (if model correct) | Cumulative Probability | Zone | Capital Multiplier k |
|---|---|---|---|---|
| 0 | 8.1% | 8.1% | Green | 3.00 |
| 1 | 20.5% | 28.6% | Green | 3.00 |
| 2 | 25.7% | 54.4% | Green | 3.00 |
| 3 | 21.5% | 75.9% | Green | 3.00 |
| 4 | 13.5% | 89.4% | Green | 3.00 |
| 5 | 6.8% | 96.2% | Yellow | 3.40 |
| 6–9 | 3.5% | 99.7% | Yellow | 3.50–3.85 |
| 10+ | 0.3% | ≥99.7% | Red | 4.00 |
Why is backtesting important for VaR models?
Backtesting is the empirical validation that a VaR model produces the calibration it claims. A bank claiming 99% VaR should, by definition, experience losses exceeding VaR approximately 1% of the time. Without backtesting, a model could systematically understate risk (producing too-low VaR, which reduces capital requirements) with no accountability mechanism. Backtesting creates a regulatory feedback loop: models that fail are penalized with capital add-ons, creating financial incentives for banks to maintain well-calibrated models. Backtesting also helps risk managers identify when models are becoming obsolete due to changed market dynamics.
What is the difference between clean and dirty P&L for backtesting?
Clean P&L (hypothetical P&L) measures the gain or loss on yesterday's portfolio positions repriced using today's market prices — isolating pure market risk exposure without the effect of new trades, fees, or portfolio changes. Dirty P&L (actual P&L) includes all P&L from all sources: trading revenues, new positions, fee income, and operational items. Basel FRTB requires backtesting against hypothetical clean P&L, because the VaR model estimates the risk of the existing portfolio, not the evolving portfolio. If VaR is compared to dirty P&L, good trading days can obscure risk model failures, and vice versa.
How many backtesting years are needed for reliable model validation?
The minimum regulatory requirement is 250 trading days (≈1 year). However, this is statistically quite limited: at 99% VaR, only 2.5 expected exceptions occur per year. The standard error of the estimated exception rate with 2.5 observations is very high — it is impossible to distinguish between a model that produces 0.5%, 1%, 1.5%, or 2% true exception rates with only 250 observations. For robust validation, 3–5 years of data are preferred. Even then, the Kupiec test has low power at distinguishing small systematic errors. This is why multiple statistical tests and supplementary validation methods (stress testing, P&L attribution) are required alongside backtesting.
What is the Basel traffic light system for VaR backtesting?
The Basel traffic light system (introduced in 1996 and updated in Basel III) classifies VaR model performance based on the number of backtesting exceptions in the most recent 250 trading days: Green Zone (0–4 exceptions) — model passes, no capital penalty; Yellow Zone (5–9 exceptions) — capital multiplier k increases from 3.0 to 3.4–4.0 depending on the number, and regulatory scrutiny increases; Red Zone (10+ exceptions) — model fails, k=4.0 or higher, and the bank may be required to switch to the standardized approach. The thresholds are set to balance Type I error (rejecting good models) and Type II error (accepting bad models) for a 250-day sample.
What is P&L attribution and how does it relate to backtesting?
P&L attribution (PLA) requires banks to explain each day's actual P&L by decomposing it into contributions from the risk factors captured in the VaR model. Under Basel FRTB, the difference between the VaR model's theoretical P&L (using model risk factors) and the actual hypothetical P&L must be small and uncorrelated with actual P&L. If significant P&L unexplained by the model exists (model residuals are large), this suggests the VaR model is missing important risk factors and may be understating risk. PLA is performed at the trading desk level, while backtesting is also conducted at the desk level under FRTB.
Can backtesting detect model misspecification beyond just frequency?
Standard Kupiec backtesting only detects incorrect exceedance frequency. It cannot detect: (1) incorrect tail shape — if the model underestimates loss severity on exception days; (2) incorrect risk factor sensitivities — if the model correctly predicts the frequency but for the wrong reasons; (3) correlation misspecification — if diversification benefits are overstated. The Christoffersen test adds independence testing. Loss function-based tests (compare expected vs. realized P&L distribution) and regression-based tests provide additional diagnostic power. A complete model validation program uses multiple tests, stress testing, sensitivity analysis, and expert model review to identify different types of misspecification.
What should a risk manager do when a VaR model shows too many exceptions?
Excess exceptions trigger a structured model review process: (1) Investigate each exception: was the loss driven by a specific event, data error, or genuine model shortcoming? (2) Check whether volatility estimates are current — stale volatility assumptions are a common cause of VaR underestimation in rapidly changing markets. (3) Review correlation assumptions — check whether crisis correlations are being used when appropriate. (4) Test alternative distribution assumptions — replace normal with t-distribution or historical simulation. (5) Examine concentration risks not fully captured by the model. (6) If the model cannot be quickly remediated, increase capital charges or reduce position limits until the model is recalibrated.
Wskazówka Pro
Maintain a rolling backtest chart showing cumulative exceptions over the last 250 days alongside the green/yellow/red thresholds. Plot this daily so model deterioration is visible as a trend before the model crosses into the yellow zone — enabling proactive recalibration.
Czy wiedziałeś?
The Basel traffic light backtesting framework was introduced in the 1996 Market Risk Amendment after regulators discovered that some banks' VaR models, while passing internal validation, were producing systematically low VaR estimates — effectively gaming the capital system. The 250-day, 99% threshold with green/yellow/red zones was chosen specifically to balance two competing risks: incorrectly penalizing good models (Type I error) vs. failing to detect bad models (Type II error). Even today, the framework is recognized as statistically underpowered — it takes many months of persistent model failure before the evidence accumulates enough to trigger the red zone.
Źródła
- ›Kupiec, P.H. (1995): Techniques for Verifying the Accuracy of Risk Measurement Models, Journal of Derivatives
- ›Christoffersen, P. (1998): Evaluating Interval Forecasts, International Economic Review
- ›Basel Committee: Supervisory Framework for the Use of Backtesting (1996)
- ›McNeil, Frey & Embrechts: Quantitative Risk Management — Backtesting Chapter