How Level III tests benchmark quality, liability-based and asset-based benchmarks, benchmark misspecification, and appraisal of manager skill.
At Level III, a benchmark is not just a comparison convenience. It is part of the logic of evaluation. If the benchmark is wrong, attribution and appraisal can both look precise while still being misleading.
Candidates often know a few appraisal metrics but miss the real question:
Level III often rewards benchmark judgment before metric calculation.
| Benchmark type | Best use |
|---|---|
| Liability-based benchmark | When the investor’s obligations are central to success |
| Asset-based benchmark | When the mandate is framed around investable market opportunity sets |
| Peer or custom composite benchmark | When the portfolio structure is too specific for a broad public index alone |
The stronger answer starts by asking what success actually means for the investor.
| Benchmark-quality test | Why it matters |
|---|---|
| Unambiguous | The benchmark should be defined clearly enough that evaluation is consistent |
| Investable | The manager should be able to implement something meaningfully comparable |
| Measurable | Data should exist to evaluate it reliably |
| Appropriate | It should match the mandate, objective, and opportunity set |
| Specified in advance | It should not be changed after performance is known |
This quality logic often matters more than the exact benchmark label.
| If the benchmark is wrong | What goes wrong next |
|---|---|
| Too easy | The manager may look skillful without true value add |
| Too hard or structurally mismatched | The manager may look weak for reasons unrelated to process quality |
| Misaligned with liability or investor objective | “Outperformance” may still fail the real purpose of the portfolio |
Level III often tests whether the candidate can spot that the benchmark problem came first.
The Sortino ratio is often written as:
$$ \text{Sortino ratio} = \frac{R_P - R_T}{\sigma_D} $$
where (R_T) is the target or required return and (\sigma_D) is downside deviation.
The appraisal ratio is commonly written as:
$$ \text{Appraisal ratio} = \frac{\alpha_P}{\sigma_{\varepsilon}} $$
where active return is scaled by residual or specific risk.
| Metric | What it emphasizes | Limitation |
|---|---|---|
| Sortino ratio | Downside-risk-adjusted performance | Depends on the chosen target return |
| Appraisal ratio | Active value added per unit of residual risk | Only useful if the benchmark model is sensible |
| Upside or downside capture ratios | How the manager behaves in up and down markets | Can hide path dependency and regime effects |
| Maximum drawdown and drawdown duration | Capital-loss depth and recovery pain | Sensitive to sample period and path |
The exam may ask you to compute or interpret one, but it still expects judgment about fit and limitation.
| Apparent good result | Stronger Level III question |
|---|---|
| High active return | Was the benchmark appropriate and was the risk intentional? |
| Strong downside capture profile | Was it caused by process quality or by one favorable regime? |
| Low drawdown | Was the portfolio simply running lower market exposure than the benchmark? |
This is why skill evaluation is more than comparing one ratio.
Alternative investments complicate evaluation because of:
Level III often uses this to test whether the candidate realizes that neat public-market benchmarking tools may not translate cleanly.
A private DB plan outperformed its asset-based policy benchmark, but the liability benchmark still deteriorated because discount-rate sensitivity was managed poorly. The manager’s appraisal ratio versus the policy benchmark looks attractive.
A weak answer praises the manager’s strong risk-adjusted performance.
A stronger answer asks whether the benchmark used for appraisal ignored the plan’s actual liability problem, making the skill conclusion incomplete.
Why can a strong appraisal ratio still be misleading?
Best answer: Because the ratio may look strong even when the benchmark is misspecified or misaligned with the portfolio’s true objective.
Why: Level III rewards evaluation logic, not blind metric worship.