Draft:Deflated Sharpe Ratio

The Deflated Sharpe Ratio (DSR) is a statistical method used to determine whether the Sharpe Ratio of an investment strategy is statistically significant, after correcting for selection bias, backtest overfitting, and non-normality in return distributions. It provides a more reliable test of financial performance, especially when many trials are evaluated.

Important:

The application of the DSR, helps practitioners to detect false investment strategies.

Background on the Sharpe Ratio

The most important statistic in investment management is the Sharpe Ratio (SR), which was developed by William F. Sharpe and is a widely used measure of risk-adjusted return, calculated as the ratio of excess return over the risk-free rate to the standard deviation of returns. While useful, the Sharpe Ratio has limitations, especially when applied to multiple strategy evaluations. Issues such as selection bias, where the best-performing strategy is chosen from a large set, and backtest overfitting, where a strategy is tailored to past data, can inflate the Sharpe Ratio, leading to misleading conclusions about a strategy's effectiveness. Additionally, the Sharpe Ratio assumes normally distributed returns, an assumption often violated in practice, and it does not take into account sample length.

Applying the Deflated Sharpe Ratio in Practice

1. Get a record of all the trials.

In order to apply the DSR, researchers need to record the investment performance in returns (%), for every backtest that they ran. This is in relation to the development of a single specific strategy. For example: when building a momentum based strategy that trades at the end-of-day, 100 historical simulations were run to evaluate the performance and the best set of parameters were selected for the final strategy. Here all 100 simulations need to be recorded, with the strategies daily returns in %.

2. Estimating the Effective Number of Trials N.

In practice, many trials are not independent due to overlapping features. To estimate the effective number of independent trials N, López de Prado (2018) proposes 3 techniques to clustering similar strategies using unsupervised learning techniques:

1. The Optimal Number of Clusters (ONC) algorithm.
2. Hierarchical clustering could be used to get a conservative lower bound for N.
3. Alternatively, spectral methods (e.g. eigenvalue distribution of the correlation matrix) can also provide estimates of N.

Tip:

• Multiple testing exercises should be carefully planned in advance, so as to avoid running an unnecessary large number of trials. Investment theory, not computational power, should motivate what experiments are worth conducting.

Steps to estimate N:

2.1. Convert the correlation matrix to a distance matrix.

In order to apply a clustering algorithm to the returns data, we need make use of a statistical association measure (such as a correlation matrix) and we need to transform it into a distance matrix (such as angular distance) so that elements that are very similar to each other will be close together in their higher-dimensional space.

2.2. Apply a clustering algorithm to estimate the number of independent trials.

The number of clusters N, are an estimate of the number of independent trials.

2.3 Plot the Block Correlation Matrix

In the figure below we can see a correlation matrix before and after clustering has been applied. Note how we can see blocks down the diagonal, each block corresponds to a cluster.

Clustering of 6,385 trials, typical of multiple testing of a group of strategies, before and after clustering.

Tip: If you don't use the ONC algorithm to cluster, then you can have blocks with trials that don't match very closely. The ONC algorithm uses silhouette scores to make sure each trial is in the best cluster, at the expense of high computational complexity and longer run times.

3. Compute the Sharpe ratio variance, across clusters.

3.1 Calculate the Sharpe ratio for each cluster.

Each cluster will now form a collection of time series returns (in%), for each cluster you need to create a new time series which represents that cluster using the Inverse Variance Portfolio (IVP) and then compute the Sharpe Ratio for each IVP portfolio. One doesn't need to use the IVP - the goal is to form an aggregate cluster return time series. For this a weighting scheme needs to be used, another alternative could be the minimum variance portfolio.

3.2 Compute the variance of these Sharpe Ratios

$\mathbf {V} [{\hat {SR}}_{n}]$ is used in the next step, where we apply the False Strategy Theorem to determine the Expected Maximum Sharpe ratio.

4. Compute the Expected Maximum Sharpe ratio using the False Strategy Theorem.

Using the equation from the False Strategy Theorem (FST) we can compute $SR_{0}$, which is the threshold Sharpe Ratio that reflects the highest Sharpe Ratio expected from $N$ unskilled strategies.

$SR_{0}={\sqrt {\mathbf {V} [{\hat {SR}}_{n}]}}\left((1-\gamma )\Phi ^{-1}\left[1-{\frac {1}{N}}\right]+\gamma \Phi ^{-1}\left[1-{\frac {1}{Ne}}\right]\right)$

Where:

• $\mathbf {V} [{\hat {SR}}_{n}]$ is the cross-sectional variance of Sharpe Ratios across trials,
• $\gamma$ is the Euler-Mascheroni constant (approx. 0.5772),
• $e$ is Euler's number,
• $\Phi ^{-1}$ is the inverse standard normal CDF,
• $N$ is the number of independent strategy trials.

Note:

The FST highlights that the optimal outcome of an unknown number of historical simulations is right-unbounded, with enough trials, there is no Sharpe ratio sufficiently large enough to reject the hypothesis that a strategy is false, i.e., that it is over-fit and wont generalize in the out-of-sample data.

The figure shows how the $Max\{SR\}$ increases as a function of purely the number of trials.

5. Compute the DSR for each cluster.

You now have all the variables you need to compute the DSR.

${\text{DSR}}=\Phi \left({\frac {({\hat {SR}}-SR_{0})\cdot {\sqrt {T-1}}}{\sqrt {1-{\hat {\gamma }}_{3}{\hat {SR}}+{\frac {{\hat {\gamma }}_{4}-1}{4}}{\hat {SR}}^{2}}}}\right)$

Where:

• ${\hat {SR}}$ is the observed Sharpe Ratio (not annualized),
• $SR_{0}$ is the threshold Sharpe Ratio that reflects the highest Sharpe Ratio expected from $N$ unskilled strategies,
• ${\hat {\gamma }}_{3}$ is the skewnewss of the returns,
• ${\hat {\gamma }}_{4}$ is the kurtosis of the returns,
• $T$ is the returns' sample length.
• $\Phi$ is the standard normal cumulative distribution function.

Notes:

• Readers will see that the DSR is the Probabilistic Sharpe Ratio (PSR), where $SR_{0}$ is the maximum expected Sharpe Ratio (estimated using the False Strategy Theorem) instead of a simple threshold SR (often 0).
• The PSR assumes that only 1 trial was run and is often used to determine that the observed SR is greater than 0.
• To account for multiple-testing, use the DSR.
• The DSR will increase with:
  • Greater observed SRs.
  • Longer track records.
  • Positively skewed returns.
• The DSR decreases with:
  • Fatter tails (Kurtosis).

6. Complete the Template for Disclosing Multiple Tests.

6.1 Aggregate statistics into a table.

Several peer reviewed papers recommend to aggregate the cluster statistics into a table format.

The table below is Exhibit 7 from "A Practitioner’s Guide to the Optimal Number of Clusters Algorithm".

Template for Disclosing Multiple Tests

Where:

• Cluster is the index of the cluster; there are N clusters.
• Strat Count is the number of strategies included in that cluster.
• aSR is the annualized Sharpe Ratio of that cluster's inverse variance portfolio (IVP).
• SR is the non-annualized Sharpe Ratio of that cluster's IVP.
• Skew is the skew of the returns of that cluster's IVP.
• Kurt is the kurtosis of the returns of that cluster's IVP.
• T is the number of observations in the cluster's IVP.
• sqrt(V[SR]) is the square root of the variance of Sharpe Ratios that was computed in step 3.
• E[max SR] is the Expected Maximum Sharpe ratio ($SR_{0}$), computed in step 4.
• DSR is the Deflated Sharpe Ratio for that cluster's IVP.

6.2 Plot the Sharpe Ratios, for each cluster.

Bar plot of Deflated Sharpe Ratios, for each cluster.

In the figure above, we can see a collection of non-annualized Sharpe ratios for the 26 independent trials that were tested in the development of this investment strategy. The bars are highlighted based on if they passed the DSR at a 95% confidence level.

Note that this bar chart doesn't correspond to table above in Exhibit 7 but shares the result that only 1 cluster passed the DSR. The goal with this analysis is to show that for all clusters, except 1 - all of them failed the DSR. This would indicate that the strategy is over-fit and is likely to be a false investment strategy.

6.3 Plot the cumulative returns of the strategies.

ICumulative Returns of Trials

In the figure above the cumulative returns are plotted. On the y axis is the total return in% and the x axis are the time indexes. Do you see the very straight line (the strategy with an outlier performance)?

7. Derive a conclusion from these results.

As seen in the plot of cumulative returns, there is one outlier strategy which is likely a false investment strategy as this outlier has very high performance relative to its own cluster and others.

We can see in the bar plots that all the cluster portfolios failed to pass the DSR at a 95% confidence level, except for the one that included this outlier strategy.

Mathematical Definitions

The Deflated Sharpe Ratio (DSR)

${\text{DSR}}=\Phi \left({\frac {({\hat {SR}}-SR_{0})\cdot {\sqrt {T-1}}}{\sqrt {1-{\hat {\gamma }}_{3}{\hat {SR}}+{\frac {{\hat {\gamma }}_{4}-1}{4}}{\hat {SR}}^{2}}}}\right)$

Where:

• ${\hat {SR}}$ is the observed Sharpe Ratio (not annualized),
• $SR_{0}$ is the threshold Sharpe Ratio that reflects the highest Sharpe Ratio expected from $N$ unskilled strategies,
• ${\hat {\gamma }}_{3}$ is the skewnewss of the returns,
• ${\hat {\gamma }}_{4}$ is the kurtosis of the returns,
• $T$ is the returns' sample length.
• $\Phi$ is the standard normal cumulative distribution function.

The threshold $SR_{0}$ is approximated by:

$SR_{0}={\sqrt {\mathbf {V} [{\hat {SR}}_{n}]}}\left((1-\gamma )\Phi ^{-1}\left[1-{\frac {1}{N}}\right]+\gamma \Phi ^{-1}\left[1-{\frac {1}{Ne}}\right]\right)$

Where:

• $\mathbf {V} [{\hat {SR}}_{n}]$ is the cross-sectional variance of Sharpe Ratios across trials,
• $\gamma$ is the Euler-Mascheroni constant (approx. 0.5772),
• $e$ is Euler's number,
• $\Phi ^{-1}$ is the inverse standard normal CDF,
• $N$ is the number of independent strategy trials.

False Strategy Theorem: Statement and Proof

The False Strategy Theorem provides the theoretical foundation for the Deflated Sharpe Ratio (DSR) by quantifying how much the best Sharpe Ratio among many unskilled strategies is expected to exceed zero purely due to chance. Even if all tested strategies have true Sharpe Ratios of zero, the highest observed Sharpe Ratio will typically be positive and statistically significant—unless corrected. The DSR corrects for this inflation.

Statement

Let $\{{\hat {SR}}_{1},{\hat {SR}}_{2},\dots ,{\hat {SR}}_{N}\}$ be $N$ Sharpe Ratios independently drawn from a normal distribution with mean zero and variance $\sigma ^{2}$. Then the expected maximum Sharpe Ratio among these $N$ trials is approximately:

$SR_{0}={\sqrt {\sigma ^{2}}}\cdot \left((1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)\right)$

Where:

• $\Phi ^{-1}$ is the quantile function (inverse CDF) of the standard normal distribution,
• $\gamma \approx 0.5772$ is the Euler–Mascheroni constant,
• $e\approx 2.718$ is Euler’s number,
• $N$ is the number of independent trials.

This value $SR_{0}$ is the **expected maximum Sharpe Ratio** under the null hypothesis of no skill. It represents a benchmark that any observed Sharpe Ratio must exceed in order to be considered statistically significant.

Proof Sketch

Let $X_{1},X_{2},\dots ,X_{N}\sim {\mathcal {N}}(0,1)$ be independent standard normal variables. The expected maximum of $N$ such variables is approximated by:

$\mathbb {E} [\max(X_{1},\dots ,X_{N})]\approx (1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)$

Now let ${\hat {SR}}_{i}\sim {\mathcal {N}}(0,\sigma ^{2})$ for each $i$. Then:

$\mathbb {E} \left[\max({\hat {SR}}_{1},\dots ,{\hat {SR}}_{N})\right]=\sigma \cdot \mathbb {E} [\max(X_{1},\dots ,X_{N})]$

Combining the two expressions gives:

$SR_{0}=\sigma \cdot \left((1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)\right)$

If $\sigma ^{2}$ is estimated as the cross-sectional variance of Sharpe Ratios $\mathbf {V} [{\hat {SR}}_{n}]$, then:

$SR_{0}={\sqrt {\mathbf {V} [{\hat {SR}}_{n}]}}\cdot \left((1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)\right)$

This completes the derivation.

Implication for the DSR

The False Strategy Theorem shows that in large-scale testing, even unskilled strategies will produce apparently "significant" Sharpe Ratios. To correct for this, the DSR adjusts the observed Sharpe Ratio by subtracting the expected maximum from noise, $SR_{0}$, and scaling by its standard error:

${\text{DSR}}=\Phi \left({\frac {{\hat {SR}}-SR_{0}}{\sigma _{\hat {SR}}}}\right)$

This yields the probability that the observed Sharpe Ratio reflects true skill, not selection bias or overfitting.

Confidence and Power of the Sharpe Ratio under Multiple Testing

To assess the significance of Sharpe Ratios under multiple testing, López de Prado (2018) derives closed-form expressions for the Type I and Type II errors.

Confidence

The probability that a discovered strategy is not a false positive (i.e., the confidence) is:

${\text{Confidence}}=1-\alpha _{K}=\left(\Phi \left({\frac {{\hat {SR}}\cdot {\sqrt {T-1}}}{\sqrt {1-\gamma _{3}{\hat {SR}}+{\frac {\gamma _{4}-1}{4}}{\hat {SR}}^{2}}}}\right)\right)^{K}$

Where:

• $T$ is the number of return observations,
• $\gamma _{3}$ and $\gamma _{4}$ are the skewness and kurtosis of returns,
• $K$ is the number of effectively independent trials.

Power

The power of the test, i.e., the probability of correctly identifying a strategy with true Sharpe Ratio $SR^{*}$, is:

${\text{Power}}=1-\beta _{K}=1-\left(\Phi \left(\Phi ^{-1}\left[(1-\alpha _{K})^{1/K}\right]-\theta \right)\right)^{K}$

With:

$\theta ={\frac {SR^{*}\cdot {\sqrt {T-1}}}{\sqrt {1-\gamma _{3}{\hat {SR}}+{\frac {\gamma _{4}-1}{4}}{\hat {SR}}^{2}}}}$

These equations quantify the reliability of observed Sharpe Ratios under multiple testing and return non-normality.

See also

• Sharpe Ratio
• Backtesting
• Overfitting
• Selection bias
• Multiple comparisons problem