royy/chinese-equity-quant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use next-open returns for research eval

Browse Source
This commit is contained in:
Yuxuan Yan
2026-06-12 18:41:18 +08:00
parent 16b4988f16
commit 3c58a1372e
10 changed files with 552 additions and 270 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reversal_5d_all_universe_pipeline_report.md
+93 -107
View File
@@ -1,14 +1,15 @@
# Tutorial: Testing a 5-Day Reversal Alpha
This document is a teaching walkthrough for someone who is new to this
research framework and only lightly familiar with quant research. We will use
one concrete experiment, a 5-day reversal alpha on the full downloaded Chinese
Generated: 2026-06-12T18:30:56
This document is a teaching walkthrough for someone who is new to this research
framework and only lightly familiar with quant research. We will use one
concrete experiment, a 5-day reversal alpha on the full downloaded Chinese
A-share universe, to learn how the framework defines an alpha, stores it, tests
it, turns it into a portfolio, and explains the gap between a research result
and simulated trading PnL.
The original experiment was generated on 2026-06-11. The important point is not
the timestamp; it is the research method.
The important point is not the timestamp; it is the research method.
## The Research Question
@@ -29,10 +30,13 @@ The central research question is:
The answer from this run is nuanced:
- The naive built-in version loses badly on the full universe because raw
z-score weighting is too sensitive to A-share outliers.
- The naive built-in version is positive under the tradable
next-open-to-next-open research convention (**41.40%**),
but its stored weights still show that raw z-score weighting is too sensitive
to A-share outliers.
- A rank-weighted version on a liquid, non-ST, tradable universe has a positive
costless research result.
costless research result: **209.58%**
at Sharpe **1.44**.
- The daily-traded implementation is still not tradable after costs because
turnover is too high.
@@ -90,11 +94,11 @@ For this experiment, the important phases are:
| --- | --- | --- |
| Data | `cli.py data download` | What market data is available. |
| Alpha compute | `cli.py alpha compute` | How a raw research idea becomes stored weights. |
| Alpha eval | `cli.py alpha eval` | How those weights perform in a clean costless research view. |
| Alpha eval | `cli.py alpha eval` | How close-formed weights perform over the tradable next-open-to-next-open interval. |
| Combo | `cli.py combo combine` | How one or more alphas become one combined book. |
| Portfolio build | `cli.py portfolio build` | How weights become target values and integer shares. |
| Portfolio simulate | `cli.py portfolio simulate` | How the integer book trades at next open with constraints and costs. |
| Portfolio eval | `cli.py portfolio eval` | How the continuous target portfolio behaves as a research portfolio. |
| Portfolio eval | `cli.py portfolio eval` | How the continuous target portfolio behaves over the same costless open-to-open research interval. |
In a real research workflow, you should learn to pause after every phase and
inspect the parquet output. Most mistakes are easier to find at the interface
@@ -123,25 +127,9 @@ So:
Notice the timing. The signal uses prices through date `t`. It must not use the
return from `t` to `t+1`, because that is the future. The costless alpha
evaluator tests the weight formed on date `t` against the next close-to-close
return; the later execution simulator is the separate layer that trades the
constructed integer book at the next open.
The code lives in `pipeline/alpha/library/reversal.py`:
```python
class ReversalAlpha(BaseAlpha):
name = "reversal"
def __init__(self, lookback: int = 5):
self.lookback = lookback
def signal(self, close: pd.DataFrame) -> pd.DataFrame:
return -close.pct_change(self.lookback, fill_method=None)
```
The alpha class only defines the raw signal. The base class then turns that
signal into weights.
evaluator tests the weight formed on date `t` over the tradable interval from
`open[t+1]` to `open[t+2]`; the later execution simulator is the separate layer
that trades the constructed integer book at the next open.
## Step 2: Turn A Signal Into Cross-Sectional Weights
@@ -161,17 +149,19 @@ returns because they are newly listed, suspended, illiquid, or limit-constrained
z-scoring can put a very large amount of relative exposure into exactly those
names.
That is what happened in the naive full-universe run. Stored weights reached
about `-52` standard deviations. The research result collapsed:
That is visible in the naive full-universe run. Stored weights reached about
`-52` standard deviations. The
result is positive under the open-to-open convention, but it is much weaker and
less robust than the rank-weighted versions:
| run | weighting | research cumulative return | research Sharpe | research turnover/year |
| --- | --- | --- | --- | --- |
| naive z-score, full universe | z-score | -87.45% | -2.4515 | 160x |
| naive z-score, full universe | z-score | 41.40% | 0.4514 | 160x |
The lesson is not "reversal is bad." The lesson is:
The lesson is not "reversal is solved." The lesson is:
> The same raw signal can become a bad portfolio if the weighting method reacts
> badly to outliers.
> The same raw signal can become a fragile portfolio if the weighting method
> reacts badly to outliers.
## Step 3: Make The Weighting More Robust
@@ -186,15 +176,15 @@ weights = ranks.subtract(ranks.mean(axis=1), axis=0)
Rank weighting keeps the ordering of stocks but removes the importance of the
exact outlier magnitude. A stock can be "the worst recent loser" or "the best
recent winner," but it cannot become 52 standard deviations important just
because its raw percentage move is unusual.
recent winner," but it cannot become dozens of standard deviations important
just because its raw percentage move is unusual.
The full-universe rank version was much less pathological, but still not a
clean signal:
| run | weighting | research cumulative return | research Sharpe | research turnover/year |
| --- | --- | --- | --- | --- |
| rank, full universe | rank | -3.48% | -0.0198 | 143x |
| rank, full universe | rank | 73.44% | 0.8860 | 143x |
That tells us the weighting fix helped, but the universe still contains many
names that are poor candidates for a daily reversal strategy.
@@ -218,7 +208,7 @@ The liquid rank result is the cleanest research result:
| run | weighting | universe | research cumulative return | research Sharpe | hit rate |
| --- | --- | --- | --- | --- | --- |
| rank, liquid subset | rank | top 1000 liquid, tradable, non-ST | 72.24% | 0.7310 | 54.31% |
| rank, liquid subset | rank | top 1000 liquid, tradable, non-ST | 209.58% | 1.4422 | 55.68% |
This is the first point where a researcher can say:
@@ -229,13 +219,6 @@ That last phrase, **before trading costs**, is essential.
![Research equity](assets/reversal_5d_research_equity.png)
When reading this chart, focus on the shape and relative behavior:
- The naive z-score line shows the outlier problem.
- The rank full-universe line shows that robust weighting helps but does not
fully solve the universe problem.
- The liquid rank line shows the signal-level edge before execution costs.
## Step 5: Check That The Alpha File Is Sane
Before trusting any metric, inspect the stored alpha artifact. The run checked:
@@ -247,7 +230,7 @@ Before trusting any metric, inspect the stored alpha artifact. The run checked:
- The daily cross-sectional mean is approximately zero.
- A one-alpha combo is an exact identity transform.
| run | schema ok | null weights | non-finite weights | duplicate keys | max abs daily mean | weight range | combo identity diff |
| run | schema ok | null w | non-finite w | dup keys | max \|daily mean\| | weight range | combo identity Δ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| naive z-score (full) | True | 0 | 0 | 0 | 3.32e-16 | [-52.2, 19.2] | 0.00e+00 |
| rank (full) | True | 0 | 0 | 0 | 0.00e+00 | [-2603.0, 2603.0] | 0.00e+00 |
@@ -261,51 +244,42 @@ raw size of an abnormal stock move.
![Weight distributions](assets/reversal_5d_weight_distributions.png)
This is a good habit: when a backtest looks strange, plot the weights before
debugging the PnL. A broken or concentrated weight distribution often explains
the result.
## Step 6: Understand The Alpha Evaluation Formula
The costless alpha evaluator asks:
The costless alpha evaluator now asks:
> If we held the alpha weights from date `t`, what close-to-close return would
> we earn from `t` to `t+1`?
> If we compute alpha weights after close on date `t`, trade them at `open[t+1]`,
> and hold them until `open[t+2]`, what return would we earn before costs?
This is intentionally a **research-layer approximation**, not the trading
simulator. At this stage the framework has only an alpha weight file. It has not
yet rounded shares, checked limits, clipped trades, or paid costs. The purpose
is to answer a clean signal question: "Do these close-formed weights line up
with the next day's returns?"
The actual trading layer comes later. `portfolio simulate` takes the integer
`position_shares` from the portfolio builder, executes the target from signal
date `t` at `open[t+1]`, then marks PnL as overnight movement on the old book
plus intraday movement on the newly filled book, minus trading cost.
This is still a **research-layer approximation**, not the trading simulator. At
this stage the framework has only an alpha weight file. It has not yet rounded
shares, checked limits, clipped trades, or paid costs. The purpose is to answer
a clean signal question: "Do these close-formed weights line up with returns
over the interval we could actually own after next-open execution?"
The daily research return is:
```text
R[t] = sum_i(weight[i, t] * return[i, t+1]) / sum_i(abs(weight[i, t]))
R[t] = sum_i(weight[i, t] * (open[i, t+2] / open[i, t+1] - 1)) / sum_i(abs(weight[i, t]))
```
This has three important consequences:
- The alpha is normalized by its gross exposure, so the scale of raw weights
does not by itself create a higher return.
- The next day's return is used, so the test avoids look-ahead.
- The last signal date is dropped from performance metrics because there is no
next return for it.
- The new signal does not receive credit for the overnight gap from `close[t]`
to `open[t+1]`, because it cannot be traded until `open[t+1]`.
- The final two signal dates are dropped from performance metrics because they
do not have a complete next-open-to-next-open holding interval.
Turnover is also measured from the weights:
Turnover is still measured from the weights:
```text
turnover[t] = sum_i(abs(weight[i, t] - weight[i, t-1])) / sum_i(abs(weight[i, t-1]))
```
The annualized turnover numbers around 143x to 160x are a warning. Even a
positive signal can be hard to monetize if it asks the portfolio to trade too
much every day.
The annualized turnover numbers are a warning. Even a positive signal can be
hard to monetize if it asks the portfolio to trade too much every day.
## Step 7: Build A Portfolio From The Alpha
@@ -325,24 +299,14 @@ logic. This is where a research portfolio starts to become a tradable portfolio.
The continuous target portfolio matched the stored alpha almost exactly:
| run | target value identity max abs diff | alpha to target max abs diff | research correlation alpha vs portfolio | mean integer gross | mean L1 tracking |
| run | target_value identity max\|Δ\| | alpha→target max\|Δ\| | research corr(alpha,portfolio) | mean integer gross | mean L1 tracking |
| --- | --- | --- | --- | --- | --- |
| naive z-score (full) | 0.0000 | 0.00e+00 | 1.000000 | 9,138,331 | 2,542,655 |
| rank (full) | 0.0000 | 0.00e+00 | 1.000000 | 8,984,098 | 2,678,278 |
| rank (liquid subset) | 0.0000 | 0.00e+00 | 1.000000 | 9,810,256 | 862,303 |
The integer book is not exact because small target positions can be rounded
away. The liquid subset has lower tracking error because it spreads the book
over fewer and more tradable names.
![Portfolio tracking](assets/reversal_5d_portfolio_tracking.png)
When you research a new alpha, ask two separate questions:
- Does the continuous target portfolio match the alpha? It should.
- Does the integer tradable portfolio still resemble the target? It may not,
especially for small books or very broad universes.
## Step 8: Simulate Execution And Costs
Research returns are not the same as tradable PnL. The simulator executes the
@@ -366,17 +330,19 @@ The execution results explain the final research conclusion:
| run | corr(alpha, exec net) | PnL before cost | total cost | net PnL | mean daily turnover |
| --- | --- | --- | --- | --- | --- |
| naive z-score (full) | 0.9675 | 1,838,974 | 13,032,720 | -11,193,746 | 0.5711 |
| rank (full) | 0.9613 | 5,052,067 | 11,713,451 | -6,661,383 | 0.5133 |
| rank (liquid subset) | 0.9762 | 11,017,842 | 12,733,803 | -1,715,960 | 0.5715 |
| naive z-score (full) | 0.8956 | 1,838,974 | 13,032,720 | -11,193,746 | 0.5711 |
| rank (full) | 0.9126 | 5,052,067 | 11,713,451 | -6,661,383 | 0.5133 |
| rank (liquid subset) | 0.8884 | 11,017,842 | 12,733,803 | -1,715,960 | 0.5715 |
The liquid rank run made about 11.0 million before cost, but paid about 12.7
million in cost. That is why the final net PnL is negative.
For the liquid rank run, simulated PnL before cost is about
11,017,842, but total
cost is about 12,733,803. That is why
the final net PnL is weak or negative.
This is not a contradiction. It is exactly what a research pipeline should show:
> The signal exists in the costless layer, but the daily implementation trades
> too much to keep the edge.
> The signal can exist in the costless layer, but the daily implementation can
> still trade too much to keep the edge.
![Execution vs research](assets/reversal_5d_exec_vs_research.png)
@@ -384,22 +350,27 @@ This is not a contradiction. It is exactly what a research pipeline should show:
The complete summary is:
| run | weighting | research cumulative return | research Sharpe | research turnover/year | exec before cost | exec net | exec net Sharpe |
| run | weighting | research cum | research Sharpe | research turn/yr | exec before cost | exec net | exec net Sharpe |
| --- | --- | --- | --- | --- | --- | --- | --- |
| naive z-score (full) | z-score | -87.45% | -2.4515 | 160x | 18.39% | -111.94% | -1.4508 |
| rank (full) | rank | -3.48% | -0.0198 | 143x | 50.52% | -66.61% | -1.1839 |
| rank (liquid subset) | rank | 72.24% | 0.7310 | 148x | 110.18% | -17.16% | -0.2226 |
| naive z-score (full) | z-score | 41.40% | 0.4514 | 160× | 18.39% | -111.94% | -1.4508 |
| rank (full) | rank | 73.44% | 0.8860 | 143× | 50.52% | -66.61% | -1.1839 |
| rank (liquid subset) | rank | 209.58% | 1.4422 | 148× | 110.18% | -17.16% | -0.2226 |
*Research = costless, no-look-ahead weights over the next-open-to-next-open
holding interval. Execution = next-open fills on the discretized integer book
under suspension / price-limit / volume-cap constraints, 5 bps commission + 5
bps slippage.*
Here is the interpretation:
- **Naive z-score full universe**: not a useful test of the reversal idea,
because the weighting scheme lets outliers dominate the book.
- **Naive z-score full universe**: positive under open-to-open research, but a
less reliable test of the reversal idea because the weighting scheme lets
outliers dominate parts of the book.
- **Rank full universe**: a better test of the same idea, but still noisy
because the universe includes too many problematic names.
- **Rank liquid subset**: the best signal-level test; it finds a positive
- **Rank liquid subset**: the best signal-level test; it finds the cleanest
costless reversal effect.
- **Execution net**: all variants lose after cost at daily rebalance frequency,
so the implementation is not yet tradable.
- **Execution net**: daily rebalancing remains heavily constrained by cost.
A beginner might look only at the final net PnL and say "the alpha failed." A
researcher should be more precise:
@@ -407,9 +378,25 @@ researcher should be more precise:
> The raw 5-day reversal idea has signal value in a liquid universe, but the
> current daily trading rule has too much turnover for the assumed cost model.
That distinction tells you what to try next.
## Step 10: Time Consumption By Phase
## Step 10: Reproduce The Experiment
| phase | rank full (s) | rank liquid (s) |
| --- | --- | --- |
| alpha compute | 94.1 | 107.8 |
| alpha eval | 93.3 | 96.9 |
| combo combine | 21.6 | 21.7 |
| portfolio build | 537.6 | 236.7 |
| portfolio eval | 95.1 | 88.3 |
| portfolio simulate | 139.6 | 139.1 |
| total | 981.3 | 690.5 |
![Phase timings](assets/reversal_5d_phase_timings.png)
`portfolio build` usually dominates because it iterates per signal date and
repairs a multi-thousand-name integer book under lot rules. The liquid run is
faster because it carries fewer non-zero names per date.
## Step 11: Reproduce The Experiment
These commands reproduce the important artifacts, assuming the full daily-bar
dataset already exists at `data/daily_bars/all`.
@@ -423,9 +410,7 @@ uv run python cli.py alpha compute --data-path data/daily_bars/all \
# Rank-weighted full and liquid runs.
bash scripts/run_reversal_rank_e2e.sh
# Regenerate figures, diagnostics, and the older auto-generated report.
# This command rewrites this markdown file, so run it only when you want
# generated output to replace the tutorial.
# Regenerate figures, diagnostics, and this tutorial report.
uv run python scripts/generate_reversal_5d_report.py
```
@@ -452,8 +437,9 @@ Use this checklist for a new idea.
illiquid names.
5. Evaluate the alpha as a portfolio, not as a prediction.
Check cumulative return, Sharpe, drawdown, hit rate, and turnover. Do not add
IC/IR unless the framework's alpha convention changes.
Check cumulative return, Sharpe, drawdown, hit rate, and turnover over the
next-open-to-next-open holding interval. Do not add IC/IR unless the
framework's alpha convention changes.
6. Build the portfolio and inspect tracking.
Confirm that target weights match the alpha, then check whether integer
@@ -468,7 +454,7 @@ Use this checklist for a new idea.
universe, construction, execution constraints, turnover, or cost.
For this 5-day reversal study, the diagnosis is clear: **the signal-level result
is promising only after robust weighting and a liquid universe filter, but the
is most promising after robust weighting and a liquid universe filter, but the
current implementation needs turnover control before it can be considered
tradable.**