Document implemented portfolio workflow
This commit is contained in:
Yuxuan Yan
@@ -23,12 +23,15 @@ parquet, so phases run, cache, and inspect independently.
|
||||
(DATA_COLUMNS) │ │
|
||||
└──────── price ───────┴───────────────────────────────────────┘
|
||||
│
|
||||
▼ (planned — not yet implemented)
|
||||
┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ┐
|
||||
PORTFOLIO BACKTEST PAPER TRADING
|
||||
│ construct │ │ simulate │ │ forward / live │ TODO
|
||||
positions fills + costs execution
|
||||
└ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ─ ─ ┘
|
||||
▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ┐
|
||||
│ PORTFOLIO │ │ SIMULATE │ PAPER TRADING
|
||||
│ construct │──▶│ fills + costs│ │ forward / live │ TODO
|
||||
│ positions │ │ + P&L │ execution
|
||||
└──────┬───────┘ └──────────────┘ └ ─ ─ ─ ─ ─ ─ ─ ─ ┘
|
||||
▼
|
||||
portfolio/*.pq
|
||||
(POSITION_COLUMNS)
|
||||
|
||||
Each phase reads parquet and writes parquet — run, cache, and inspect
|
||||
independently. The only interface between phases is the parquet schema.
|
||||
@@ -64,8 +67,26 @@ uv run python cli.py alpha eval \
|
||||
--alpha-path alphas/reversal_5d.pq \
|
||||
--data-path "data/daily_bars/sh600000,sz000001,sh600519"
|
||||
|
||||
# 4. Build tradable integer positions from alpha or combo weights.
|
||||
uv run python cli.py portfolio build \
|
||||
--weights-path alphas/reversal_5d.pq \
|
||||
--data-path "data/daily_bars/sh600000,sz000001,sh600519" \
|
||||
--booksize 1000000 --portfolio-name reversal_port
|
||||
|
||||
# 5. Simulate next-open execution with A-share constraints, costs, and slippage.
|
||||
uv run python cli.py portfolio simulate \
|
||||
--positions-path portfolio/reversal_port.pq \
|
||||
--data-path "data/daily_bars/sh600000,sz000001,sh600519" \
|
||||
--constraint suspension --constraint price_limit --constraint volume_cap \
|
||||
--cost-bps 5 --slippage-bps 5
|
||||
|
||||
# 6. Evaluate the constructed target weights as a continuous research portfolio.
|
||||
uv run python cli.py portfolio eval \
|
||||
--positions-path portfolio/reversal_port.pq \
|
||||
--data-path "data/daily_bars/sh600000,sz000001,sh600519"
|
||||
|
||||
# Tests
|
||||
uv run python -m pytest tests/ -v # tests/test_alpha.py is network-free; test_downloader.py hits the network
|
||||
uv run python -m pytest tests/ -v # alpha/portfolio tests are network-free; downloader tests hit the network
|
||||
```
|
||||
|
||||
## CLI reference
|
||||
@@ -155,6 +176,64 @@ uv run python cli.py combo combine \
|
||||
--combo-name eq --method equal_weight
|
||||
```
|
||||
|
||||
### `portfolio build` — weights → tradable positions
|
||||
|
||||
Turns alpha/combo weights into target weights, target yuan exposure, continuous
|
||||
shares, and a lot-valid integer position book under A-share board rules.
|
||||
Non-finite / non-positive construction prices are excluded before target
|
||||
normalization. If a date has zero gross target after filtering, the previous
|
||||
book is carried in `position_shares` and a warning is logged.
|
||||
|
||||
| Option | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `--weights-path` | (required) | Alpha or combo parquet with `symbol_id, date, weight` |
|
||||
| `--data-path` | (required) | Data parquet file or partitioned dataset directory |
|
||||
| `--booksize` | (required) | Target gross yuan exposure |
|
||||
| `--portfolio-name` | (required) | Label stored in `portfolio_name` and output filename |
|
||||
| `--price-field` | `close` | Data column used as construction price |
|
||||
| `--output-dir` | `portfolio` | Output directory |
|
||||
|
||||
```bash
|
||||
uv run python cli.py portfolio build \
|
||||
--weights-path combos/eq.pq --data-path data/daily_bars/csi500 \
|
||||
--booksize 10000000 --portfolio-name eq_10m
|
||||
```
|
||||
|
||||
### `portfolio simulate` — constructed positions → fills + P&L
|
||||
|
||||
Executes the constructed `position_shares` book at the next available open,
|
||||
clipping trades through repeatable constraints. It writes `fills/<name>.pq` and
|
||||
`pnl/<name>.pq`.
|
||||
|
||||
| Option | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `--positions-path` | (required) | Positions parquet from `portfolio build` |
|
||||
| `--data-path` | (required) | Data parquet file or partitioned dataset directory |
|
||||
| `--constraint` | — | Repeatable: `suspension`, `price_limit`, `volume_cap` |
|
||||
| `--cost-bps` | `0.0` | Commission in basis points |
|
||||
| `--slippage-bps` | `0.0` | Slippage in basis points |
|
||||
| `--volume-frac` | `0.10` | Max traded value fraction for `volume_cap` |
|
||||
| `--output-dir` | `.` | Base directory for `fills/` and `pnl/` |
|
||||
|
||||
```bash
|
||||
uv run python cli.py portfolio simulate \
|
||||
--positions-path portfolio/eq_10m.pq --data-path data/daily_bars/csi500 \
|
||||
--constraint suspension --constraint price_limit --constraint volume_cap \
|
||||
--cost-bps 5 --slippage-bps 5
|
||||
```
|
||||
|
||||
### `portfolio eval` — score constructed target weights
|
||||
|
||||
```bash
|
||||
uv run python cli.py portfolio eval \
|
||||
--positions-path portfolio/eq_10m.pq --data-path data/daily_bars/csi500
|
||||
```
|
||||
|
||||
Uses `target_weight` for a continuous research view: cumulative return,
|
||||
annual Sharpe, annual turnover, max drawdown, Fitness, hit rate, and date count.
|
||||
There is deliberately **no IC/IR**. Zero-gross carry dates remain flat in this
|
||||
research view even though execution carries `position_shares`.
|
||||
|
||||
### `pqcat` — inspect a parquet file, like `cat`
|
||||
|
||||
Quickly dump any pipeline parquet (a single `.pq` file or a partitioned dataset
|
||||
@@ -284,6 +363,12 @@ between phases (data is stored long/tidy):
|
||||
baostock valuation ratios.)
|
||||
- **alpha** (`ALPHA_COLUMNS`): `symbol_id, date, alpha_name, weight`
|
||||
- **combo** (`COMBO_COLUMNS`): `symbol_id, date, combo_name, weight`
|
||||
- **portfolio positions** (`POSITION_COLUMNS`): `symbol_id, date, portfolio_name, target_weight, target_value, target_shares, position_shares, position_value, price`
|
||||
(`target_*` are continuous construction targets; `position_shares` is the
|
||||
discretized + repaired integer book used by execution.)
|
||||
- **fills** (`FILL_COLUMNS`): `symbol_id, date, portfolio_name, prev_shares, target_shares, traded_shares, realized_shares, blocked, trade_cost`
|
||||
(`date` is the execution date, i.e. the next open after the target date.)
|
||||
- **pnl** (`PNL_COLUMNS`): `date, portfolio_name, gross_exposure, net_exposure, pnl, cost, turnover, n_positions`
|
||||
|
||||
The data phase writes a month-partitioned dataset, so reading the dataset
|
||||
directory yields an extra `month` (`YYYY-MM`) partition column on top of
|
||||
@@ -291,36 +376,34 @@ directory yields an extra `month` (`YYYY-MM`) partition column on top of
|
||||
|
||||
## Layout
|
||||
|
||||
- `cli.py` — entry point wiring the three phases together
|
||||
- `cli.py` — entry point wiring the file-based phases together
|
||||
- `pipeline/data/` — universe resolution + download → `data/daily_bars/{universe}/month=YYYY-MM/*.pq`
|
||||
- `pipeline/alpha/` — `base.py` (`BaseAlpha`), `registry.py` (factory + plugin loader),
|
||||
`library/` (built-in alphas), `compute.py` (`compute_alpha` / `evaluate_alpha`)
|
||||
- `pipeline/combo/` — alpha combination → `combos/*.pq`
|
||||
- `pipeline/portfolio/` — construction, A-share lot/limit rules, constraints,
|
||||
reference next-open simulator, and research metrics
|
||||
- `pipeline/common/schema.py` — parquet column contracts
|
||||
- `data/downloader.py`, `data/universe.py` — baostock/akshare download + constituents
|
||||
- `tools/pqcat.py` — standalone parquet inspector (`pqcat`), also wired as `cli.py pqcat`
|
||||
- `tools/alphaview.py` — standalone alpha-vs-bar viewer (`alphaview`), also wired as `cli.py alphaview`
|
||||
- `examples/alphas/` — example external alpha(s)
|
||||
|
||||
## Roadmap (not yet implemented)
|
||||
## Roadmap / current limits
|
||||
|
||||
The pipeline currently ends at `combo`, and `alpha eval` only interprets a weight
|
||||
series as a portfolio for quick scoring (return / Sharpe / turnover / drawdown).
|
||||
It is **not** a true backtest — there is no transaction-cost, slippage, or
|
||||
execution modeling. The following phases are planned but not built yet:
|
||||
The pipeline is implemented through portfolio construction and a reference
|
||||
daily execution simulator. `alpha eval` remains a fast sanity check on raw
|
||||
weights; use `portfolio build`, `portfolio simulate`, and `portfolio eval` for
|
||||
constructed positions, fills/costs, P&L, and target-weight research metrics.
|
||||
|
||||
- [ ] **Portfolio construction** — turn combo weights into target positions
|
||||
(gross/net exposure caps, per-name and sector limits, capital allocation,
|
||||
rebalance schedule).
|
||||
- [ ] **Backtesting** — event-driven simulation over the constructed positions
|
||||
with realistic fills, transaction costs, slippage, and borrow constraints;
|
||||
richer P&L / risk attribution than `alpha eval`.
|
||||
- [x] **Portfolio construction** — turn alpha/combo weights into continuous
|
||||
targets and lot-valid integer positions under A-share board rules.
|
||||
- [x] **Reference execution simulation** — next-open fills over constructed
|
||||
`position_shares`, with suspension, price-limit, volume-cap, transaction-cost,
|
||||
and slippage controls.
|
||||
- [ ] **Forward / paper trading** — run the same construction logic on live
|
||||
daily data, track simulated fills and a running P&L without real capital.
|
||||
- [ ] **Intraday / microstructure data** — bid/ask prices & sizes, mid-price,
|
||||
and intraday VWAP. These need a tick / L1–L2 quote feed (typically a paid or
|
||||
brokerage data tier); the free daily sources here only expose daily bars, so
|
||||
this is a separate data phase rather than extra columns on the daily schema.
|
||||
|
||||
Until these land, treat `alpha eval` as a fast sanity check on a weight series,
|
||||
not a performance estimate.
|
||||
|
||||
Reference in New Issue
Block a user