Compare commits

...

4 Commits

Author SHA1 Message Date
Yuxuan Yan b5c8c0b8da Improve offline coverage for data boundaries 2026-06-16 17:42:20 +08:00
Yuxuan Yan 31baa18ce5 Add offline workflow and coverage tests 2026-06-16 17:37:16 +08:00
Yuxuan Yan 8d908477e2 Add daily derived data pipeline 2026-06-16 15:55:30 +08:00
Yuxuan Yan 83a006bbe4 Add minute bar feature pipeline 2026-06-16 13:57:17 +08:00
43 changed files with 4646 additions and 20 deletions
+1
View File
@@ -1,6 +1,7 @@
__pycache__/
*.py[cod]
.pytest_cache/
.coverage
*.egg-info/
.venv/
venv/
+78
View File
@@ -120,6 +120,49 @@ partitions already written. Pass the **dataset directory** (`{output_dir}/{unive
as `--data-path` to later phases — `pd.read_parquet` reads the whole partitioned
set. Symbols use the internal `sh600000` / `sz000001` form (exchange prefix + code).
### `derived` — daily custom/derived data
Derived data is daily-only v1 research data keyed by `symbol_id,date`, with one
or more numeric value columns. It can come from user CSV/parquet files or Python
plugins, and is written as a single parquet file at `derived/{name}.pq`.
The validator normalizes `date` to the trading day, requires unique
`symbol_id,date` keys, rejects duplicate columns, and rejects non-numeric value
columns. Alpha computation consumes derived data through the existing
`--feature-path` flag.
```bash
# Validate a user file without writing output.
uv run python cli.py derived validate --input-path vendor_factor.csv
# Ingest CSV/parquet into the canonical derived/ layout.
uv run python cli.py derived ingest \
--input-path vendor_factor.csv \
--derived-name vendor_factor
# List built-in and external derived-data plugin types.
uv run python cli.py derived list
uv run python cli.py derived list --derived-module path/to/my_derived.py
# Compute a derived file from daily and/or minute inputs.
uv run python cli.py derived compute \
--minute-path data/minute_bars/sh600000 \
--daily-path data/daily_bars/sh600000 \
--derived-type minute_daily_summary \
--derived-name minute_summary
# Join derived columns into a feature-aware alpha.
uv run python cli.py alpha compute \
--data-path data/daily_bars/sh600000 \
--feature-path derived/minute_summary.pq \
--alpha-type my_feature_aware_alpha \
--alpha-name my_run
```
For compatibility, `feature list` and `feature compute` remain available and
delegate to the same derived-data registry. Existing `features/*.pq` files are
still valid `--feature-path` inputs when they satisfy the daily numeric contract.
### `alpha list` — show registered alpha types
```bash
@@ -137,6 +180,7 @@ uv run python cli.py alpha list --alpha-module path/to/my_alpha.py # include a
| `--output-dir` | `alphas` | Output directory |
| `--lookback` | `5` | Lookback days (passed to alphas that accept it) |
| `--vol-window` | `20` | Volatility window (passed to alphas that accept it) |
| `--feature-path` | — | Daily derived/feature parquet file or dataset to left-join on `symbol_id,date`; repeatable |
| `--alpha-module` | — | External module(s) to import first; repeatable. Dotted path or `.py` file |
| `--param` | — | Extra constructor param as `name=value`; repeatable |
@@ -370,6 +414,8 @@ between phases (data is stored long/tidy):
OHLC scale under `qfq`/`hfq`; `turn` is turnover %, `pctChg` daily % change,
`tradestatus`/`isST` are 0/1 flags, and `peTTM`/`pbMRQ`/`psTTM`/`pcfNcfTTM` are
baostock valuation ratios.)
- **derived** (`DERIVED_KEY_COLUMNS` + values): required keys `symbol_id, date`;
value columns are user/plugin-defined and must be numeric in v1.
- **alpha** (`ALPHA_COLUMNS`): `symbol_id, date, alpha_name, weight`
- **combo** (`COMBO_COLUMNS`): `symbol_id, date, combo_name, weight`
- **portfolio positions** (`POSITION_COLUMNS`): `symbol_id, date, portfolio_name, target_weight, target_value, target_shares, position_shares, position_value, price`
@@ -387,8 +433,11 @@ directory yields an extra `month` (`YYYY-MM`) partition column on top of
- `cli.py` — entry point wiring the file-based phases together
- `pipeline/data/` — universe resolution + download → `data/daily_bars/{universe}/month=YYYY-MM/*.pq`
- `pipeline/derived/` — daily derived-data ingestion, validation, plugin registry,
and built-in derived computations → `derived/*.pq`
- `pipeline/alpha/``base.py` (`BaseAlpha`), `registry.py` (factory + plugin loader),
`library/` (built-in alphas), `compute.py` (`compute_alpha` / `evaluate_alpha`)
- `pipeline/features/` — compatibility wrappers for the derived-data registry
- `pipeline/combo/` — alpha combination → `combos/*.pq`
- `pipeline/portfolio/` — construction, A-share lot/limit rules, constraints,
reference next-open simulator, and research metrics
@@ -410,6 +459,9 @@ constructed positions, fills/costs, P&L, and target-weight research metrics.
- [x] **Reference execution simulation** — next-open fills over constructed
`position_shares`, with suspension, price-limit, volume-cap, transaction-cost,
and slippage controls.
- [x] **Derived/custom daily data ("Level 2")** — ingest user CSV/parquet files
or compute plugin outputs as validated numeric daily datasets under
`derived/{name}.pq`; alpha joins continue through `--feature-path`.
- [ ] **Optional Backtrader adapter** — Backtrader is available as the
`backtrader` extra for possible future event-driven/broker-style experiments,
but it is not part of the current canonical portfolio workflow.
@@ -419,3 +471,29 @@ constructed positions, fills/costs, P&L, and target-weight research metrics.
and intraday VWAP. These need a tick / L1L2 quote feed (typically a paid or
brokerage data tier); the free daily sources here only expose daily bars, so
this is a separate data phase rather than extra columns on the daily schema.
### Additional TODOs
The following items are intended extensions beyond the current daily
alpha-to-portfolio pipeline:
- **Long-only portfolio mode** — add a construction option that converts
alpha/combo weights into a long-only book while preserving existing lot,
price, suspension, and volume-cap handling.
- **Index-short hedging mode** — support portfolios that hold long A-share
names while shorting an index or index proxy for market exposure control.
- **Expanded universe presets** — add explicit universe aliases for CSI 300,
CSI 500, CSI 1000, and CSI 1800, while keeping file-based and comma-separated
custom universes available.
- **Categorical derived data** — extend the numeric-only derived-data v1 contract
to support categorical inputs such as industry classifications. In this
project, "Level 2" means customized second-level research data produced by
users or plugins; it does not necessarily mean exchange order-book/L2 quote
feeds.
- **Minute bar data** — continue extending the raw minute-bar and feature
workflow. The initial Baostock 5-minute download and daily feature plugin path
exist; intraday execution and replacing canonical daily bars remain out of
scope unless explicitly added later.
- **Industry data** — add industry classification inputs for filtering,
grouping, exposure reporting, neutralization, or industry-aware portfolio
construction.
+6
View File
@@ -3,7 +3,9 @@
Phases:
data — Download daily bars to parquet
derived — Ingest or compute daily derived data
alpha — Compute alpha weights from data
feature — Compute daily features from minute bars
combo — Combine alphas into a single weight
portfolio — Build tradable positions and simulate execution
"""
@@ -13,7 +15,9 @@ import logging
import click
from pipeline.data.cli import data
from pipeline.derived.cli import derived
from pipeline.alpha.cli import alpha
from pipeline.features.cli import feature
from pipeline.combo.cli import combo
from pipeline.portfolio.cli import portfolio
from tools.pqcat import pqcat
@@ -39,7 +43,9 @@ def cli(log_level):
cli.add_command(data)
cli.add_command(derived)
cli.add_command(alpha)
cli.add_command(feature)
cli.add_command(combo)
cli.add_command(portfolio)
cli.add_command(pqcat)
+158
View File
@@ -31,11 +31,68 @@ _BATCH_COLUMNS = [
"peTTM", "pbMRQ", "psTTM", "pcfNcfTTM",
]
# Raw Baostock minute bars. The ``time`` field is usually compact
# YYYYMMDDHHMMSSmmm; parsing below also tolerates HH:MM:SS strings in tests.
_MINUTE_FIELDS = "date,time,code,open,high,low,close,volume,amount,adjustflag"
_MINUTE_NUMERIC = ["open", "high", "low", "close", "volume", "amount"]
_MINUTE_COLUMNS = [
"symbol", "datetime", "date", "time", "frequency",
"open", "high", "low", "close", "volume", "amount", "vwap", "adjustflag",
]
_MINUTE_FREQUENCIES = {"5", "15", "30", "60"}
class _SessionLost(Exception):
"""baostock reported the session was dropped (``用户未登录``)."""
def _normalize_minute_frequency(frequency: str | int) -> tuple[str, str]:
"""Return Baostock frequency and partition label for a minute interval."""
raw = str(frequency).strip().lower()
if raw.endswith("m"):
raw = raw[:-1]
if raw not in _MINUTE_FREQUENCIES:
raise ValueError(
f"Unsupported minute frequency '{frequency}'. "
f"Expected one of {sorted(_MINUTE_FREQUENCIES)} minutes."
)
return raw, f"{raw}m"
def _parse_minute_datetime(date: pd.Series, time: pd.Series) -> pd.Series:
"""Parse Baostock minute timestamps into pandas datetimes."""
date_dt = pd.to_datetime(date, errors="coerce")
date_compact = date_dt.dt.strftime("%Y%m%d")
time_text = time.astype(str).str.strip()
time_digits = time_text.str.replace(r"\D", "", regex=True)
full_digits = time_digits.str.slice(0, 14)
from_full = pd.to_datetime(full_digits, format="%Y%m%d%H%M%S", errors="coerce")
from_short = pd.Series(pd.NaT, index=date.index, dtype="datetime64[ns]")
short_time = time_digits.str.len().between(1, 6)
if short_time.any():
short_digits = (
time_digits.loc[short_time]
.str.pad(6, side="right", fillchar="0")
.str.slice(0, 6)
)
from_short.loc[short_time] = pd.to_datetime(
date_compact.loc[short_time] + short_digits,
format="%Y%m%d%H%M%S",
errors="coerce",
)
from_text = pd.Series(pd.NaT, index=date.index, dtype="datetime64[ns]")
text_time = time_text.str.contains(":", regex=False)
if text_time.any():
from_text.loc[text_time] = pd.to_datetime(
date.astype(str).loc[text_time] + " " + time_text.loc[text_time],
errors="coerce",
)
return from_full.fillna(from_short).fillna(from_text)
def _download_akshare(symbol: str, start: str, end: str, adjust: str = "qfq") -> Optional[pd.DataFrame]:
"""Download daily bars from akshare. Returns DataFrame with OHLCV columns."""
try:
@@ -239,3 +296,104 @@ def download_daily_batch(
except Exception:
pass
def download_minute_batch(
symbols: Iterable[str],
start: str,
end: str,
frequency: str | int = 5,
relogin_every: int = 200,
) -> Iterator[Tuple[str, Optional[pd.DataFrame]]]:
"""Download raw Baostock minute bars for many symbols.
Minute bars are intentionally unadjusted (`adjustflag='3'`) because the
output is raw intraday market data for downstream feature aggregation, not a
tradable daily price series.
Args:
symbols: Internal-form symbols (``sh600000`` / ``sz000001``).
start, end: ``YYYY-MM-DD`` bounds.
frequency: Baostock minute frequency. ``5``/``"5"``/``"5m"`` all mean
5-minute bars.
relogin_every: Proactively refresh the baostock session every N symbols.
Yields:
``(symbol, df)`` where ``df`` has raw minute bars or ``None`` when no
data is available.
"""
query_frequency, frequency_label = _normalize_minute_frequency(frequency)
adjustflag = _BAOSTOCK_ADJUST["none"]
def _relogin() -> None:
try:
bs.logout()
except Exception:
pass
bs.login()
def _fetch(symbol: str) -> Optional[pd.DataFrame]:
"""One Baostock minute query; returns df, None, or raises _SessionLost."""
code = f"{symbol[:2]}.{symbol[2:]}"
rs = bs.query_history_k_data_plus(
code=code,
fields=_MINUTE_FIELDS,
start_date=start,
end_date=end,
frequency=query_frequency,
adjustflag=adjustflag,
)
if rs.error_code != "0":
if "未登录" in (rs.error_msg or ""):
raise _SessionLost(rs.error_msg)
logger.warning("baostock minute error for %s: %s", symbol, rs.error_msg)
return None
rows = []
while rs.next():
rows.append(rs.get_row_data())
if not rows:
return None
df = pd.DataFrame(rows, columns=_MINUTE_FIELDS.split(","))
df[_MINUTE_NUMERIC] = df[_MINUTE_NUMERIC].apply(pd.to_numeric, errors="coerce")
df["datetime"] = _parse_minute_datetime(df["date"], df["time"])
bad_timestamps = df["datetime"].isna()
if bad_timestamps.any():
raise ValueError(
f"Could not parse {int(bad_timestamps.sum())} minute timestamp(s)"
)
df["date"] = pd.to_datetime(df["date"], errors="coerce")
df["time"] = df["datetime"].dt.strftime("%H:%M:%S")
df["frequency"] = frequency_label
df["vwap"] = (df["amount"] / df["volume"]).where(df["volume"] > 0)
df["symbol"] = symbol
return df[_MINUTE_COLUMNS].sort_values("datetime").reset_index(drop=True)
bs.login()
try:
for i, symbol in enumerate(symbols):
if i and relogin_every and i % relogin_every == 0:
_relogin()
df: Optional[pd.DataFrame] = None
for attempt in (1, 2):
try:
df = _fetch(symbol)
break
except _SessionLost:
if attempt == 1:
_relogin()
continue
logger.warning("baostock minute session lost for %s after relogin", symbol)
except Exception as e:
logger.warning("baostock minute download failed for %s: %s", symbol, e)
break
if df is not None and not df.empty:
yield symbol, df
else:
yield symbol, None
finally:
try:
bs.logout()
except Exception:
pass
+70
View File
@@ -0,0 +1,70 @@
# Minute Bar Data Notes
The minute-bar path downloads raw Baostock intraday bars and stores them as a
Hive-partitioned dataset:
```bash
uv run python cli.py data download-minute \
--universe sh600000 \
--start-date 2024-01-02 --end-date 2024-01-05 \
--frequency 5
```
The default layout is:
```text
data/minute_bars/{universe}/frequency=5m/month=YYYY-MM/*.pq
```
Derived-data plugins can aggregate those bars to daily `symbol_id,date` numeric
files, for example:
```bash
uv run python cli.py derived compute \
--minute-path data/minute_bars/sh600000 \
--daily-path data/daily_bars/sh600000 \
--derived-type minute_daily_summary \
--derived-name minute_summary
```
The legacy `feature compute` command delegates to the same derived-data
registry and remains available for existing scripts.
## Daily vs Minute Reconciliation
Baostock's daily raw bars and 5-minute raw bars are close, but they should not
be treated as perfectly reconstructible from each other.
When checking consistency, compare daily raw bars (`data download --adjust none`)
against minute bars on the same raw price scale. The minute aggregation should
use:
- `open`: first minute open
- `high`: max minute high
- `low`: min minute low
- `close`: last minute close
- `volume`: sum minute volume
- `amount`: sum minute amount
- `vwap`: `sum(amount) / sum(volume)`
In a sanity check for `sh600000` from `2024-01-02` through `2024-01-05`, Baostock
returned 4 daily rows and 192 5-minute bars, exactly 48 bars per day. Open, low,
and close matched daily exactly on all 4 days. High matched on 3 of 4 days; on
`2024-01-04`, the daily high was `6.67` while the max 5-minute high was `6.66`.
Minute-summed volume and amount were higher than daily by roughly `0.16%` to
`1.23%`. VWAP remained very close, with max relative difference around
`0.0043%`.
This appears to be a source-level Baostock reconciliation caveat, not a parser
or ordering issue: the minute bars covered the regular `09:35:00` through
`15:00:00` range and sorted correctly by timestamp.
Practical guidance:
- Use tolerance-based daily-vs-minute checks; do not require exact equality for
high, volume, or amount.
- Expect open/close alignment to be a stronger sanity check than exact volume
reconstruction.
- Use minute-derived values as separate daily features, not as replacements for
the canonical daily bar dataset unless a strategy explicitly wants that
source convention.
+19 -5
View File
@@ -5,7 +5,7 @@ position weights. Subclasses implement :meth:`signal` — the raw, unnormalized
score. The base class turns a signal into cross-sectionally z-scored weights
via :meth:`to_weights` (override it for a different normalization).
"""
from abc import ABC, abstractmethod
from abc import ABC
import numpy as np
import pandas as pd
@@ -15,15 +15,14 @@ class BaseAlpha(ABC):
"""A position-weight alpha over a cross-section of stocks.
Concrete subclasses must set a unique class-level :attr:`name` (the registry
key) and implement :meth:`signal`. Construct subclasses with their own typed
parameters (e.g. ``lookback``); the factory passes only the parameters a
given ``__init__`` accepts.
key) and implement either :meth:`signal` or :meth:`signal_from_data`.
Construct subclasses with their own typed parameters (e.g. ``lookback``);
the factory passes only the parameters a given ``__init__`` accepts.
"""
#: Unique registry key. Every concrete alpha must set this to a non-empty str.
name: str = ""
@abstractmethod
def signal(self, close: pd.DataFrame) -> pd.DataFrame:
"""Compute the raw signal.
@@ -34,6 +33,21 @@ class BaseAlpha(ABC):
A wide DataFrame aligned to ``close`` where higher values indicate a
stronger long. Use NaN where the signal is undefined.
"""
raise NotImplementedError(
f"{type(self).__name__} must implement signal() or signal_from_data()"
)
def signal_from_data(
self,
data: pd.DataFrame,
close: pd.DataFrame,
) -> pd.DataFrame:
"""Compute the raw signal from long daily data plus wide closes.
Feature-aware alphas can override this to pivot joined feature columns
from ``data``. The default preserves the existing close-only alpha API.
"""
return self.signal(close)
def to_weights(self, signal: pd.DataFrame) -> pd.DataFrame:
"""Cross-sectionally z-score a signal into signed position weights.
+10 -4
View File
@@ -56,6 +56,10 @@ def list_(alpha_modules):
@click.option("--output-dir", default="alphas", help="Directory to save alpha parquet")
@click.option("--lookback", default=5, type=int, help="Lookback days")
@click.option("--vol-window", default=20, type=int, help="Volatility window (reversal_vol only)")
@click.option(
"--feature-path", "feature_paths", multiple=True,
help="Daily derived/feature parquet file or dataset to left-join on symbol_id,date (repeatable)",
)
@click.option(
"--alpha-module", "alpha_modules", multiple=True,
help="External module(s) to import so their alphas register (dotted path or .py file)",
@@ -74,7 +78,7 @@ def list_(alpha_modules):
help="Most-liquid names kept per date when --liquid-universe is set",
)
def compute(data_path, alpha_name, alpha_type, output_dir, lookback, vol_window,
alpha_modules, extra_params, liquid_universe, universe_top_n):
feature_paths, alpha_modules, extra_params, liquid_universe, universe_top_n):
"""Compute one alpha from raw data and save as parquet."""
for spec in alpha_modules:
load_alpha_module(spec)
@@ -100,6 +104,7 @@ def compute(data_path, alpha_name, alpha_type, output_dir, lookback, vol_window,
alpha_name=alpha_name,
alpha_type=alpha_type,
universe=universe,
feature_paths=feature_paths,
**params,
)
@@ -155,7 +160,8 @@ def reversal_vol(data_path, output_dir, lookback, vol_window):
@alpha.command("eval")
@click.option("--alpha-path", required=True, help="Path to alpha parquet file")
@click.option("--data-path", required=True, help="Path to data parquet (for price data)")
def eval_(alpha_path, data_path):
@click.option("--report-dir", default="reports", help="Directory to save JSON report")
def eval_(alpha_path, data_path, report_dir):
"""Evaluate an alpha's performance (return, Sharpe, turnover).
Alphas are interpreted as position WEIGHTS, not return predictors.
@@ -178,9 +184,9 @@ def eval_(alpha_path, data_path):
click.echo("=" * 50)
# Also dump JSON
os.makedirs("reports", exist_ok=True)
os.makedirs(report_dir, exist_ok=True)
alpha_name = alpha_df["alpha_name"].iloc[0]
json_path = f"reports/{alpha_name}_eval.json"
json_path = os.path.join(report_dir, f"{alpha_name}_eval.json")
with open(json_path, "w") as f:
json.dump(metrics, f, indent=2)
click.echo(f"\nReport saved: {json_path}")
+55 -2
View File
@@ -7,12 +7,19 @@ through :mod:`pipeline.alpha.registry`.
"""
import logging
from pathlib import Path
from typing import Iterable
import numpy as np
import pandas as pd
from pipeline.alpha.registry import get_alpha
from pipeline.common.schema import ALPHA_COLUMNS
from pipeline.derived.compute import (
DERIVED_KEY_COLUMNS,
read_derived_frames,
validate_derived_frame,
)
logger = logging.getLogger(__name__)
@@ -33,6 +40,38 @@ def _pivot_open(df: pd.DataFrame) -> pd.DataFrame:
return pivot.sort_index()
def join_feature_frames(
data: pd.DataFrame,
feature_frames: Iterable[pd.DataFrame],
) -> pd.DataFrame:
"""Left-join validated daily derived/feature frames onto long daily data."""
out = data.copy()
out["date"] = pd.to_datetime(out["date"])
existing = set(out.columns)
joined_cols: list[str] = []
for frame in feature_frames:
features = validate_derived_frame(frame)
feature_cols = [col for col in features.columns if col not in DERIVED_KEY_COLUMNS]
overlap = sorted(existing.intersection(feature_cols))
if overlap:
raise ValueError(
f"Feature columns conflict with existing daily data columns: {overlap}"
)
out = out.merge(
features,
on=DERIVED_KEY_COLUMNS,
how="left",
validate="many_to_one",
)
existing.update(feature_cols)
joined_cols.extend(feature_cols)
if joined_cols:
logger.info("Joined feature columns into daily data: %s", joined_cols)
return out
def _forward_open_to_open_returns(open_: pd.DataFrame) -> pd.DataFrame:
"""Return earned by a close-formed signal after next-open execution.
@@ -105,6 +144,8 @@ def compute_alpha(
alpha_name: str,
alpha_type: str,
universe: dict | None = None,
feature_paths: Iterable[str | Path] | None = None,
feature_frames: Iterable[pd.DataFrame] | None = None,
**params,
) -> pd.DataFrame:
"""Compute alpha weights from raw data.
@@ -118,6 +159,10 @@ def compute_alpha(
:func:`investable_universe_mask`) *before* it is turned into
weights, so unheld names get weight 0. Keys are forwarded as keyword
arguments to :func:`investable_universe_mask`.
feature_paths: Optional parquet files/datasets keyed by ``symbol_id``
and ``date``. Their numeric feature columns are left-joined onto
``data`` before alpha logic runs.
feature_frames: Optional in-memory feature frames with the same schema.
**params: Constructor parameters for the alpha (e.g. ``lookback``,
``vol_window``). Only the params the alpha's ``__init__`` accepts are
used; extras are ignored.
@@ -128,12 +173,20 @@ def compute_alpha(
Raises:
KeyError: If ``alpha_type`` is not registered.
"""
feature_inputs: list[pd.DataFrame] = []
if feature_paths:
feature_inputs.extend(read_derived_frames(feature_paths))
if feature_frames:
feature_inputs.extend(feature_frames)
if feature_inputs:
data = join_feature_frames(data, feature_inputs)
alpha = get_alpha(alpha_type, **params)
close = _pivot_close(data)
signal = alpha.signal_from_data(data, close)
if universe is None:
weights = alpha.weights(close)
weights = alpha.to_weights(signal)
else:
signal = alpha.signal(close)
mask = investable_universe_mask(data, signal, **universe)
weights = alpha.to_weights(signal.where(mask))
+25
View File
@@ -26,6 +26,31 @@ DATA_COLUMNS: Final[list[str]] = [
"pcfNcfTTM", # float64: P/CF (net cash flow, TTM)
]
# Required columns for raw intraday minute bar parquet files.
MINUTE_BAR_COLUMNS: Final[list[str]] = [
"symbol_id", # str: internal code like 'sh600000'
"symbol_name", # str: stock name like '浦发银行'
"datetime", # datetime64: intraday bar timestamp
"date", # date component, aligned with daily DATA_COLUMNS date
"time", # str: HH:MM:SS bar time
"frequency", # str: e.g. '5m'
"open", # float64
"high", # float64
"low", # float64
"close", # float64
"volume", # float64 (shares)
"amount", # float64 (turnover in yuan, raw/unadjusted)
"vwap", # float64: amount / volume
"adjustflag", # str: baostock adjustment flag; '3' for raw/unadjusted
]
# Required key columns for daily derived-data parquet files. Value columns are
# user/plugin-defined and must be numeric.
DERIVED_KEY_COLUMNS: Final[list[str]] = [
"symbol_id", # str
"date", # date: normalized daily timestamp
]
# Required columns for alpha parquet files.
# Alphas are position WEIGHTS: positive=long, negative=short.
ALPHA_COLUMNS: Final[list[str]] = [
+35 -1
View File
@@ -3,7 +3,7 @@
import click
from datetime import date
from pipeline.data.downloader import download_universe
from pipeline.data.downloader import download_minute_universe, download_universe
@click.group(name="data")
@@ -42,3 +42,37 @@ def download(universe, start_date, end_date, output_dir, symbols, chunk_size, ad
f"{stats['n_rows']:,} bars, {stats['date_min']}{stats['date_max']}"
)
click.echo(f"Dataset: {stats['dataset_path']}")
@data.command("download-minute")
@click.option(
"--universe", default="csi500",
help="Which universe: hs300, csi500, all (~5000 A-shares), file path, or comma-separated symbols",
)
@click.option("--start-date", default="2017-01-01", help="Start date YYYY-MM-DD")
@click.option("--end-date", default=str(date.today()), help="End date YYYY-MM-DD")
@click.option("--output-dir", default="data/minute_bars", help="Root for the partitioned dataset")
@click.option("--symbols", default=0, type=int, help="Max symbols (0=all)")
@click.option("--chunk-size", default=100, type=int, help="Symbols per durability flush")
@click.option("--frequency", default="5", help="Minute frequency: 5, 15, 30, or 60")
def download_minute(universe, start_date, end_date, output_dir, symbols, chunk_size, frequency):
"""Download raw Baostock minute bars into a partitioned parquet dataset.
Writes ``{output_dir}/{universe}/frequency=5m/month=YYYY-MM/*.pq`` for the
default 5-minute frequency.
"""
stats = download_minute_universe(
universe=universe,
start_date=start_date,
end_date=end_date,
output_dir=output_dir,
max_symbols=symbols,
chunk_size=chunk_size,
frequency=frequency,
)
click.echo(
f"\nSummary: {stats['n_symbols']}/{stats['n_requested']} symbols, "
f"{stats['n_rows']:,} bars, {stats['date_min']}{stats['date_max']}, "
f"frequency={stats['frequency']}"
)
click.echo(f"Dataset: {stats['dataset_path']}")
+126 -2
View File
@@ -11,9 +11,13 @@ import pyarrow.dataset as pads
# Reuse existing downloader and universe modules
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from data.downloader import download_daily_batch
from data.downloader import (
_normalize_minute_frequency,
download_daily_batch,
download_minute_batch,
)
from data.universe import get_all_stocks, get_hs300_stocks, get_zz500_stocks
from pipeline.common.schema import DATA_COLUMNS
from pipeline.common.schema import DATA_COLUMNS, MINUTE_BAR_COLUMNS
logger = logging.getLogger(__name__)
@@ -89,6 +93,25 @@ def _write_month_partitions(df: pd.DataFrame, base_dir: Path, basename_prefix: s
)
def _write_minute_partitions(df: pd.DataFrame, base_dir: Path, basename_prefix: str) -> None:
"""Append rows to a Hive-partitioned minute dataset.
Layout: ``frequency=5m/month=YYYY-MM/*.pq``.
"""
out = df.copy()
out["month"] = pd.to_datetime(out["date"]).dt.strftime("%Y-%m")
table = pa.Table.from_pandas(out, preserve_index=False)
pads.write_dataset(
table,
str(base_dir),
format="parquet",
partitioning=["frequency", "month"],
partitioning_flavor="hive",
basename_template=f"{basename_prefix}-{{i}}.pq",
existing_data_behavior="overwrite_or_ignore",
)
def download_universe(
universe: str = "csi500",
start_date: str = "2017-01-01",
@@ -177,3 +200,104 @@ def download_universe(
"date_min": None if date_min is None else str(pd.Timestamp(date_min).date()),
"date_max": None if date_max is None else str(pd.Timestamp(date_max).date()),
}
def download_minute_universe(
universe: str = "csi500",
start_date: str = "2017-01-01",
end_date: str = "2026-12-31",
output_dir: str = "data/minute_bars",
max_symbols: int = 0,
chunk_size: int = 100,
frequency: str | int = 5,
) -> dict:
"""Download raw minute bars into a frequency/month-partitioned dataset.
Args:
universe: ``hs300``, ``csi500``, ``all``/``full``, a file path, or a
comma-separated symbol list.
start_date, end_date: ``YYYY-MM-DD`` bounds.
output_dir: Root under which ``{universe}/frequency=5m/month=YYYY-MM``
is written.
max_symbols: Cap on symbols (0 = all).
chunk_size: Symbols per durability flush.
frequency: Minute interval. ``5``/``"5"``/``"5m"`` are 5-minute bars.
Returns:
Stats dict with dataset path, row count, symbol count, date range, and
frequency label.
"""
_, frequency_label = _normalize_minute_frequency(frequency)
constituents = _resolve_universe(universe, max_symbols)
symbols = constituents["symbol_id"].tolist()
names = dict(zip(constituents["symbol_id"], constituents["symbol_name"]))
n_requested = len(symbols)
logger.info(
"Minute universe %s: %d symbols, %s%s, frequency=%s",
universe,
n_requested,
start_date,
end_date,
frequency,
)
base_dir = Path(output_dir) / universe
target_frequency_dir = base_dir / f"frequency={frequency_label}"
if target_frequency_dir.exists():
shutil.rmtree(target_frequency_dir)
base_dir.mkdir(parents=True, exist_ok=True)
buffer: list[pd.DataFrame] = []
chunk_idx = 0
succeeded = 0
n_rows = 0
date_min = None
date_max = None
def flush() -> None:
nonlocal buffer, chunk_idx, n_rows, date_min, date_max
if not buffer:
return
chunk = pd.concat(buffer, ignore_index=True)
_write_minute_partitions(chunk, base_dir, basename_prefix=f"chunk{chunk_idx:04d}")
n_rows += len(chunk)
cmin, cmax = chunk["date"].min(), chunk["date"].max()
date_min = cmin if date_min is None else min(date_min, cmin)
date_max = cmax if date_max is None else max(date_max, cmax)
logger.info(
"Flushed minute chunk %d: %d rows (%d symbols done)",
chunk_idx,
len(chunk),
succeeded,
)
buffer = []
chunk_idx += 1
for i, (symbol, df) in enumerate(
download_minute_batch(symbols, start_date, end_date, frequency=frequency), start=1
):
if df is None:
logger.warning(" %s: no minute data", symbol)
else:
df["symbol_id"] = symbol
df["symbol_name"] = names.get(symbol, symbol)
buffer.append(df[MINUTE_BAR_COLUMNS])
succeeded += 1
if len(buffer) >= chunk_size:
flush()
if i % 100 == 0:
logger.info("Minute progress: %d/%d symbols", i, n_requested)
flush()
if succeeded == 0:
raise RuntimeError("No minute data downloaded for any symbol")
return {
"dataset_path": str(base_dir),
"frequency": frequency_label,
"n_symbols": succeeded,
"n_requested": n_requested,
"n_rows": n_rows,
"date_min": None if date_min is None else str(pd.Timestamp(date_min).date()),
"date_max": None if date_max is None else str(pd.Timestamp(date_max).date()),
}
+2
View File
@@ -0,0 +1,2 @@
"""Daily derived-data plugin package."""
+38
View File
@@ -0,0 +1,38 @@
"""Base class for daily derived-data plugins."""
from abc import ABC, abstractmethod
import pandas as pd
class BaseDerivedData(ABC):
"""Compute daily, symbol-keyed numeric derived data.
Derived-data plugins may use daily bars, minute bars, or both as inputs, but
they must always return daily rows keyed by ``symbol_id,date``.
"""
#: Unique registry key. Every concrete derived-data plugin must set this.
name: str = ""
@abstractmethod
def compute(
self,
daily: pd.DataFrame | None = None,
minute: pd.DataFrame | None = None,
) -> pd.DataFrame:
"""Compute daily derived data.
Args:
daily: Optional daily market data.
minute: Optional raw minute bars.
Returns:
DataFrame with ``symbol_id``, ``date``, and one or more numeric
derived-data columns.
"""
def __repr__(self) -> str:
params = ", ".join(f"{k}={v!r}" for k, v in vars(self).items())
return f"{type(self).__name__}({params})"
+145
View File
@@ -0,0 +1,145 @@
"""CLI for daily derived-data ingestion and computation."""
import click
import pandas as pd
from pipeline.derived.compute import (
DERIVED_KEY_COLUMNS,
compute_derived,
read_derived_frame,
write_derived_frame,
)
from pipeline.derived.registry import (
available_derived,
load_derived_module,
)
@click.group(name="derived")
def derived():
"""Ingest, compute, and validate daily derived data."""
def _coerce(value: str):
"""Best-effort coercion of a CLI string to int, then float, else str."""
for cast in (int, float):
try:
return cast(value)
except ValueError:
continue
return value
def _parse_params(pairs: tuple[str, ...]) -> dict:
"""Parse repeated ``name=value`` options into a params dict."""
params: dict = {}
for pair in pairs:
if "=" not in pair:
raise click.BadParameter(f"--param must be name=value, got '{pair}'")
key, value = pair.split("=", 1)
params[key.strip()] = _coerce(value.strip())
return params
def _read_optional_parquet(path: str | None) -> pd.DataFrame | None:
return None if path is None else pd.read_parquet(path)
def _summarize(result: pd.DataFrame) -> str:
value_cols = [col for col in result.columns if col not in DERIVED_KEY_COLUMNS]
return f"{len(result):,} rows, {len(value_cols)} columns"
@derived.command("list")
@click.option(
"--derived-module", "derived_modules", multiple=True,
help="External module(s) to import first (dotted path or .py file)",
)
def list_(derived_modules):
"""List the registered derived-data plugin types."""
for spec in derived_modules:
load_derived_module(spec)
for name in available_derived():
click.echo(name)
@derived.command("validate")
@click.option("--input-path", required=True, help="CSV/parquet file or parquet dataset to validate")
def validate(input_path):
"""Validate a daily derived-data file without writing output."""
try:
result = read_derived_frame(input_path)
except Exception as exc:
raise click.ClickException(str(exc)) from exc
click.echo(f"Valid derived data: {input_path} ({_summarize(result)})")
@derived.command("ingest")
@click.option("--input-path", required=True, help="CSV/parquet file to ingest")
@click.option("--derived-name", required=True, help="Name for this derived-data output file")
@click.option("--output-dir", default="derived", help="Directory to save derived parquet")
def ingest(input_path, derived_name, output_dir):
"""Ingest a user-provided daily derived-data CSV/parquet file."""
try:
result = read_derived_frame(input_path)
out_path = write_derived_frame(result, derived_name, output_dir=output_dir)
except Exception as exc:
raise click.ClickException(str(exc)) from exc
click.echo(f"Saved derived data: {out_path} ({_summarize(result)})")
@derived.command("compute")
@click.option("--daily-path", default=None, help="Optional daily data parquet/dataset")
@click.option("--minute-path", default=None, help="Optional minute parquet/dataset")
@click.option("--derived-type", required=True, help="Registry key of the derived-data plugin")
@click.option("--derived-name", required=True, help="Name for this derived-data output file")
@click.option("--output-dir", default="derived", help="Directory to save derived parquet")
@click.option(
"--derived-module", "derived_modules", multiple=True,
help="External module(s) to import so their derived-data plugins register",
)
@click.option(
"--param", "extra_params", multiple=True,
help="Extra derived-data constructor param as name=value (repeatable)",
)
def compute(
daily_path,
minute_path,
derived_type,
derived_name,
output_dir,
derived_modules,
extra_params,
):
"""Compute one daily derived-data file from daily and/or minute inputs."""
for spec in derived_modules:
load_derived_module(spec)
options = available_derived()
if derived_type not in options:
raise click.BadParameter(
f"Unknown derived-type '{derived_type}'. Available: {options}. "
f"Use --derived-module to register an external derived-data plugin.",
param_hint="--derived-type",
)
if daily_path is None and minute_path is None:
raise click.UsageError("At least one of --daily-path or --minute-path is required")
daily = _read_optional_parquet(daily_path)
if daily_path:
click.echo(f"Loaded daily data: {len(daily):,} rows from {daily_path}")
minute = _read_optional_parquet(minute_path)
if minute_path:
click.echo(f"Loaded minute bars: {len(minute):,} rows from {minute_path}")
try:
result = compute_derived(
derived_type=derived_type,
daily=daily,
minute=minute,
**_parse_params(extra_params),
)
out_path = write_derived_frame(result, derived_name, output_dir=output_dir)
except Exception as exc:
raise click.ClickException(str(exc)) from exc
click.echo(f"Saved derived data: {out_path} ({_summarize(result)})")
+115
View File
@@ -0,0 +1,115 @@
"""Derived-data computation and validation."""
import csv
import logging
from pathlib import Path
from typing import Iterable
import pandas as pd
from pandas.api.types import is_bool_dtype, is_numeric_dtype
from pipeline.common.schema import DERIVED_KEY_COLUMNS
from pipeline.derived.registry import get_derived
logger = logging.getLogger(__name__)
def validate_derived_frame(derived: pd.DataFrame) -> pd.DataFrame:
"""Validate and normalize a daily derived-data frame.
A valid derived frame is keyed by unique ``symbol_id,date`` rows and has at
least one numeric value column beyond those keys. Dates are normalized to
daily timestamps before duplicate-key checks.
"""
duplicated = derived.columns[derived.columns.duplicated()].tolist()
if duplicated:
raise ValueError(f"Derived data has duplicate columns: {duplicated}")
missing = [col for col in DERIVED_KEY_COLUMNS if col not in derived.columns]
if missing:
raise ValueError(f"Derived data missing required columns: {missing}")
out = derived.copy()
out["date"] = pd.to_datetime(out["date"]).dt.normalize()
if out.duplicated(DERIVED_KEY_COLUMNS).any():
raise ValueError("Derived data has duplicate symbol_id,date rows")
value_cols = [col for col in out.columns if col not in DERIVED_KEY_COLUMNS]
if not value_cols:
raise ValueError("Derived data must include at least one value column")
non_numeric = [
col
for col in value_cols
if is_bool_dtype(out[col]) or not is_numeric_dtype(out[col])
]
if non_numeric:
raise ValueError(f"Derived data value columns must be numeric: {non_numeric}")
out = out[DERIVED_KEY_COLUMNS + value_cols].copy()
return out.sort_values(DERIVED_KEY_COLUMNS).reset_index(drop=True)
def compute_derived(
derived_type: str,
daily: pd.DataFrame | None = None,
minute: pd.DataFrame | None = None,
**params,
) -> pd.DataFrame:
"""Compute one registered derived-data plugin."""
if daily is None and minute is None:
raise ValueError("Derived data computation requires --daily-path or --minute-path")
derived = get_derived(derived_type, **params)
result = validate_derived_frame(derived.compute(daily=daily, minute=minute))
value_cols = [col for col in result.columns if col not in DERIVED_KEY_COLUMNS]
logger.info(
"Derived data '%s' (%r): %d symbols × %d dates, columns=%s",
derived_type,
derived,
result["symbol_id"].nunique(),
result["date"].nunique(),
value_cols,
)
return result
def read_derived_frame(path: str | Path) -> pd.DataFrame:
"""Read and validate one derived CSV/parquet file or parquet dataset."""
path = Path(path)
if path.suffix.lower() == ".csv":
return validate_derived_frame(_read_csv_with_duplicate_header_check(path))
return validate_derived_frame(pd.read_parquet(path))
def read_derived_frames(derived_paths: Iterable[str | Path]) -> list[pd.DataFrame]:
"""Read and validate derived-data files."""
return [read_derived_frame(path) for path in derived_paths]
def write_derived_frame(
derived: pd.DataFrame,
derived_name: str,
output_dir: str | Path = "derived",
) -> Path:
"""Validate and write derived data to ``{output_dir}/{derived_name}.pq``."""
result = validate_derived_frame(derived)
output_dir = Path(output_dir)
output_dir.mkdir(parents=True, exist_ok=True)
out_path = output_dir / f"{derived_name}.pq"
result.to_parquet(out_path, index=False)
return out_path
def _read_csv_with_duplicate_header_check(path: Path) -> pd.DataFrame:
with path.open(newline="") as fh:
reader = csv.reader(fh)
try:
header = next(reader)
except StopIteration as exc:
raise ValueError("CSV input is empty") from exc
duplicated = sorted({col for col in header if header.count(col) > 1})
if duplicated:
raise ValueError(f"Derived data has duplicate columns: {duplicated}")
return pd.read_csv(path)
+4
View File
@@ -0,0 +1,4 @@
"""Built-in derived-data library."""
from pipeline.derived.library import minute_daily_summary # noqa: F401
@@ -0,0 +1,88 @@
"""Daily summary data derived from raw minute bars."""
import numpy as np
import pandas as pd
from pipeline.derived.base import BaseDerivedData
from pipeline.derived.registry import register_derived
@register_derived
class MinuteDailySummaryDerived(BaseDerivedData):
"""Aggregate intraday bars into daily summary columns."""
name = "minute_daily_summary"
def compute(
self,
daily: pd.DataFrame | None = None,
minute: pd.DataFrame | None = None,
) -> pd.DataFrame:
if minute is None:
raise ValueError("minute_daily_summary requires minute input")
minute = minute.copy()
minute["date"] = pd.to_datetime(minute["date"]).dt.normalize()
sort_cols = ["symbol_id", "date"]
if "datetime" in minute.columns:
minute["datetime"] = pd.to_datetime(minute["datetime"])
sort_cols.append("datetime")
elif "time" in minute.columns:
sort_cols.append("time")
minute = minute.sort_values(sort_cols)
grouped = minute.groupby(["symbol_id", "date"], sort=True)
summary = grouped.agg(
minute_bar_count=("close", "count"),
first_open=("open", "first"),
last_close=("close", "last"),
high=("high", "max"),
low=("low", "min"),
volume_sum=("volume", "sum"),
amount_sum=("amount", "sum"),
)
summary["minute_intraday_return"] = (
summary["last_close"] / summary["first_open"] - 1.0
)
summary["minute_intraday_range"] = summary["high"] / summary["low"] - 1.0
summary["minute_vwap"] = (
summary["amount_sum"] / summary["volume_sum"].where(summary["volume_sum"] > 0)
)
summary = summary.reset_index()
if daily is not None:
daily_keys = daily[["symbol_id", "date"]].copy()
daily_keys["date"] = pd.to_datetime(daily_keys["date"]).dt.normalize()
daily_keys = daily_keys.drop_duplicates(["symbol_id", "date"])
result = daily_keys.merge(summary, on=["symbol_id", "date"], how="left")
if "close" in daily.columns:
daily_close = daily[["symbol_id", "date", "close"]].copy()
daily_close["date"] = pd.to_datetime(daily_close["date"]).dt.normalize()
daily_close = daily_close.drop_duplicates(["symbol_id", "date"])
result = result.merge(
daily_close.rename(columns={"close": "daily_close"}),
on=["symbol_id", "date"],
how="left",
)
reference_close = result["daily_close"].fillna(result["last_close"])
else:
reference_close = result["last_close"]
else:
result = summary
reference_close = result["last_close"]
result["minute_vwap_deviation"] = (
result["minute_vwap"] / reference_close.replace(0.0, np.nan) - 1.0
)
return result[
[
"symbol_id",
"date",
"minute_bar_count",
"minute_intraday_return",
"minute_intraday_range",
"minute_vwap",
"minute_vwap_deviation",
]
]
+80
View File
@@ -0,0 +1,80 @@
"""Registry and factory for daily derived-data plugins."""
import importlib
import importlib.util
import inspect
from pathlib import Path
from typing import Optional, Type
from pipeline.derived.base import BaseDerivedData
_REGISTRY: dict[str, Type[BaseDerivedData]] = {}
_builtins_loaded = False
def register_derived(cls: Type[BaseDerivedData]) -> Type[BaseDerivedData]:
"""Class decorator that registers derived data under ``BaseDerivedData.name``."""
if not (isinstance(cls, type) and issubclass(cls, BaseDerivedData)):
raise TypeError(f"{cls!r} is not a BaseDerivedData subclass")
key = getattr(cls, "name", "")
if not key:
raise ValueError(f"{cls.__name__} must set a non-empty class attribute `name`")
existing = _REGISTRY.get(key)
if existing is not None and existing is not cls:
raise ValueError(
f"Derived data name '{key}' already registered by {existing.__name__}"
)
_REGISTRY[key] = cls
return cls
def available_derived() -> list[str]:
"""Sorted names of all registered derived-data plugins."""
_ensure_builtins()
return sorted(_REGISTRY)
def get_derived(name: str, **params) -> BaseDerivedData:
"""Instantiate a registered derived-data plugin by name.
Only parameters accepted by the plugin class's ``__init__`` are forwarded.
"""
_ensure_builtins()
if name not in _REGISTRY:
raise KeyError(f"Unknown derived data '{name}'. Available: {sorted(_REGISTRY)}")
cls = _REGISTRY[name]
accepted = _accepted_params(cls)
kwargs = params if accepted is None else {k: v for k, v in params.items() if k in accepted}
return cls(**kwargs)
def load_derived_module(spec: str) -> None:
"""Import an external module so its ``@register_derived`` classes register."""
looks_like_file = spec.endswith(".py") or Path(spec).expanduser().exists()
if looks_like_file:
path = Path(spec).expanduser().resolve()
if not path.exists():
raise FileNotFoundError(f"Derived data module not found: {path}")
module_spec = importlib.util.spec_from_file_location(path.stem, path)
if module_spec is None or module_spec.loader is None:
raise ImportError(f"Cannot load derived data module from {path}")
module = importlib.util.module_from_spec(module_spec)
module_spec.loader.exec_module(module)
else:
importlib.import_module(spec)
def _accepted_params(cls: Type[BaseDerivedData]) -> Optional[set[str]]:
"""Param names ``cls.__init__`` accepts, or None if it takes ``**kwargs``."""
sig = inspect.signature(cls.__init__)
if any(p.kind is p.VAR_KEYWORD for p in sig.parameters.values()):
return None
return {name for name in sig.parameters if name != "self"}
def _ensure_builtins() -> None:
global _builtins_loaded
if not _builtins_loaded:
import pipeline.derived.library # noqa: F401
_builtins_loaded = True
+1
View File
@@ -0,0 +1 @@
"""Daily feature plugin package."""
+8
View File
@@ -0,0 +1,8 @@
"""Compatibility alias for daily feature plugins.
The canonical plugin API is ``pipeline.derived``. ``BaseFeature`` remains as an
alias so existing external feature modules continue to register unchanged.
"""
from pipeline.derived.base import BaseDerivedData as BaseFeature
+108
View File
@@ -0,0 +1,108 @@
"""CLI for daily feature computation."""
import os
import click
import pandas as pd
from pipeline.features.compute import compute_feature
from pipeline.features.registry import available_features, load_feature_module
@click.group(name="feature")
def feature():
"""Compute daily feature parquet files from minute bars."""
def _coerce(value: str):
"""Best-effort coercion of a CLI string to int, then float, else str."""
for cast in (int, float):
try:
return cast(value)
except ValueError:
continue
return value
def _parse_params(pairs: tuple[str, ...]) -> dict:
"""Parse repeated ``name=value`` options into a params dict."""
params: dict = {}
for pair in pairs:
if "=" not in pair:
raise click.BadParameter(f"--param must be name=value, got '{pair}'")
key, value = pair.split("=", 1)
params[key.strip()] = _coerce(value.strip())
return params
@feature.command("list")
@click.option(
"--feature-module", "feature_modules", multiple=True,
help="External module(s) to import first (dotted path or .py file)",
)
def list_(feature_modules):
"""List the registered feature types."""
for spec in feature_modules:
load_feature_module(spec)
for name in available_features():
click.echo(name)
@feature.command("compute")
@click.option("--minute-path", required=True, help="Path to minute parquet dataset/file")
@click.option("--daily-path", default=None, help="Optional daily data parquet for alignment")
@click.option("--feature-type", required=True, help="Registry key of the feature class")
@click.option("--feature-name", required=True, help="Name for this feature run/output file")
@click.option("--output-dir", default="features", help="Directory to save feature parquet")
@click.option(
"--feature-module", "feature_modules", multiple=True,
help="External module(s) to import so their features register (dotted path or .py file)",
)
@click.option(
"--param", "extra_params", multiple=True,
help="Extra feature constructor param as name=value (repeatable)",
)
def compute(
minute_path,
daily_path,
feature_type,
feature_name,
output_dir,
feature_modules,
extra_params,
):
"""Compute one daily feature file from raw minute bars."""
for spec in feature_modules:
load_feature_module(spec)
options = available_features()
if feature_type not in options:
raise click.BadParameter(
f"Unknown feature-type '{feature_type}'. Available: {options}. "
f"Use --feature-module to register an external feature.",
param_hint="--feature-type",
)
minute = pd.read_parquet(minute_path)
click.echo(f"Loaded minute bars: {len(minute):,} rows from {minute_path}")
daily = None
if daily_path:
daily = pd.read_parquet(daily_path)
click.echo(f"Loaded daily data: {len(daily):,} rows from {daily_path}")
result = compute_feature(
minute=minute,
daily=daily,
feature_type=feature_type,
**_parse_params(extra_params),
)
os.makedirs(output_dir, exist_ok=True)
out_path = f"{output_dir}/{feature_name}.pq"
result.to_parquet(out_path, index=False)
feature_cols = [col for col in result.columns if col not in ("symbol_id", "date")]
click.echo(
f"Saved feature: {out_path} ({len(result):,} rows, "
f"{len(feature_cols)} columns)"
)
+41
View File
@@ -0,0 +1,41 @@
"""Compatibility wrappers for daily feature computation and validation."""
from pathlib import Path
from typing import Iterable
import pandas as pd
from pipeline.derived.compute import (
DERIVED_KEY_COLUMNS,
compute_derived,
read_derived_frames,
validate_derived_frame,
)
FEATURE_KEY_COLUMNS = DERIVED_KEY_COLUMNS
def validate_feature_frame(features: pd.DataFrame) -> pd.DataFrame:
"""Validate and normalize a legacy daily feature frame."""
return validate_derived_frame(features)
def compute_feature(
minute: pd.DataFrame,
feature_type: str,
daily: pd.DataFrame | None = None,
**params,
) -> pd.DataFrame:
"""Compute one registered feature through the derived-data registry."""
return compute_derived(
derived_type=feature_type,
daily=daily,
minute=minute,
**params,
)
def read_feature_frames(feature_paths: Iterable[str | Path]) -> list[pd.DataFrame]:
"""Read and validate feature/derived-data parquet files."""
return read_derived_frames(feature_paths)
+3
View File
@@ -0,0 +1,3 @@
"""Built-in feature library."""
from pipeline.features.library import minute_daily_summary # noqa: F401
@@ -0,0 +1,16 @@
"""Compatibility wrapper for the built-in minute daily summary plugin."""
import pandas as pd
from pipeline.derived.library.minute_daily_summary import MinuteDailySummaryDerived
class MinuteDailySummaryFeature(MinuteDailySummaryDerived):
"""Legacy minute-first wrapper around the derived-data implementation."""
def compute(
self,
minute: pd.DataFrame,
daily: pd.DataFrame | None = None,
) -> pd.DataFrame:
return super().compute(daily=daily, minute=minute)
+36
View File
@@ -0,0 +1,36 @@
"""Compatibility registry wrappers for daily feature plugins."""
from typing import Type
from pipeline.derived.base import BaseDerivedData
from pipeline.derived.registry import (
available_derived,
get_derived,
load_derived_module,
register_derived,
)
from pipeline.features.base import BaseFeature
def register_feature(cls: Type[BaseFeature]) -> Type[BaseFeature]:
"""Register a legacy feature plugin in the derived-data registry."""
return register_derived(cls)
def available_features() -> list[str]:
"""Sorted names of all registered feature/derived-data plugins."""
return available_derived()
def get_feature(name: str, **params) -> BaseDerivedData:
"""Instantiate a registered feature/derived-data plugin by name."""
if name == "minute_daily_summary":
from pipeline.features.library.minute_daily_summary import MinuteDailySummaryFeature
return MinuteDailySummaryFeature(**params)
return get_derived(name, **params)
def load_feature_module(spec: str) -> None:
"""Import an external module so its ``@register_feature`` classes register."""
load_derived_module(spec)
+6 -5
View File
@@ -166,13 +166,14 @@ class ReferenceSimulator(ExecutionSimulator):
st = wide(data_df, "isST") if "isST" in data_df.columns else opn * 0.0
symbols = sorted(set(tgt.columns) | set(opn.columns))
data_index = close.index
tgt = tgt.reindex(columns=symbols)
opn = opn.reindex(columns=symbols)
opn = opn.reindex(index=data_index, columns=symbols)
close = close.reindex(columns=symbols)
preclose = preclose.reindex(columns=symbols)
amount = amount.reindex(columns=symbols)
tstat = tstat.reindex(columns=symbols)
st = st.reindex(columns=symbols)
preclose = preclose.reindex(index=data_index, columns=symbols)
amount = amount.reindex(index=data_index, columns=symbols)
tstat = tstat.reindex(index=data_index, columns=symbols)
st = st.reindex(index=data_index, columns=symbols)
sym_arr = np.asarray(symbols, dtype=object)
n = len(symbols)
+24
View File
@@ -19,8 +19,32 @@ backtrader = [
[dependency-groups]
dev = [
"coverage>=7.14.1",
"pytest>=7.0.0",
]
[tool.uv]
package = false
[tool.pytest.ini_options]
markers = [
"network: tests that call live external data providers and are skipped unless explicitly enabled",
]
[tool.coverage.run]
branch = true
relative_files = true
source = [
".",
]
[tool.coverage.report]
fail_under = 80
show_missing = true
skip_covered = false
omit = [
"tests/*",
"docs/*",
"scripts/*",
".venv/*",
]
+40
View File
@@ -0,0 +1,40 @@
"""Run pytest under coverage in this environment.
The plain ``coverage run -m pytest`` path reloads NumPy under the current VS Code
Python startup environment, which breaks pandas/numpy reductions. Import NumPy
before starting coverage so the measured test run uses one stable NumPy module.
"""
from __future__ import annotations
import sys
import numpy # noqa: F401
import coverage
import pytest
def main(argv: list[str]) -> int:
pytest_args = argv or ["tests/", "-v"]
cov = coverage.Coverage(config_file=True)
cov.erase()
cov.start()
test_status = pytest.main(pytest_args)
cov.stop()
cov.save()
if test_status != 0:
return int(test_status)
total = cov.report()
print(f"\nCoverage total: {total:.2f}%")
fail_under = float(cov.config.fail_under)
if total < fail_under:
print(f"Coverage failure: total {total:.2f}% is below {fail_under:.2f}%")
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
Binary file not shown.
+261
View File
@@ -0,0 +1,261 @@
"""Shared deterministic test data for offline workflow tests."""
from __future__ import annotations
import numpy as np
import pandas as pd
from pipeline.common.schema import (
ALPHA_COLUMNS,
COMBO_COLUMNS,
DATA_COLUMNS,
MINUTE_BAR_COLUMNS,
)
GENERATED_SYMBOLS: tuple[str, ...] = (
"sh600000",
"sz000001",
"sh600519",
"sz300750",
)
GENERATED_SYMBOL_NAMES: dict[str, str] = {
"sh600000": "PF Bank",
"sz000001": "Ping An Bank",
"sh600519": "Kweichow Moutai",
"sz300750": "CATL",
}
def generated_sessions(n_sessions: int = 12) -> pd.DatetimeIndex:
"""Return a fixed business-day calendar used by generated fixtures."""
return pd.bdate_range("2024-01-02", periods=n_sessions)
def make_generated_daily_bars(
n_sessions: int = 12,
include_missing: bool = True,
) -> pd.DataFrame:
"""Build daily bars with explicit edge cases and no randomness.
The panel covers four A-share symbols and includes a suspended row, an ST
flag, a zero-volume row, a missing symbol-date, and limit-style open/close
moves. Values are deterministic so tests can assert exact identities.
"""
dates = generated_sessions(n_sessions)
base_close = {
"sh600000": 10.00,
"sz000001": 15.00,
"sh600519": 1200.00,
"sz300750": 180.00,
}
returns = {
"sh600000": [0.000, 0.012, -0.006, 0.018, 0.100, -0.014, 0.006, 0.000, 0.008, -0.011, 0.004, 0.009],
"sz000001": [0.000, -0.008, 0.011, -0.004, 0.006, 0.000, -0.012, 0.009, 0.005, -0.007, 0.010, -0.003],
"sh600519": [0.000, 0.006, 0.004, -0.010, 0.012, -0.006, 0.005, 0.003, -0.009, 0.007, -0.004, 0.006],
"sz300750": [0.000, -0.010, 0.014, 0.006, -0.008, 0.011, -0.004, 0.009, -0.200, 0.012, -0.006, 0.008],
}
base_volume = {
"sh600000": 1_200_000.0,
"sz000001": 900_000.0,
"sh600519": 80_000.0,
"sz300750": 240_000.0,
}
rows: list[dict[str, object]] = []
for sym in GENERATED_SYMBOLS:
closes = [base_close[sym]]
pattern = returns[sym]
for step in range(1, n_sessions):
ret = pattern[step % len(pattern)]
closes.append(closes[-1] * (1.0 + ret))
closes_arr = np.asarray(closes, dtype=float)
precloses = np.concatenate([[closes_arr[0]], closes_arr[:-1]])
for i, date in enumerate(dates):
preclose = float(precloses[i])
close = float(closes_arr[i])
open_price = preclose * (1.0 + 0.25 * (close / preclose - 1.0))
high = max(open_price, close) * 1.01
low = min(open_price, close) * 0.99
volume = base_volume[sym] + 10_000.0 * i
tradestatus = 1
is_st = 0
if sym == "sh600000" and i == 4:
open_price = preclose * 1.10
close = open_price
high = open_price
low = open_price
if sym == "sh600000" and i == 7:
volume = 0.0
if sym == "sz000001" and i == 5:
open_price = preclose
close = preclose
high = preclose
low = preclose
volume = 0.0
tradestatus = 0
if sym == "sh600519" and i == 7:
is_st = 1
if sym == "sz300750" and i == 8:
open_price = preclose * 0.80
close = open_price
high = open_price
low = open_price
amount = volume * ((open_price + close) / 2.0)
vwap = amount / volume if volume > 0 else np.nan
pct_chg = (close / preclose - 1.0) * 100.0 if preclose else 0.0
rows.append({
"symbol_id": sym,
"symbol_name": GENERATED_SYMBOL_NAMES[sym],
"date": date,
"open": open_price,
"high": high,
"low": low,
"close": close,
"preclose": preclose,
"volume": volume,
"amount": amount,
"vwap": vwap,
"turn": volume / 1_000_000.0,
"pctChg": pct_chg,
"tradestatus": tradestatus,
"isST": is_st,
"peTTM": 8.0 + i,
"pbMRQ": 1.0 + 0.05 * i,
"psTTM": 2.0 + 0.03 * i,
"pcfNcfTTM": 5.0 + 0.1 * i,
})
result = pd.DataFrame(rows)
if include_missing and n_sessions > 6:
missing_mask = (
(result["symbol_id"] == "sz300750")
& (result["date"] == dates[6])
)
result = result.loc[~missing_mask].copy()
result = result[DATA_COLUMNS]
return result.sort_values(["date", "symbol_id"]).reset_index(drop=True)
def make_generated_minute_bars(
daily: pd.DataFrame | None = None,
) -> pd.DataFrame:
"""Expand generated daily bars into a tiny deterministic intraday panel."""
daily = make_generated_daily_bars() if daily is None else daily.copy()
rows: list[dict[str, object]] = []
bar_times = ("09:35:00", "10:30:00", "14:55:00")
for daily_row in daily.sort_values(["date", "symbol_id"]).itertuples(index=False):
if int(getattr(daily_row, "tradestatus", 1)) == 0:
continue
volume = float(daily_row.volume)
volume_slices = [0.25 * volume, 0.35 * volume, 0.40 * volume]
prices = np.linspace(float(daily_row.open), float(daily_row.close), len(bar_times))
for j, time_text in enumerate(bar_times):
dt = pd.Timestamp(daily_row.date) + pd.Timedelta(time_text)
open_price = prices[j - 1] if j else float(daily_row.open)
close_price = float(prices[j])
high = max(open_price, close_price) * 1.002
low = min(open_price, close_price) * 0.998
minute_volume = float(volume_slices[j])
amount = minute_volume * ((open_price + close_price) / 2.0)
rows.append({
"symbol_id": daily_row.symbol_id,
"symbol_name": daily_row.symbol_name,
"datetime": dt,
"date": pd.Timestamp(daily_row.date).normalize(),
"time": time_text,
"frequency": "5m",
"open": open_price,
"high": high,
"low": low,
"close": close_price,
"volume": minute_volume,
"amount": amount,
"vwap": amount / minute_volume if minute_volume > 0 else np.nan,
"adjustflag": "3",
})
return pd.DataFrame(rows, columns=MINUTE_BAR_COLUMNS)
def make_generated_derived_features(
daily: pd.DataFrame | None = None,
) -> pd.DataFrame:
"""Return numeric daily derived values, including NaN and infinity cells."""
daily = make_generated_daily_bars() if daily is None else daily.copy()
keys = (
daily[["symbol_id", "date"]]
.drop_duplicates()
.sort_values(["date", "symbol_id"])
.reset_index(drop=True)
)
date_rank = keys["date"].rank(method="dense").astype(float)
symbol_rank = keys["symbol_id"].map({
"sh600000": 1.0,
"sz000001": 2.0,
"sh600519": 3.0,
"sz300750": 4.0,
})
out = keys.copy()
out["toy_feature"] = symbol_rank + date_rank / 100.0
out["finite_feature"] = symbol_rank * date_rank
out["nan_feature"] = out["toy_feature"]
out["inf_feature"] = out["toy_feature"]
if len(out) >= 2:
out.loc[out.index[0], "nan_feature"] = np.nan
out.loc[out.index[1], "inf_feature"] = np.inf
return out
def make_generated_alpha_weights(
alpha_name: str = "alpha_a",
*,
scale: float = 1.0,
offset: float = 0.0,
zero_date_index: int | None = None,
n_sessions: int = 10,
) -> pd.DataFrame:
"""Create a deterministic long alpha grid with optional zero-gross date."""
dates = generated_sessions(n_sessions)
even = np.array([1.20, -0.80, 0.40, -0.80], dtype=float)
odd = np.array([-0.60, 1.10, -0.90, 0.40], dtype=float)
rows: list[dict[str, object]] = []
for i, date in enumerate(dates):
vector = even.copy() if i % 2 == 0 else odd.copy()
vector = vector + offset
vector = vector - vector.mean()
if zero_date_index is not None and i == zero_date_index:
vector = np.zeros_like(vector)
for sym, weight in zip(GENERATED_SYMBOLS, scale * vector):
rows.append({
"symbol_id": sym,
"date": date,
"alpha_name": alpha_name,
"weight": float(weight),
})
result = pd.DataFrame(rows, columns=ALPHA_COLUMNS)
return result.sort_values(["symbol_id", "date"]).reset_index(drop=True)
def make_generated_combo_weights(
combo_name: str = "combo",
*,
zero_date_index: int | None = 2,
n_sessions: int = 10,
) -> pd.DataFrame:
"""Create deterministic combo weights for portfolio construction tests."""
alpha = make_generated_alpha_weights(
"combo_source",
zero_date_index=zero_date_index,
n_sessions=n_sessions,
)
combo = alpha.rename(columns={"alpha_name": "combo_name"}).copy()
combo["combo_name"] = combo_name
return combo[COMBO_COLUMNS].sort_values(["symbol_id", "date"]).reset_index(drop=True)
+155
View File
@@ -347,3 +347,158 @@ def test_universe_filter_does_not_corrupt_signal_history():
held = set(filtered.loc[filtered["weight"] != 0.0, "symbol_id"].unique())
# The two most liquid names (highest amount) are sh600519, sz300750.
assert held == {"sh600519", "sz300750"}
# --- feature-aware alpha integration ----------------------------------------
def test_compute_alpha_without_feature_path_matches_empty_feature_paths():
data = _make_data()
base = compute_alpha(data, "rev5", "reversal", lookback=5)
with_empty_features = compute_alpha(
data,
"rev5",
"reversal",
lookback=5,
feature_paths=[],
)
pd.testing.assert_frame_equal(base, with_empty_features)
def test_feature_aware_alpha_reads_joined_feature_column(tmp_path):
module_path = tmp_path / "feature_aware_alpha.py"
module_path.write_text(textwrap.dedent('''
import pandas as pd
from pipeline.alpha.base import BaseAlpha
from pipeline.alpha.registry import register_alpha
@register_alpha
class FeatureAwareAlpha(BaseAlpha):
name = "feature_aware_test_alpha"
def signal_from_data(
self,
data: pd.DataFrame,
close: pd.DataFrame,
) -> pd.DataFrame:
signal = data.pivot_table(
index="date",
columns="symbol_id",
values="toy_feature",
aggfunc="first",
)
return signal.reindex(index=close.index, columns=close.columns)
'''))
data = _make_data()
feature = data[["symbol_id", "date"]].copy()
feature["toy_feature"] = feature["symbol_id"].map({
"sh600000": 1.0,
"sz000001": 2.0,
"sh600519": 3.0,
})
feature_path = tmp_path / "toy_feature.pq"
feature.to_parquet(feature_path, index=False)
load_alpha_module(str(module_path))
result = compute_alpha(
data,
"feature_run",
"feature_aware_test_alpha",
feature_paths=[str(feature_path)],
)
assert list(result.columns) == ALPHA_COLUMNS
assert (result["alpha_name"] == "feature_run").all()
last = result[result["date"] == result["date"].max()]
assert last.set_index("symbol_id")["weight"].idxmax() == "sh600519"
def test_feature_paths_join_multiple_files_and_normalize_dates(tmp_path):
module_path = tmp_path / "multi_feature_alpha.py"
module_path.write_text(textwrap.dedent('''
import pandas as pd
from pipeline.alpha.base import BaseAlpha
from pipeline.alpha.registry import register_alpha
@register_alpha
class MultiFeatureAlpha(BaseAlpha):
name = "multi_feature_test_alpha"
def signal_from_data(
self,
data: pd.DataFrame,
close: pd.DataFrame,
) -> pd.DataFrame:
data = data.copy()
data["combined_feature"] = data["toy_a"] + data["toy_b"]
signal = data.pivot_table(
index="date",
columns="symbol_id",
values="combined_feature",
aggfunc="first",
)
return signal.reindex(index=close.index, columns=close.columns)
'''))
data = _make_data(n_days=8)
symbol_score = {"sh600000": 1.0, "sz000001": 2.0, "sh600519": 3.0}
feature_a = data[["symbol_id", "date"]].copy()
feature_a["date"] = feature_a["date"] + pd.Timedelta(hours=15)
feature_a["toy_a"] = feature_a["symbol_id"].map(symbol_score)
feature_b = data[["symbol_id", "date"]].copy()
feature_b["date"] = feature_b["date"].dt.strftime("%Y-%m-%d 09:30:00")
feature_b["toy_b"] = feature_b["symbol_id"].map(symbol_score) * 10.0
feature_a_path = tmp_path / "toy_a.pq"
feature_b_path = tmp_path / "toy_b.pq"
feature_a.to_parquet(feature_a_path, index=False)
feature_b.to_parquet(feature_b_path, index=False)
load_alpha_module(str(module_path))
result = compute_alpha(
data,
"multi_feature_run",
"multi_feature_test_alpha",
feature_paths=[str(feature_a_path), str(feature_b_path)],
)
assert list(result.columns) == ALPHA_COLUMNS
assert (result["alpha_name"] == "multi_feature_run").all()
last = result[result["date"] == result["date"].max()]
assert last.set_index("symbol_id")["weight"].idxmax() == "sh600519"
def test_compute_alpha_rejects_duplicate_feature_frame_columns():
data = _make_data()
duplicate_columns = pd.DataFrame(
[["sh600000", pd.Timestamp("2024-01-01"), 1.0, 2.0]],
columns=["symbol_id", "date", "toy_feature", "toy_feature"],
)
with pytest.raises(ValueError, match="duplicate columns"):
compute_alpha(
data,
"bad_features",
"reversal",
feature_frames=[duplicate_columns],
)
def test_compute_alpha_rejects_feature_path_collision_with_daily_data(tmp_path):
data = _make_data()
close_collision = data[["symbol_id", "date"]].copy()
close_collision["close"] = 1.0
close_collision_path = tmp_path / "close_collision.pq"
close_collision.to_parquet(close_collision_path, index=False)
with pytest.raises(ValueError, match="conflict"):
compute_alpha(
data,
"close_collision",
"reversal",
feature_paths=[str(close_collision_path)],
)
+545
View File
@@ -0,0 +1,545 @@
"""CLI handoff tests for the offline daily workflow."""
from __future__ import annotations
import textwrap
from pathlib import Path
import pandas as pd
from click.testing import CliRunner
from cli import cli
from tests.helpers import (
make_generated_daily_bars,
make_generated_derived_features,
make_generated_minute_bars,
)
FIXTURE_PATH = Path(__file__).parent / "fixtures" / "daily_bars_real_2024_01_sample.pq"
def _invoke_ok(runner: CliRunner, args: list[str]):
result = runner.invoke(cli, args)
assert result.exit_code == 0, result.output
return result
def _invoke_error(runner: CliRunner, args: list[str]):
result = runner.invoke(cli, args)
assert result.exit_code != 0, result.output
return result
def test_cli_daily_workflow_handoffs_stay_in_tmp_path(tmp_path):
runner = CliRunner()
daily_bars = make_generated_daily_bars()
minute_bars = make_generated_minute_bars(daily_bars)
derived_features = make_generated_derived_features(daily_bars)
daily_path = tmp_path / "daily_bars.pq"
minute_path = tmp_path / "minute_bars.pq"
derived_input_path = tmp_path / "derived_input.pq"
daily_bars.to_parquet(daily_path, index=False)
minute_bars.to_parquet(minute_path, index=False)
derived_features.to_parquet(derived_input_path, index=False)
ingest_dir = tmp_path / "derived_ingested"
ingest_result = _invoke_ok(runner, [
"derived", "ingest",
"--input-path", str(derived_input_path),
"--derived-name", "toy_features",
"--output-dir", str(ingest_dir),
])
ingested_feature_path = ingest_dir / "toy_features.pq"
assert "Saved derived data:" in ingest_result.output
assert ingested_feature_path.exists()
validate_result = _invoke_ok(runner, [
"derived", "validate",
"--input-path", str(ingested_feature_path),
])
assert "Valid derived data:" in validate_result.output
assert "rows" in validate_result.output
computed_derived_dir = tmp_path / "derived_computed"
derived_compute_result = _invoke_ok(runner, [
"derived", "compute",
"--daily-path", str(daily_path),
"--minute-path", str(minute_path),
"--derived-type", "minute_daily_summary",
"--derived-name", "minute_summary",
"--output-dir", str(computed_derived_dir),
])
minute_summary_path = computed_derived_dir / "minute_summary.pq"
assert "Loaded daily data:" in derived_compute_result.output
assert "Loaded minute bars:" in derived_compute_result.output
assert "Saved derived data:" in derived_compute_result.output
assert minute_summary_path.exists()
assert "minute_vwap" in pd.read_parquet(minute_summary_path).columns
alpha_module_path = tmp_path / "cli_feature_alpha.py"
alpha_module_path.write_text(textwrap.dedent("""
import pandas as pd
from pipeline.alpha.base import BaseAlpha
from pipeline.alpha.registry import register_alpha
@register_alpha
class CliFeatureAlpha(BaseAlpha):
name = "cli_feature_alpha_workflow"
def __init__(self, **kwargs):
self.kwargs = kwargs
def signal_from_data(
self,
data: pd.DataFrame,
close: pd.DataFrame,
) -> pd.DataFrame:
signal = data.pivot_table(
index="date",
columns="symbol_id",
values="minute_intraday_return",
aggfunc="first",
)
fallback = close.pct_change(1, fill_method=None)
feature_signal = signal.reindex(index=close.index, columns=close.columns)
toy_signal = data.pivot_table(
index="date",
columns="symbol_id",
values="toy_feature",
aggfunc="first",
)
toy_signal = toy_signal.reindex(index=close.index, columns=close.columns)
return feature_signal.fillna(fallback) + toy_signal / 1000.0
"""))
alpha_dir = tmp_path / "alphas"
alpha_result = _invoke_ok(runner, [
"alpha", "compute",
"--data-path", str(daily_path),
"--feature-path", str(minute_summary_path),
"--feature-path", str(ingested_feature_path),
"--alpha-module", str(alpha_module_path),
"--alpha-type", "cli_feature_alpha_workflow",
"--alpha-name", "cli_feature_alpha",
"--output-dir", str(alpha_dir),
])
alpha_path = alpha_dir / "cli_feature_alpha.pq"
assert "Loaded data:" in alpha_result.output
assert "Saved alpha:" in alpha_result.output
assert "Weight stats" in alpha_result.output
assert alpha_path.exists()
assert not pd.read_parquet(alpha_path).empty
alpha_report_dir = tmp_path / "alpha_reports"
alpha_eval_result = _invoke_ok(runner, [
"alpha", "eval",
"--alpha-path", str(alpha_path),
"--data-path", str(daily_path),
"--report-dir", str(alpha_report_dir),
])
alpha_report_path = alpha_report_dir / "cli_feature_alpha_eval.json"
assert "ALPHA EVALUATION" in alpha_eval_result.output
assert "Report saved:" in alpha_eval_result.output
assert alpha_report_path.exists()
combo_dir = tmp_path / "combos"
combo_result = _invoke_ok(runner, [
"combo", "combine",
"--alpha-paths", f"{alpha_path},{alpha_path}",
"--combo-name", "cli_combo",
"--method", "equal_weight",
"--output-dir", str(combo_dir),
])
combo_path = combo_dir / "cli_combo.pq"
assert "Saved combo:" in combo_result.output
assert "Weight stats" in combo_result.output
assert combo_path.exists()
portfolio_dir = tmp_path / "portfolio"
build_result = _invoke_ok(runner, [
"portfolio", "build",
"--weights-path", str(combo_path),
"--data-path", str(daily_path),
"--booksize", "2000000",
"--portfolio-name", "cli_portfolio",
"--output-dir", str(portfolio_dir),
])
positions_path = portfolio_dir / "cli_portfolio.pq"
assert "Saved positions:" in build_result.output
assert "Gross exposure" in build_result.output
assert positions_path.exists()
execution_dir = tmp_path / "execution"
simulate_result = _invoke_ok(runner, [
"portfolio", "simulate",
"--positions-path", str(positions_path),
"--data-path", str(daily_path),
"--constraint", "suspension",
"--constraint", "price_limit",
"--constraint", "volume_cap",
"--cost-bps", "5",
"--slippage-bps", "5",
"--volume-frac", "0.02",
"--output-dir", str(execution_dir),
])
fills_path = execution_dir / "fills" / "cli_portfolio.pq"
pnl_path = execution_dir / "pnl" / "cli_portfolio.pq"
assert "Saved fills:" in simulate_result.output
assert "Saved pnl:" in simulate_result.output
assert "Total PnL:" in simulate_result.output
assert fills_path.exists()
assert pnl_path.exists()
eval_result = _invoke_ok(runner, [
"portfolio", "eval",
"--positions-path", str(positions_path),
"--data-path", str(daily_path),
])
assert "Research-portfolio metrics:" in eval_result.output
assert "cumulative_return" in eval_result.output
assert "fitness" in eval_result.output
pqcat_result = _invoke_ok(runner, [
"pqcat",
str(positions_path),
"--info",
])
assert "shape:" in pqcat_result.output
assert "dtypes:" in pqcat_result.output
assert "position_shares" in pqcat_result.output
alphaview_result = _invoke_ok(runner, [
"alphaview",
"--data-path", str(daily_path),
"--alpha-path", str(alpha_path),
"--symbol", "sh600000",
"--start-date", "2024-01-02",
"--end-date", "2024-01-12",
"--columns", "close,volume",
])
assert "symbol: sh600000" in alphaview_result.output
assert "cli_feature_alpha" in alphaview_result.output
def test_cli_pipeline_accepts_partitioned_daily_dataset(tmp_path):
runner = CliRunner()
daily_bars = make_generated_daily_bars(include_missing=False)
dataset_dir = tmp_path / "daily_dataset"
dataset_frame = daily_bars.copy()
dataset_frame["month"] = dataset_frame["date"].dt.strftime("%Y-%m")
dataset_frame.to_parquet(dataset_dir, partition_cols=["month"], index=False)
alpha_dir = tmp_path / "alphas"
alpha_result = _invoke_ok(runner, [
"alpha", "compute",
"--data-path", str(dataset_dir),
"--alpha-type", "reversal",
"--alpha-name", "dataset_reversal",
"--lookback", "3",
"--output-dir", str(alpha_dir),
])
alpha_path = alpha_dir / "dataset_reversal.pq"
assert "Loaded data:" in alpha_result.output
assert alpha_path.exists()
combo_dir = tmp_path / "combos"
_invoke_ok(runner, [
"combo", "combine",
"--alpha-paths", str(alpha_path),
"--combo-name", "dataset_combo",
"--output-dir", str(combo_dir),
])
combo_path = combo_dir / "dataset_combo.pq"
assert combo_path.exists()
portfolio_dir = tmp_path / "portfolio"
_invoke_ok(runner, [
"portfolio", "build",
"--weights-path", str(combo_path),
"--data-path", str(dataset_dir),
"--booksize", "1000000",
"--portfolio-name", "dataset_portfolio",
"--output-dir", str(portfolio_dir),
])
positions_path = portfolio_dir / "dataset_portfolio.pq"
assert positions_path.exists()
execution_dir = tmp_path / "execution"
simulate_result = _invoke_ok(runner, [
"portfolio", "simulate",
"--positions-path", str(positions_path),
"--data-path", str(dataset_dir),
"--constraint", "suspension",
"--output-dir", str(execution_dir),
])
assert "Saved fills:" in simulate_result.output
assert (execution_dir / "fills" / "dataset_portfolio.pq").exists()
assert (execution_dir / "pnl" / "dataset_portfolio.pq").exists()
def test_cli_liquid_universe_masks_to_top_liquid_names(tmp_path):
runner = CliRunner()
daily_bars = make_generated_daily_bars(n_sessions=75, include_missing=False)
daily_path = tmp_path / "daily_bars_75d.pq"
daily_bars.to_parquet(daily_path, index=False)
alpha_dir = tmp_path / "alphas"
result = _invoke_ok(runner, [
"alpha", "compute",
"--data-path", str(daily_path),
"--alpha-type", "reversal_rank",
"--alpha-name", "liquid_rank",
"--lookback", "3",
"--liquid-universe",
"--universe-top-n", "2",
"--output-dir", str(alpha_dir),
])
alpha_path = alpha_dir / "liquid_rank.pq"
alpha = pd.read_parquet(alpha_path)
nonzero = alpha[alpha["weight"] != 0.0]
assert "Saved alpha:" in result.output
assert alpha_path.exists()
assert not nonzero.empty
assert nonzero.groupby("date")["symbol_id"].nunique().max() <= 2
def test_cli_real_fixture_round_trips_through_portfolio(tmp_path):
runner = CliRunner()
alpha_dir = tmp_path / "alphas"
_invoke_ok(runner, [
"alpha", "compute",
"--data-path", str(FIXTURE_PATH),
"--alpha-type", "reversal_vol",
"--alpha-name", "real_cli_reversal_vol",
"--lookback", "3",
"--vol-window", "3",
"--output-dir", str(alpha_dir),
])
alpha_path = alpha_dir / "real_cli_reversal_vol.pq"
assert alpha_path.exists()
assert not pd.read_parquet(alpha_path).empty
combo_dir = tmp_path / "combos"
_invoke_ok(runner, [
"combo", "combine",
"--alpha-paths", str(alpha_path),
"--combo-name", "real_cli_combo",
"--output-dir", str(combo_dir),
])
combo_path = combo_dir / "real_cli_combo.pq"
assert combo_path.exists()
portfolio_dir = tmp_path / "portfolio"
_invoke_ok(runner, [
"portfolio", "build",
"--weights-path", str(combo_path),
"--data-path", str(FIXTURE_PATH),
"--booksize", "1000000",
"--portfolio-name", "real_cli_portfolio",
"--output-dir", str(portfolio_dir),
])
positions_path = portfolio_dir / "real_cli_portfolio.pq"
positions = pd.read_parquet(positions_path)
assert not positions.empty
eval_result = _invoke_ok(runner, [
"portfolio", "eval",
"--positions-path", str(positions_path),
"--data-path", str(FIXTURE_PATH),
])
assert "Research-portfolio metrics:" in eval_result.output
def test_cli_error_paths_are_clear_for_bad_user_inputs(tmp_path):
runner = CliRunner()
daily_bars = make_generated_daily_bars()
daily_path = tmp_path / "daily_bars.pq"
daily_bars.to_parquet(daily_path, index=False)
unknown_alpha = _invoke_error(runner, [
"alpha", "compute",
"--data-path", str(daily_path),
"--alpha-type", "does_not_exist",
"--alpha-name", "bad",
"--output-dir", str(tmp_path / "alphas"),
])
assert "Unknown alpha-type" in unknown_alpha.output
malformed_param = _invoke_error(runner, [
"alpha", "compute",
"--data-path", str(daily_path),
"--alpha-type", "reversal",
"--alpha-name", "bad_param",
"--param", "not-an-assignment",
"--output-dir", str(tmp_path / "alphas"),
])
assert "--param must be name=value" in malformed_param.output
unknown_derived = _invoke_error(runner, [
"derived", "compute",
"--daily-path", str(daily_path),
"--derived-type", "does_not_exist",
"--derived-name", "bad",
"--output-dir", str(tmp_path / "derived"),
])
assert "Unknown derived-type" in unknown_derived.output
bad_constraint_positions = pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [pd.Timestamp("2024-01-02")],
"portfolio_name": ["bad_constraint"],
"target_weight": [1.0],
"target_value": [1000.0],
"target_shares": [100.0],
"position_shares": [100],
"position_value": [1000.0],
"price": [10.0],
})
positions_path = tmp_path / "positions.pq"
bad_constraint_positions.to_parquet(positions_path, index=False)
unknown_constraint = _invoke_error(runner, [
"portfolio", "simulate",
"--positions-path", str(positions_path),
"--data-path", str(daily_path),
"--constraint", "not_a_constraint",
"--output-dir", str(tmp_path / "execution"),
])
assert isinstance(unknown_constraint.exception, KeyError)
assert "not_a_constraint" in str(unknown_constraint.exception)
pqcat_missing_column = _invoke_error(runner, [
"pqcat",
str(daily_path),
"--columns", "close,not_a_column",
])
assert "Columns not found: not_a_column" in pqcat_missing_column.output
alphaview_missing_symbol = _invoke_error(runner, [
"alphaview",
"--data-path", str(daily_path),
"--alpha-path", str(positions_path),
"--symbol", "sh999999",
])
assert "Symbol 'sh999999' not found" in alphaview_missing_symbol.output
def test_cli_list_and_legacy_feature_paths(tmp_path):
runner = CliRunner()
daily_bars = make_generated_daily_bars(n_sessions=3, include_missing=False)
minute_bars = make_generated_minute_bars(daily_bars)
daily_path = tmp_path / "daily_bars.pq"
minute_path = tmp_path / "minute_bars.pq"
daily_bars.to_parquet(daily_path, index=False)
minute_bars.to_parquet(minute_path, index=False)
derived_list = _invoke_ok(runner, ["derived", "list"])
feature_list = _invoke_ok(runner, ["feature", "list"])
assert "minute_daily_summary" in derived_list.output
assert "minute_daily_summary" in feature_list.output
feature_dir = tmp_path / "features"
feature_compute = _invoke_ok(runner, [
"feature",
"compute",
"--minute-path",
str(minute_path),
"--daily-path",
str(daily_path),
"--feature-type",
"minute_daily_summary",
"--feature-name",
"legacy_summary",
"--output-dir",
str(feature_dir),
])
feature_path = feature_dir / "legacy_summary.pq"
assert "Loaded minute bars:" in feature_compute.output
assert "Loaded daily data:" in feature_compute.output
assert "Saved feature:" in feature_compute.output
assert feature_path.exists()
no_input = _invoke_error(runner, [
"derived",
"compute",
"--derived-type",
"minute_daily_summary",
"--derived-name",
"missing_inputs",
"--output-dir",
str(tmp_path / "derived"),
])
assert "At least one of --daily-path or --minute-path is required" in no_input.output
missing_minute = _invoke_error(runner, [
"derived",
"compute",
"--daily-path",
str(daily_path),
"--derived-type",
"minute_daily_summary",
"--derived-name",
"daily_only",
"--output-dir",
str(tmp_path / "derived"),
])
assert "minute_daily_summary requires minute input" in missing_minute.output
malformed_feature_param = _invoke_error(runner, [
"feature",
"compute",
"--minute-path",
str(minute_path),
"--feature-type",
"minute_daily_summary",
"--feature-name",
"bad_param",
"--param",
"not-an-assignment",
"--output-dir",
str(tmp_path / "features_bad"),
])
assert "--param must be name=value" in malformed_feature_param.output
unknown_feature = _invoke_error(runner, [
"feature",
"compute",
"--minute-path",
str(minute_path),
"--feature-type",
"does_not_exist",
"--feature-name",
"bad_feature",
"--output-dir",
str(tmp_path / "features_unknown"),
])
assert "Unknown feature-type" in unknown_feature.output
def test_cli_pqcat_row_modes(tmp_path):
runner = CliRunner()
daily_bars = make_generated_daily_bars(n_sessions=3, include_missing=False)
daily_path = tmp_path / "daily_bars.pq"
daily_bars.to_parquet(daily_path, index=False)
head_result = _invoke_ok(runner, [
"pqcat",
str(daily_path),
"--head",
"2",
"--columns",
"symbol_id,close",
])
tail_result = _invoke_ok(runner, [
"pqcat",
str(daily_path),
"--tail",
"1",
])
assert "symbol_id" in head_result.output
assert "close" in head_result.output
assert "date" in tail_result.output
+148
View File
@@ -0,0 +1,148 @@
"""Offline coverage for data CLI and universe resolution glue."""
from __future__ import annotations
import pandas as pd
from click.testing import CliRunner
from cli import cli
import pipeline.data.cli as data_cli
import pipeline.data.downloader as pipeline_downloader
def test_resolve_universe_handles_named_file_all_and_symbol_list(tmp_path, monkeypatch):
hs300_raw = pd.DataFrame({
"updateDate": ["2024-01-12", "2024-01-12"],
"stockName": ["浦发银行", "平安银行"],
"stockCode": ["sh.600000", "sz.000001"],
})
zz500_raw = pd.DataFrame({
"name": ["东风汽车"],
"code": ["sh.600006"],
"date": ["2024-01-12"],
})
monkeypatch.setattr(pipeline_downloader, "get_hs300_stocks", lambda: hs300_raw)
monkeypatch.setattr(pipeline_downloader, "get_zz500_stocks", lambda: zz500_raw)
monkeypatch.setattr(
pipeline_downloader,
"get_all_stocks",
lambda: pd.DataFrame({
"code": ["sh600000", "sz000001", "sh600519"],
"name": ["浦发银行", "平安银行", "贵州茅台"],
}),
)
symbol_file = tmp_path / "symbols.txt"
symbol_file.write_text("sh600000\n\nsz000001\n")
hs300 = pipeline_downloader._resolve_universe("hs300")
zz500 = pipeline_downloader._resolve_universe("csi500")
all_capped = pipeline_downloader._resolve_universe("all", max_symbols=2)
from_file = pipeline_downloader._resolve_universe(str(symbol_file))
from_list = pipeline_downloader._resolve_universe("sh600000, sz000001")
assert hs300.to_dict("list") == {
"symbol_name": ["浦发银行", "平安银行"],
"symbol_id": ["sh600000", "sz000001"],
}
assert zz500.to_dict("list") == {
"symbol_name": ["东风汽车"],
"symbol_id": ["sh600006"],
}
assert all_capped["symbol_id"].tolist() == ["sh600000", "sz000001"]
assert from_file["symbol_id"].tolist() == ["sh600000", "sz000001"]
assert from_file["symbol_name"].tolist() == ["sh600000", "sz000001"]
assert from_list["symbol_id"].tolist() == ["sh600000", "sz000001"]
def test_data_cli_download_commands_print_summaries_without_network(monkeypatch, tmp_path):
runner = CliRunner()
daily_calls: list[dict] = []
minute_calls: list[dict] = []
def fake_daily(**kwargs):
daily_calls.append(kwargs)
return {
"dataset_path": str(tmp_path / "daily" / kwargs["universe"]),
"n_symbols": 2,
"n_requested": 3,
"n_rows": 18,
"date_min": "2024-01-02",
"date_max": "2024-01-12",
}
def fake_minute(**kwargs):
minute_calls.append(kwargs)
return {
"dataset_path": str(tmp_path / "minute" / kwargs["universe"]),
"frequency": "15m",
"n_symbols": 1,
"n_requested": 1,
"n_rows": 32,
"date_min": "2024-01-02",
"date_max": "2024-01-03",
}
monkeypatch.setattr(data_cli, "download_universe", fake_daily)
monkeypatch.setattr(data_cli, "download_minute_universe", fake_minute)
daily_result = runner.invoke(cli, [
"data",
"download",
"--universe",
"sh600000,sz000001",
"--start-date",
"2024-01-02",
"--end-date",
"2024-01-12",
"--output-dir",
str(tmp_path / "daily"),
"--symbols",
"3",
"--chunk-size",
"2",
"--adjust",
"none",
])
minute_result = runner.invoke(cli, [
"data",
"download-minute",
"--universe",
"toy",
"--start-date",
"2024-01-02",
"--end-date",
"2024-01-03",
"--output-dir",
str(tmp_path / "minute"),
"--symbols",
"1",
"--chunk-size",
"1",
"--frequency",
"15",
])
assert daily_result.exit_code == 0, daily_result.output
assert "Summary: 2/3 symbols, 18 bars" in daily_result.output
assert daily_calls == [{
"universe": "sh600000,sz000001",
"start_date": "2024-01-02",
"end_date": "2024-01-12",
"output_dir": str(tmp_path / "daily"),
"max_symbols": 3,
"chunk_size": 2,
"adjust": "none",
}]
assert minute_result.exit_code == 0, minute_result.output
assert "frequency=15m" in minute_result.output
assert minute_calls == [{
"universe": "toy",
"start_date": "2024-01-02",
"end_date": "2024-01-03",
"output_dir": str(tmp_path / "minute"),
"max_symbols": 1,
"chunk_size": 1,
"frequency": "15",
}]
+261
View File
@@ -0,0 +1,261 @@
"""Tests for daily derived-data ingestion and plugins."""
import textwrap
import numpy as np
import pandas as pd
import pytest
from click.testing import CliRunner
from cli import cli
from pipeline.alpha.compute import join_feature_frames
from pipeline.derived.compute import compute_derived, validate_derived_frame
from pipeline.derived.registry import available_derived, get_derived, load_derived_module
def _daily_bars() -> pd.DataFrame:
return pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sh600000"],
"date": pd.to_datetime(["2024-01-02", "2024-01-02", "2024-01-03"]),
"open": [10.0, 20.0, 11.0],
"close": [10.5, 20.5, 11.5],
"volume": [1000.0, 2000.0, 1200.0],
})
def _minute_bars() -> pd.DataFrame:
return pd.DataFrame({
"symbol_id": ["sh600000", "sh600000", "sz000001"],
"datetime": pd.to_datetime([
"2024-01-02 09:35:00",
"2024-01-02 09:40:00",
"2024-01-02 09:35:00",
]),
"date": pd.to_datetime(["2024-01-02", "2024-01-02", "2024-01-02"]),
"time": ["09:35:00", "09:40:00", "09:35:00"],
"open": [10.0, 10.5, 20.0],
"high": [11.0, 12.0, 21.0],
"low": [9.0, 10.0, 19.0],
"close": [10.5, 11.0, 20.5],
"volume": [100.0, 300.0, 200.0],
"amount": [1000.0, 3300.0, 4100.0],
})
def test_validate_derived_frame_normalizes_and_sorts():
result = validate_derived_frame(pd.DataFrame({
"symbol_id": ["sz000001", "sh600000"],
"date": ["2024-01-02 15:00:00", "2024-01-02 09:30:00"],
"custom_value": [2.0, 1.0],
}))
assert result["symbol_id"].tolist() == ["sh600000", "sz000001"]
assert result["date"].tolist() == [
pd.Timestamp("2024-01-02"),
pd.Timestamp("2024-01-02"),
]
def test_validate_derived_frame_rejects_missing_keys():
with pytest.raises(ValueError, match="missing required"):
validate_derived_frame(pd.DataFrame({"symbol_id": ["sh600000"], "x": [1.0]}))
def test_validate_derived_frame_rejects_duplicate_normalized_keys():
with pytest.raises(ValueError, match="duplicate symbol_id,date"):
validate_derived_frame(pd.DataFrame({
"symbol_id": ["sh600000", "sh600000"],
"date": ["2024-01-02 09:30:00", "2024-01-02 15:00:00"],
"x": [1.0, 2.0],
}))
def test_validate_derived_frame_rejects_duplicate_columns():
bad = pd.DataFrame(
[["sh600000", pd.Timestamp("2024-01-02"), 1.0, 2.0]],
columns=["symbol_id", "date", "dup", "dup"],
)
with pytest.raises(ValueError, match="duplicate columns"):
validate_derived_frame(bad)
def test_validate_derived_frame_rejects_non_numeric_values():
with pytest.raises(ValueError, match="numeric"):
validate_derived_frame(pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [pd.Timestamp("2024-01-02")],
"bad": ["not numeric"],
}))
def test_derived_ingest_cli_accepts_csv_and_parquet(tmp_path):
runner = CliRunner()
source = pd.DataFrame({
"symbol_id": ["sz000001", "sh600000"],
"date": ["2024-01-02", "2024-01-02"],
"custom_value": [2.0, 1.0],
})
csv_path = tmp_path / "custom.csv"
parquet_path = tmp_path / "custom.pq"
out_dir = tmp_path / "derived"
source.to_csv(csv_path, index=False)
source.to_parquet(parquet_path, index=False)
csv_result = runner.invoke(cli, [
"derived", "ingest",
"--input-path", str(csv_path),
"--derived-name", "csv_custom",
"--output-dir", str(out_dir),
])
assert csv_result.exit_code == 0, csv_result.output
parquet_result = runner.invoke(cli, [
"derived", "ingest",
"--input-path", str(parquet_path),
"--derived-name", "parquet_custom",
"--output-dir", str(out_dir),
])
assert parquet_result.exit_code == 0, parquet_result.output
written = pd.read_parquet(out_dir / "csv_custom.pq")
assert written["symbol_id"].tolist() == ["sh600000", "sz000001"]
assert (out_dir / "parquet_custom.pq").exists()
def test_derived_validate_cli_rejects_duplicate_csv_columns(tmp_path):
runner = CliRunner()
csv_path = tmp_path / "bad.csv"
csv_path.write_text("symbol_id,date,x,x\nsh600000,2024-01-02,1.0,2.0\n")
result = runner.invoke(cli, [
"derived", "validate",
"--input-path", str(csv_path),
])
assert result.exit_code != 0
assert "duplicate columns" in result.output
def test_external_derived_plugin_loads_filters_params_and_uses_inputs(tmp_path):
module_path = tmp_path / "external_derived.py"
module_path.write_text(textwrap.dedent('''
import pandas as pd
from pipeline.derived.base import BaseDerivedData
from pipeline.derived.registry import register_derived
@register_derived
class FlexibleDerived(BaseDerivedData):
name = "flexible_derived_test"
def __init__(self, scale: float = 1.0):
self.scale = scale
def compute(self, daily=None, minute=None) -> pd.DataFrame:
result = None
if daily is not None:
result = daily[["symbol_id", "date", "close"]].copy()
result["daily_scaled_close"] = result.pop("close") * self.scale
if minute is not None:
minute_out = (
minute.groupby(["symbol_id", "date"], as_index=False)["volume"]
.sum()
.rename(columns={"volume": "minute_volume_sum"})
)
minute_out["minute_volume_sum"] *= self.scale
result = minute_out if result is None else result.merge(
minute_out, on=["symbol_id", "date"], how="left"
)
return result
'''))
load_derived_module(str(module_path))
assert "flexible_derived_test" in available_derived()
instance = get_derived("flexible_derived_test", scale=2.0, ignored=99)
assert instance.scale == 2.0
assert not hasattr(instance, "ignored")
daily_result = compute_derived(
"flexible_derived_test",
daily=_daily_bars(),
scale=2.0,
ignored=99,
)
assert "daily_scaled_close" in daily_result.columns
assert np.isclose(daily_result["daily_scaled_close"].iloc[0], 21.0)
minute_result = compute_derived(
"flexible_derived_test",
minute=_minute_bars(),
scale=2.0,
)
assert "minute_volume_sum" in minute_result.columns
assert np.isclose(
minute_result.loc[minute_result["symbol_id"] == "sh600000", "minute_volume_sum"].iloc[0],
800.0,
)
both_result = compute_derived(
"flexible_derived_test",
daily=_daily_bars(),
minute=_minute_bars(),
scale=1.0,
)
assert {"daily_scaled_close", "minute_volume_sum"}.issubset(both_result.columns)
def test_derived_compute_cli_writes_builtin_minute_summary(tmp_path):
runner = CliRunner()
minute_path = tmp_path / "minute.pq"
out_dir = tmp_path / "derived"
_minute_bars().to_parquet(minute_path, index=False)
result = runner.invoke(cli, [
"derived", "compute",
"--minute-path", str(minute_path),
"--derived-type", "minute_daily_summary",
"--derived-name", "minute_summary",
"--output-dir", str(out_dir),
])
assert result.exit_code == 0, result.output
written = pd.read_parquet(out_dir / "minute_summary.pq")
assert "minute_vwap" in written.columns
def test_alpha_feature_join_rejects_derived_column_collisions():
data = _daily_bars()
derived_a = data[["symbol_id", "date"]].copy()
derived_a["custom_value"] = 1.0
derived_b = data[["symbol_id", "date"]].copy()
derived_b["custom_value"] = 2.0
with pytest.raises(ValueError, match="conflict"):
join_feature_frames(data, [derived_a, derived_b])
close_collision = data[["symbol_id", "date"]].copy()
close_collision["close"] = 1.0
with pytest.raises(ValueError, match="conflict"):
join_feature_frames(data, [close_collision])
def test_legacy_feature_cli_delegates_to_derived_registry(tmp_path):
runner = CliRunner()
minute_path = tmp_path / "minute.pq"
out_dir = tmp_path / "features"
_minute_bars().to_parquet(minute_path, index=False)
list_result = runner.invoke(cli, ["feature", "list"])
assert list_result.exit_code == 0, list_result.output
assert "minute_daily_summary" in list_result.output
compute_result = runner.invoke(cli, [
"feature", "compute",
"--minute-path", str(minute_path),
"--feature-type", "minute_daily_summary",
"--feature-name", "minute_summary",
"--output-dir", str(out_dir),
])
assert compute_result.exit_code == 0, compute_result.output
assert (out_dir / "minute_summary.pq").exists()
+10
View File
@@ -1,6 +1,16 @@
import os
import pytest
from data.downloader import download_daily
pytestmark = [
pytest.mark.network,
pytest.mark.skipif(
os.environ.get("CEQ_RUN_LIVE_DOWNLOADER") != "1",
reason="set CEQ_RUN_LIVE_DOWNLOADER=1 to run live baostock/akshare smoke tests",
),
]
def test_download_single_stock():
"""Smoke test: download data for 浦发银行 for a short window."""
+462
View File
@@ -0,0 +1,462 @@
"""Offline downloader contract tests with mocked data providers."""
from __future__ import annotations
import numpy as np
import pandas as pd
import pytest
import data.downloader as downloader
import pipeline.data.downloader as pipeline_downloader
from data.downloader import download_daily, download_daily_batch
from pipeline.common.schema import DATA_COLUMNS
from pipeline.data.downloader import download_universe
class _FakeResult:
def __init__(self, rows, error_code="0", error_msg=""):
self.rows = rows
self.error_code = error_code
self.error_msg = error_msg
self._idx = -1
def next(self):
self._idx += 1
return self._idx < len(self.rows)
def get_row_data(self):
return self.rows[self._idx]
def _daily_batch_row(
date: str = "2024-01-02",
open_: str = "10",
high: str = "11",
low: str = "9",
close: str = "10.5",
preclose: str = "10",
volume: str = "1000",
amount: str = "10500",
) -> list[str]:
return [
date,
open_,
high,
low,
close,
preclose,
volume,
amount,
"1.23",
"5.0",
"1",
"0",
"8.0",
"1.1",
"2.2",
"3.3",
]
def test_download_daily_uses_baostock_before_akshare_in_auto(monkeypatch):
calls: list[str] = []
expected = pd.DataFrame({
"symbol": ["sh600000"],
"date": ["2024-01-02"],
"open": [10.0],
"high": [11.0],
"low": [9.0],
"close": [10.5],
"volume": [1000.0],
"amount": [10500.0],
})
def fake_baostock(symbol, start, end, adjust):
calls.append("baostock")
return expected
def fake_akshare(symbol, start, end, adjust):
calls.append("akshare")
raise AssertionError("akshare should not be called after baostock succeeds")
monkeypatch.setattr(downloader, "_download_baostock", fake_baostock)
monkeypatch.setattr(downloader, "_download_akshare", fake_akshare)
result = download_daily("sh600000", "2024-01-02", "2024-01-02", source="auto")
assert calls == ["baostock"]
assert result["date"].tolist() == [pd.Timestamp("2024-01-02")]
assert result["close"].tolist() == [10.5]
def test_download_daily_falls_back_to_akshare_when_baostock_empty(monkeypatch):
calls: list[str] = []
fallback = pd.DataFrame({
"symbol": ["sz000001"],
"date": ["2024-01-02"],
"open": [20.0],
"high": [21.0],
"low": [19.0],
"close": [20.5],
"volume": [2000.0],
"amount": [41000.0],
})
monkeypatch.setattr(
downloader,
"_download_baostock",
lambda symbol, start, end, adjust: calls.append("baostock") or None,
)
monkeypatch.setattr(
downloader,
"_download_akshare",
lambda symbol, start, end, adjust: calls.append("akshare") or fallback,
)
result = download_daily("sz000001", "2024-01-02", "2024-01-02", source="auto")
assert calls == ["baostock", "akshare"]
assert result["symbol"].tolist() == ["sz000001"]
assert result["date"].tolist() == [pd.Timestamp("2024-01-02")]
def test_download_daily_raises_when_requested_source_has_no_data(monkeypatch):
monkeypatch.setattr(downloader, "_download_baostock", lambda *args: None)
with pytest.raises(RuntimeError, match="Failed to download data for sh600000"):
download_daily(
"sh600000",
"2024-01-02",
"2024-01-02",
source="baostock",
)
def test_akshare_daily_downloader_maps_columns_and_failures(monkeypatch):
calls: list[dict] = []
raw = pd.DataFrame({
"日期": ["2024-01-02"],
"开盘": [10.0],
"最高": [11.0],
"最低": [9.0],
"收盘": [10.5],
"成交量": [1000.0],
"成交额": [10500.0],
"换手率": [1.2],
})
def fake_hist(**kwargs):
calls.append(kwargs)
return raw.copy()
monkeypatch.setattr(downloader.ak, "stock_zh_a_hist", fake_hist)
result = downloader._download_akshare(
"sh600000",
"20240102",
"20240102",
adjust="",
)
assert calls == [{
"symbol": "600000",
"period": "daily",
"start_date": "20240102",
"end_date": "20240102",
"adjust": "",
}]
assert result is not None
assert result.columns.tolist() == [
"symbol", "date", "open", "high", "low", "close", "volume", "amount",
]
assert result["symbol"].tolist() == ["sh600000"]
monkeypatch.setattr(downloader.ak, "stock_zh_a_hist", lambda **kwargs: pd.DataFrame())
assert downloader._download_akshare("sh600000", "20240102", "20240102") is None
monkeypatch.setattr(
downloader.ak,
"stock_zh_a_hist",
lambda **kwargs: (_ for _ in ()).throw(RuntimeError("provider down")),
)
assert downloader._download_akshare("sh600000", "20240102", "20240102") is None
def test_baostock_daily_downloader_maps_errors_and_logout_failures(monkeypatch):
query_calls: list[dict] = []
row = ["2024-01-02", "10", "11", "9", "10.5", "1000", "10500"]
monkeypatch.setattr(downloader.bs, "login", lambda: None)
monkeypatch.setattr(
downloader.bs,
"logout",
lambda: (_ for _ in ()).throw(RuntimeError("logout failed")),
)
def fake_query(**kwargs):
query_calls.append(kwargs)
return _FakeResult([row])
monkeypatch.setattr(downloader.bs, "query_history_k_data_plus", fake_query)
result = downloader._download_baostock(
"sz000001",
"2024-01-02",
"2024-01-02",
adjust="none",
)
assert query_calls[0]["code"] == "sz.000001"
assert query_calls[0]["adjustflag"] == "3"
assert result is not None
assert result["symbol"].tolist() == ["sz000001"]
assert pd.api.types.is_numeric_dtype(result["close"])
monkeypatch.setattr(downloader.bs, "logout", lambda: None)
monkeypatch.setattr(
downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: _FakeResult([], error_code="1", error_msg="bad symbol"),
)
assert downloader._download_baostock("sz000001", "2024-01-02", "2024-01-02") is None
monkeypatch.setattr(
downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: _FakeResult([]),
)
assert downloader._download_baostock("sz000001", "2024-01-02", "2024-01-02") is None
monkeypatch.setattr(
downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: (_ for _ in ()).throw(RuntimeError("query failed")),
)
assert downloader._download_baostock("sz000001", "2024-01-02", "2024-01-02") is None
def test_download_daily_batch_maps_rich_schema_and_vwap(monkeypatch):
query_calls: list[dict] = []
login_count = 0
logout_count = 0
def fake_login():
nonlocal login_count
login_count += 1
def fake_logout():
nonlocal logout_count
logout_count += 1
def fake_query(**kwargs):
query_calls.append(kwargs)
rows = [
_daily_batch_row(volume="1000", amount="10500"),
_daily_batch_row(date="2024-01-03", volume="0", amount="0"),
]
return _FakeResult(rows)
monkeypatch.setattr(downloader.bs, "login", fake_login)
monkeypatch.setattr(downloader.bs, "logout", fake_logout)
monkeypatch.setattr(downloader.bs, "query_history_k_data_plus", fake_query)
[(symbol, frame)] = list(
download_daily_batch(
["sh600000"],
"2024-01-02",
"2024-01-03",
adjust="hfq",
)
)
assert symbol == "sh600000"
assert query_calls[0]["code"] == "sh.600000"
assert query_calls[0]["adjustflag"] == "1"
assert login_count == 1
assert logout_count == 1
assert frame is not None
assert frame.columns.tolist() == [
"symbol", "date", "open", "high", "low", "close", "preclose",
"volume", "amount", "vwap", "turn", "pctChg", "tradestatus", "isST",
"peTTM", "pbMRQ", "psTTM", "pcfNcfTTM",
]
assert np.isclose(frame["vwap"].iloc[0], 10.5)
assert pd.isna(frame["vwap"].iloc[1])
assert pd.api.types.is_datetime64_any_dtype(frame["date"])
assert pd.api.types.is_numeric_dtype(frame["tradestatus"])
def test_download_daily_batch_periodic_relogin_and_none_result(monkeypatch):
responses = [
_FakeResult([], error_code="1", error_msg="bad symbol"),
_FakeResult([_daily_batch_row(date="2024-01-03")]),
]
login_count = 0
logout_count = 0
def fake_login():
nonlocal login_count
login_count += 1
def fake_logout():
nonlocal logout_count
logout_count += 1
monkeypatch.setattr(downloader.bs, "login", fake_login)
monkeypatch.setattr(downloader.bs, "logout", fake_logout)
monkeypatch.setattr(
downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: responses.pop(0),
)
results = list(
download_daily_batch(
["sh600000", "sz000001"],
"2024-01-02",
"2024-01-03",
relogin_every=1,
)
)
assert results[0] == ("sh600000", None)
assert results[1][0] == "sz000001"
assert results[1][1] is not None
assert login_count == 2
assert logout_count == 2
def test_download_daily_batch_relogs_and_retries_session_loss(monkeypatch):
responses = [
_FakeResult([], error_code="10002007", error_msg="用户未登录"),
_FakeResult([_daily_batch_row()]),
]
login_count = 0
logout_count = 0
def fake_login():
nonlocal login_count
login_count += 1
def fake_logout():
nonlocal logout_count
logout_count += 1
monkeypatch.setattr(downloader.bs, "login", fake_login)
monkeypatch.setattr(downloader.bs, "logout", fake_logout)
monkeypatch.setattr(
downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: responses.pop(0),
)
[(symbol, frame)] = list(download_daily_batch(["sh600000"], "2024-01-02", "2024-01-02"))
assert symbol == "sh600000"
assert frame is not None
assert len(frame) == 1
assert login_count == 2
assert logout_count == 2
def test_download_daily_batch_uses_akshare_fallback_when_enabled(monkeypatch):
fallback = pd.DataFrame({
"symbol": ["sh600000"],
"date": ["2024-01-02"],
"open": [10.0],
"high": [11.0],
"low": [9.0],
"close": [10.5],
"volume": [1000.0],
"amount": [10500.0],
})
monkeypatch.setattr(downloader.bs, "login", lambda: None)
monkeypatch.setattr(downloader.bs, "logout", lambda: None)
monkeypatch.setattr(
downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: _FakeResult([], error_code="1", error_msg="no data"),
)
monkeypatch.setattr(
downloader,
"_download_akshare",
lambda symbol, start, end, adjust: fallback.copy(),
)
[(symbol, frame)] = list(
download_daily_batch(
["sh600000"],
"2024-01-02",
"2024-01-02",
akshare_fallback=True,
)
)
assert symbol == "sh600000"
assert frame is not None
assert frame["date"].tolist() == [pd.Timestamp("2024-01-02")]
assert frame["close"].tolist() == [10.5]
def test_download_universe_writes_daily_partitions_from_mock_batch(tmp_path, monkeypatch):
batch_frame = pd.DataFrame({
"symbol": ["sh600000", "sh600000"],
"date": pd.to_datetime(["2024-01-02", "2024-02-01"]),
"open": [10.0, 11.0],
"high": [11.0, 12.0],
"low": [9.0, 10.0],
"close": [10.5, 11.5],
"preclose": [10.0, 10.5],
"volume": [1000.0, 1200.0],
"amount": [10500.0, 13800.0],
"vwap": [10.5, 11.5],
"turn": [1.0, 1.1],
"pctChg": [5.0, 9.5],
"tradestatus": [1, 1],
"isST": [0, 0],
"peTTM": [8.0, 8.1],
"pbMRQ": [1.1, 1.2],
"psTTM": [2.1, 2.2],
"pcfNcfTTM": [3.1, 3.2],
})
monkeypatch.setattr(
pipeline_downloader,
"_resolve_universe",
lambda universe, max_symbols=0: pd.DataFrame({
"symbol_id": ["sh600000", "sz000001"],
"symbol_name": ["PF Bank", "Ping An Bank"],
}),
)
def fake_batch(symbols, start, end, adjust="qfq"):
assert symbols == ["sh600000", "sz000001"]
assert adjust == "qfq"
yield "sh600000", batch_frame
yield "sz000001", None
monkeypatch.setattr(pipeline_downloader, "download_daily_batch", fake_batch)
stats = download_universe(
universe="toy",
start_date="2024-01-02",
end_date="2024-02-01",
output_dir=str(tmp_path),
chunk_size=1,
)
dataset_path = tmp_path / "toy"
written = pd.read_parquet(dataset_path).sort_values(["date", "symbol_id"]).reset_index(drop=True)
assert stats == {
"dataset_path": str(dataset_path),
"n_symbols": 1,
"n_requested": 2,
"n_rows": 2,
"date_min": "2024-01-02",
"date_max": "2024-02-01",
}
assert (dataset_path / "month=2024-01").exists()
assert (dataset_path / "month=2024-02").exists()
assert written[DATA_COLUMNS].columns.tolist() == DATA_COLUMNS
assert written["symbol_name"].tolist() == ["PF Bank", "PF Bank"]
+172
View File
@@ -0,0 +1,172 @@
"""Tests for minute-derived daily feature plugins."""
import textwrap
import numpy as np
import pandas as pd
import pytest
from pipeline.features.compute import compute_feature, validate_feature_frame
from pipeline.features.library.minute_daily_summary import MinuteDailySummaryFeature
from pipeline.features.registry import (
available_features,
get_feature,
load_feature_module,
)
from pipeline.derived.compute import compute_derived
def _minute_bars() -> pd.DataFrame:
return pd.DataFrame({
"symbol_id": ["sh600000", "sh600000", "sz000001"],
"symbol_name": ["PF Bank", "PF Bank", "Ping An"],
"datetime": pd.to_datetime([
"2024-01-02 09:35:00",
"2024-01-02 09:40:00",
"2024-01-02 09:35:00",
]),
"date": pd.to_datetime(["2024-01-02", "2024-01-02", "2024-01-02"]),
"time": ["09:35:00", "09:40:00", "09:35:00"],
"frequency": ["5m", "5m", "5m"],
"open": [10.0, 10.5, 20.0],
"high": [11.0, 12.0, 21.0],
"low": [9.0, 10.0, 19.0],
"close": [10.5, 11.0, 20.5],
"volume": [100.0, 300.0, 200.0],
"amount": [1000.0, 3300.0, 4100.0],
"vwap": [10.0, 11.0, 20.5],
"adjustflag": ["3", "3", "3"],
})
def test_built_in_minute_daily_summary():
daily = pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sh600000"],
"date": pd.to_datetime(["2024-01-02", "2024-01-02", "2024-01-03"]),
"close": [11.0, 20.5, 12.0],
})
result = compute_feature(
minute=_minute_bars(),
daily=daily,
feature_type="minute_daily_summary",
)
assert "minute_daily_summary" in available_features()
row = result[
(result["symbol_id"] == "sh600000")
& (result["date"] == pd.Timestamp("2024-01-02"))
].iloc[0]
assert row["minute_bar_count"] == 2
assert np.isclose(row["minute_intraday_return"], 11.0 / 10.0 - 1.0)
assert np.isclose(row["minute_intraday_range"], 12.0 / 9.0 - 1.0)
assert np.isclose(row["minute_vwap"], 4300.0 / 400.0)
assert np.isclose(row["minute_vwap_deviation"], (4300.0 / 400.0) / 11.0 - 1.0)
missing = result[
(result["symbol_id"] == "sh600000")
& (result["date"] == pd.Timestamp("2024-01-03"))
].iloc[0]
assert pd.isna(missing["minute_vwap"])
def test_minute_daily_summary_feature_preserves_legacy_positional_compute():
direct = MinuteDailySummaryFeature().compute(_minute_bars())
via_registry = get_feature("minute_daily_summary").compute(_minute_bars())
assert "minute_vwap" in direct.columns
pd.testing.assert_frame_equal(direct, via_registry)
def test_legacy_feature_compute_matches_canonical_derived_compute():
daily = pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sh600000"],
"date": pd.to_datetime(["2024-01-02", "2024-01-02", "2024-01-03"]),
"close": [11.0, 20.5, 12.0],
})
legacy_feature = compute_feature(
minute=_minute_bars(),
daily=daily,
feature_type="minute_daily_summary",
)
canonical_derived = compute_derived(
"minute_daily_summary",
daily=daily,
minute=_minute_bars(),
)
pd.testing.assert_frame_equal(legacy_feature, canonical_derived)
def test_load_external_feature_module_and_filter_params(tmp_path):
module_path = tmp_path / "external_feature.py"
module_path.write_text(textwrap.dedent('''
import pandas as pd
from pipeline.features.base import BaseFeature
from pipeline.features.registry import register_feature
@register_feature
class ExternalVolumeFeature(BaseFeature):
name = "external_volume_feature"
def __init__(self, scale: float = 1.0):
self.scale = scale
def compute(self, minute: pd.DataFrame, daily=None) -> pd.DataFrame:
out = (
minute.groupby(["symbol_id", "date"], as_index=False)["volume"]
.sum()
.rename(columns={"volume": "scaled_volume"})
)
out["scaled_volume"] *= self.scale
return out
'''))
load_feature_module(str(module_path))
assert "external_volume_feature" in available_features()
instance = get_feature("external_volume_feature", scale=2.0, ignored=99)
assert instance.scale == 2.0
assert not hasattr(instance, "ignored")
result = compute_feature(
minute=_minute_bars(),
feature_type="external_volume_feature",
scale=2.0,
ignored=99,
)
row = result[result["symbol_id"] == "sh600000"].iloc[0]
assert np.isclose(row["scaled_volume"], 800.0)
def test_validate_feature_frame_rejects_missing_keys():
with pytest.raises(ValueError, match="missing required"):
validate_feature_frame(pd.DataFrame({"symbol_id": ["sh600000"], "x": [1.0]}))
def test_validate_feature_frame_rejects_duplicate_keys_after_date_normalization():
with pytest.raises(ValueError, match="duplicate symbol_id,date"):
validate_feature_frame(pd.DataFrame({
"symbol_id": ["sh600000", "sh600000"],
"date": ["2024-01-02", pd.Timestamp("2024-01-02")],
"x": [1.0, 2.0],
}))
def test_validate_feature_frame_rejects_duplicate_columns():
bad = pd.DataFrame(
[["sh600000", pd.Timestamp("2024-01-02"), 1.0, 2.0]],
columns=["symbol_id", "date", "dup", "dup"],
)
with pytest.raises(ValueError, match="duplicate columns"):
validate_feature_frame(bad)
def test_validate_feature_frame_rejects_non_numeric_feature_columns():
with pytest.raises(ValueError, match="numeric"):
validate_feature_frame(pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [pd.Timestamp("2024-01-02")],
"bad": ["not numeric"],
}))
+288
View File
@@ -0,0 +1,288 @@
"""Tests for raw Baostock minute bar download plumbing."""
from pathlib import Path
import numpy as np
import pandas as pd
import pytest
import data.downloader as low_level_downloader
import pipeline.data.downloader as pipeline_downloader
from data.downloader import download_minute_batch
from pipeline.common.schema import MINUTE_BAR_COLUMNS
from pipeline.data.downloader import download_minute_universe
class _FakeResult:
def __init__(self, rows, error_code="0", error_msg=""):
self.rows = rows
self.error_code = error_code
self.error_msg = error_msg
self._idx = -1
def next(self):
self._idx += 1
return self._idx < len(self.rows)
def get_row_data(self):
return self.rows[self._idx]
def test_download_minute_batch_maps_and_parses_baostock_rows(monkeypatch):
rows = [
[
"2024-01-02",
"20240102093500000",
"sh.600000",
"10",
"11",
"9",
"10.5",
"1000",
"10500",
"3",
],
[
"2024-01-02",
"20240102094000000",
"sh.600000",
"10.5",
"12",
"10",
"11",
"2000",
"22000",
"3",
],
]
calls = []
def fake_query(**kwargs):
calls.append(kwargs)
return _FakeResult(rows)
monkeypatch.setattr(low_level_downloader.bs, "login", lambda: None)
monkeypatch.setattr(low_level_downloader.bs, "logout", lambda: None)
monkeypatch.setattr(
low_level_downloader.bs,
"query_history_k_data_plus",
fake_query,
)
[(symbol, df)] = list(
download_minute_batch(
["sh600000"],
"2024-01-02",
"2024-01-02",
frequency=5,
)
)
assert symbol == "sh600000"
assert calls[0]["code"] == "sh.600000"
assert calls[0]["frequency"] == "5"
assert calls[0]["adjustflag"] == "3"
assert df is not None
assert df["datetime"].iloc[0] == pd.Timestamp("2024-01-02 09:35:00")
assert df["time"].tolist() == ["09:35:00", "09:40:00"]
assert (df["frequency"] == "5m").all()
assert np.isclose(df["open"].iloc[0], 10.0)
assert np.isclose(df["vwap"].iloc[0], 10.5)
assert pd.api.types.is_numeric_dtype(df["volume"])
def test_minute_frequency_and_timestamp_parsing_edge_cases():
frequency, label = low_level_downloader._normalize_minute_frequency("15m")
assert (frequency, label) == ("15", "15m")
with pytest.raises(ValueError, match="Unsupported minute frequency"):
low_level_downloader._normalize_minute_frequency("1m")
parsed = low_level_downloader._parse_minute_datetime(
pd.Series(["2024-01-02", "2024-01-02"]),
pd.Series(["0935", "09:40:00"]),
)
assert parsed.tolist() == [
pd.Timestamp("2024-01-02 09:35:00"),
pd.Timestamp("2024-01-02 09:40:00"),
]
def test_download_minute_batch_empty_result_yields_none(monkeypatch):
monkeypatch.setattr(low_level_downloader.bs, "login", lambda: None)
monkeypatch.setattr(low_level_downloader.bs, "logout", lambda: None)
monkeypatch.setattr(
low_level_downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: _FakeResult([]),
)
assert list(download_minute_batch(["sh600000"], "2024-01-02", "2024-01-02")) == [
("sh600000", None)
]
def test_download_minute_batch_non_login_error_and_periodic_relogin(monkeypatch):
responses = [
_FakeResult([], error_code="1", error_msg="bad symbol"),
_FakeResult([]),
]
login_count = 0
logout_count = 0
def fake_login():
nonlocal login_count
login_count += 1
def fake_logout():
nonlocal logout_count
logout_count += 1
monkeypatch.setattr(low_level_downloader.bs, "login", fake_login)
monkeypatch.setattr(low_level_downloader.bs, "logout", fake_logout)
monkeypatch.setattr(
low_level_downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: responses.pop(0),
)
assert list(
download_minute_batch(
["sh600000", "sz000001"],
"2024-01-02",
"2024-01-02",
relogin_every=1,
)
) == [("sh600000", None), ("sz000001", None)]
assert login_count == 2
assert logout_count == 2
def test_download_minute_batch_second_session_loss_yields_none(monkeypatch):
responses = [
_FakeResult([], error_code="10002007", error_msg="用户未登录"),
_FakeResult([], error_code="10002007", error_msg="用户未登录"),
]
monkeypatch.setattr(low_level_downloader.bs, "login", lambda: None)
monkeypatch.setattr(low_level_downloader.bs, "logout", lambda: None)
monkeypatch.setattr(
low_level_downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: responses.pop(0),
)
assert list(download_minute_batch(["sh600000"], "2024-01-02", "2024-01-02")) == [
("sh600000", None)
]
def test_download_minute_batch_rejects_unparsed_timestamps(monkeypatch):
bad_rows = [[
"2024-01-02",
"not-a-time",
"sh.600000",
"10",
"11",
"9",
"10.5",
"1000",
"10500",
"3",
]]
monkeypatch.setattr(low_level_downloader.bs, "login", lambda: None)
monkeypatch.setattr(low_level_downloader.bs, "logout", lambda: None)
monkeypatch.setattr(
low_level_downloader.bs,
"query_history_k_data_plus",
lambda **kwargs: _FakeResult(bad_rows),
)
assert list(download_minute_batch(["sh600000"], "2024-01-02", "2024-01-02")) == [
("sh600000", None)
]
def test_download_minute_universe_writes_frequency_month_partitions(tmp_path, monkeypatch):
minute = pd.DataFrame({
"symbol": ["sh600000", "sh600000"],
"datetime": pd.to_datetime(["2024-01-02 09:35:00", "2024-01-02 09:40:00"]),
"date": pd.to_datetime(["2024-01-02", "2024-01-02"]),
"time": ["09:35:00", "09:40:00"],
"frequency": ["5m", "5m"],
"open": [10.0, 10.5],
"high": [11.0, 12.0],
"low": [9.0, 10.0],
"close": [10.5, 11.0],
"volume": [1000.0, 2000.0],
"amount": [10500.0, 22000.0],
"vwap": [10.5, 11.0],
"adjustflag": ["3", "3"],
})
monkeypatch.setattr(
pipeline_downloader,
"_resolve_universe",
lambda universe, max_symbols=0: pd.DataFrame({
"symbol_id": ["sh600000"],
"symbol_name": ["PF Bank"],
}),
)
def fake_batch(symbols, start, end, frequency=5):
assert symbols == ["sh600000"]
assert frequency == "5"
yield "sh600000", minute
monkeypatch.setattr(pipeline_downloader, "download_minute_batch", fake_batch)
preserved = tmp_path / "toy" / "frequency=15m" / "month=2024-01" / "old.pq"
preserved.parent.mkdir(parents=True)
preserved_minute = minute.copy()
preserved_minute["frequency"] = "15m"
preserved_minute["symbol_id"] = "sh600000"
preserved_minute["symbol_name"] = "PF Bank"
preserved_minute[MINUTE_BAR_COLUMNS].to_parquet(preserved, index=False)
stats = download_minute_universe(
universe="toy",
start_date="2024-01-02",
end_date="2024-01-02",
output_dir=str(tmp_path),
chunk_size=1,
frequency="5",
)
dataset_path = Path(stats["dataset_path"])
assert (dataset_path / "frequency=5m" / "month=2024-01").is_dir()
assert preserved.exists()
out = pd.read_parquet(dataset_path / "frequency=5m")
assert (set(MINUTE_BAR_COLUMNS) - {"frequency"}) <= set(out.columns)
assert set(out["symbol_id"]) == {"sh600000"}
assert set(out["symbol_name"]) == {"PF Bank"}
assert stats["n_rows"] == 2
def test_download_minute_universe_raises_when_all_symbols_empty(tmp_path, monkeypatch):
monkeypatch.setattr(
pipeline_downloader,
"_resolve_universe",
lambda universe, max_symbols=0: pd.DataFrame({
"symbol_id": ["sh600000"],
"symbol_name": ["PF Bank"],
}),
)
monkeypatch.setattr(
pipeline_downloader,
"download_minute_batch",
lambda symbols, start, end, frequency=5: iter([("sh600000", None)]),
)
with pytest.raises(RuntimeError, match="No minute data"):
download_minute_universe(
universe="toy",
start_date="2024-01-02",
end_date="2024-01-02",
output_dir=str(tmp_path),
)
+190
View File
@@ -306,6 +306,47 @@ def test_construct_positions_schema():
assert pos["position_shares"].dtype == np.int64
def test_construct_positions_empty_weights_returns_schema():
data = _make_data(n_days=3)
empty_weights = pd.DataFrame(columns=["symbol_id", "date", "combo_name", "weight"])
pos = construct_positions(
empty_weights,
data,
booksize=1e6,
portfolio_name="empty",
)
assert list(pos.columns) == POSITION_COLUMNS
assert pos.empty
def test_construct_positions_ignores_absent_or_bad_prices():
dates = pd.to_datetime(["2024-01-01", "2024-01-02"])
data = pd.DataFrame([
{"symbol_id": "sh600000", "date": dates[0], "close": np.nan, "isST": 0},
{"symbol_id": "sz000001", "date": dates[0], "close": 20.0, "isST": 0},
{"symbol_id": "sh600000", "date": dates[1], "close": 10.0, "isST": 0},
{"symbol_id": "sz000001", "date": dates[1], "close": 20.0, "isST": 0},
])
weights = pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sh600000", "sz000001"],
"date": [dates[0], dates[0], dates[1], dates[1]],
"combo_name": ["combo"] * 4,
"weight": [1.0, -1.0, 1.0, -1.0],
})
pos = construct_positions(weights, data, booksize=10000.0, portfolio_name="bad_price")
bad_price_rows = pos[
(pos["date"] == dates[0])
& (pos["symbol_id"] == "sh600000")
]
assert bad_price_rows.empty or (bad_price_rows["target_weight"] == 0.0).all()
assert np.isfinite(pos["target_value"]).all()
assert np.isfinite(pos["position_value"]).all()
def test_construct_positions_threads_state_and_closes_absent():
data = _make_data()
weights = _make_weights(data)
@@ -321,6 +362,34 @@ def test_construct_positions_threads_state_and_closes_absent():
assert final.empty or (final["position_shares"] == 0).all()
def test_construct_positions_closes_absent_short_position():
dates = pd.to_datetime(["2024-01-02", "2024-01-03"])
data = pd.DataFrame([
{"symbol_id": sym, "date": d, "close": price, "isST": 0}
for d in dates
for sym, price in (("sh600000", 10.0), ("sz000001", 20.0))
])
weights = pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sz000001"],
"date": [dates[0], dates[0], dates[1]],
"combo_name": ["combo", "combo", "combo"],
"weight": [-1.0, 1.0, 1.0],
})
pos = construct_positions(weights, data, booksize=20000.0, portfolio_name="absent_short")
first_day_short = pos[
(pos["date"] == dates[0])
& (pos["symbol_id"] == "sh600000")
]
final_day_short = pos[
(pos["date"] == dates[1])
& (pos["symbol_id"] == "sh600000")
]
assert (first_day_short["position_shares"] < 0).all()
assert final_day_short.empty or (final_day_short["position_shares"] == 0).all()
def test_construct_positions_carries_book_on_zero_gross(caplog):
dates = pd.to_datetime(["2024-01-01", "2024-01-02"])
symbols = ["sh600000", "sz000001"]
@@ -392,6 +461,42 @@ def test_volume_cap_uses_traded_value():
assert low[0] == -10000.0
def test_constraints_compose_repeatably_regardless_of_order():
n = 1
sl = _slice(
n,
tradestatus=np.array([0.0]),
limit_status=np.array([LimitStatus.UP_LIMIT.value], dtype=np.int8),
amount=np.array([1_000.0]),
price=np.array([10.0]),
)
ctx = TradeContext(np.zeros(n, np.int64), np.array([500]), sl, 1e6)
first_order = ReferenceSimulator(
constraints=[
SuspensionConstraint(),
PriceLimitConstraint(),
VolumeCapConstraint(max_frac=0.1),
],
cost_bps=10,
).fill(ctx)
reversed_order = ReferenceSimulator(
constraints=[
VolumeCapConstraint(max_frac=0.1),
PriceLimitConstraint(),
SuspensionConstraint(),
],
cost_bps=10,
).fill(ctx)
assert first_order.traded_shares.tolist() == [0]
assert first_order.realized_shares.tolist() == [0]
assert first_order.blocked.tolist() == [1]
assert np.array_equal(first_order.traded_shares, reversed_order.traded_shares)
assert np.array_equal(first_order.realized_shares, reversed_order.realized_shares)
assert np.array_equal(first_order.blocked, reversed_order.blocked)
assert np.array_equal(first_order.cost, reversed_order.cost)
# --- ReferenceSimulator ------------------------------------------------------
def test_simulator_next_open_and_blocked_buy_holds_prev():
@@ -474,6 +579,91 @@ def test_simulator_cost_only_on_nonzero_realized_trades():
assert np.isclose(res.cost[1], 50 * 20 * 10 / 1e4)
def test_simulator_short_to_long_flip_trades_full_delta():
dates = pd.to_datetime(["2024-01-01", "2024-01-02", "2024-01-03"])
positions = pd.DataFrame({
"symbol_id": ["sh600000", "sh600000"],
"date": [dates[0], dates[1]],
"portfolio_name": ["flip", "flip"],
"target_weight": [-1.0, 1.0],
"target_value": [-1000.0, 1000.0],
"target_shares": [-100.0, 100.0],
"position_shares": [-100, 100],
"position_value": [-1000.0, 1000.0],
"price": [10.0, 10.0],
})
data = pd.DataFrame({
"symbol_id": ["sh600000"] * 3,
"date": dates,
"open": [10.0, 10.0, 10.0],
"close": [10.0, 10.0, 10.0],
"preclose": [10.0, 10.0, 10.0],
"amount": [1e9, 1e9, 1e9],
"tradestatus": [1, 1, 1],
"isST": [0, 0, 0],
})
fills, _ = ReferenceSimulator().run(positions, data)
by_date = fills.set_index("date")
assert by_date.loc[dates[1], "traded_shares"] == -100
assert by_date.loc[dates[1], "realized_shares"] == -100
assert by_date.loc[dates[2], "prev_shares"] == -100
assert by_date.loc[dates[2], "traded_shares"] == 200
assert by_date.loc[dates[2], "realized_shares"] == 100
def test_simulator_volume_cap_partially_fills_sell():
sl = _slice(1, amount=np.array([10_000.0]), price=np.array([10.0]))
ctx = TradeContext(
np.array([1000], np.int64),
np.array([0], np.int64),
sl,
1_000_000.0,
)
result = ReferenceSimulator(
constraints=[VolumeCapConstraint(max_frac=0.10)]
).fill(ctx)
assert result.traded_shares.tolist() == [-100]
assert result.realized_shares.tolist() == [900]
assert result.blocked.tolist() == [1]
def test_simulator_missing_next_open_has_zero_cost_and_turnover():
dates = pd.to_datetime(["2024-01-01", "2024-01-02"])
positions = pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [dates[0]],
"portfolio_name": ["missing_open"],
"target_weight": [1.0],
"target_value": [1000.0],
"target_shares": [100.0],
"position_shares": [100],
"position_value": [1000.0],
"price": [10.0],
})
data = pd.DataFrame({
"symbol_id": ["sh600000", "sh600000"],
"date": dates,
"open": [10.0, np.nan],
"close": [10.0, 10.0],
"preclose": [10.0, 10.0],
"amount": [1e9, 1e9],
"tradestatus": [1, 1],
"isST": [0, 0],
})
fills, pnl = ReferenceSimulator(cost_bps=10, slippage_bps=5).run(positions, data)
assert fills["traded_shares"].iloc[0] == 100
assert fills["trade_cost"].iloc[0] == 0.0
assert pnl["cost"].iloc[0] == 0.0
assert pnl["turnover"].iloc[0] == 0.0
assert pnl["gross_exposure"].iloc[0] == 1000.0
def test_simple_cost_model_adds_cost_and_slippage_without_price_adjustment():
model = SimpleProportionalCostModel(cost_bps=10, slippage_bps=5)
+98
View File
@@ -0,0 +1,98 @@
"""Malformed parquet/input tests for phase boundary contracts."""
from __future__ import annotations
import pandas as pd
import pytest
from pipeline.alpha.compute import compute_alpha
from pipeline.combo.combine import combine_alphas
from pipeline.derived.compute import validate_derived_frame
from pipeline.portfolio.construct import construct_positions
from pipeline.portfolio.simulator import ReferenceSimulator
from tests.helpers import make_generated_daily_bars
def test_alpha_compute_rejects_daily_data_without_close():
daily = make_generated_daily_bars().drop(columns=["close"])
with pytest.raises(KeyError, match="close"):
compute_alpha(daily, "bad", "reversal", lookback=3)
def test_alpha_feature_path_rejects_duplicate_symbol_dates(tmp_path):
daily = make_generated_daily_bars()
feature = pd.DataFrame({
"symbol_id": ["sh600000", "sh600000"],
"date": ["2024-01-02 09:30:00", "2024-01-02 15:00:00"],
"toy_feature": [1.0, 2.0],
})
feature_path = tmp_path / "duplicate_feature_keys.pq"
feature.to_parquet(feature_path, index=False)
with pytest.raises(ValueError, match="duplicate symbol_id,date"):
compute_alpha(
daily,
"bad_features",
"reversal",
lookback=3,
feature_paths=[str(feature_path)],
)
def test_derived_validation_rejects_bool_value_columns():
derived = pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [pd.Timestamp("2024-01-02")],
"is_good": [True],
})
with pytest.raises(ValueError, match="numeric"):
validate_derived_frame(derived)
def test_combo_combine_rejects_missing_weight_column(tmp_path):
bad_alpha = pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [pd.Timestamp("2024-01-02")],
"alpha_name": ["bad"],
})
bad_alpha_path = tmp_path / "bad_alpha.pq"
bad_alpha.to_parquet(bad_alpha_path, index=False)
with pytest.raises(KeyError, match="weight"):
combine_alphas([str(bad_alpha_path)], "bad_combo")
def test_portfolio_build_rejects_weights_without_symbol_id():
daily = make_generated_daily_bars()
bad_weights = pd.DataFrame({
"date": [pd.Timestamp("2024-01-02")],
"combo_name": ["bad"],
"weight": [1.0],
})
with pytest.raises(KeyError, match="symbol_id"):
construct_positions(
bad_weights,
daily,
booksize=1_000_000.0,
portfolio_name="bad_portfolio",
)
def test_portfolio_simulate_rejects_positions_without_position_shares():
daily = make_generated_daily_bars()
bad_positions = pd.DataFrame({
"symbol_id": ["sh600000"],
"date": [pd.Timestamp("2024-01-02")],
"portfolio_name": ["bad"],
"target_weight": [1.0],
"target_value": [1000.0],
"target_shares": [100.0],
"position_value": [1000.0],
"price": [10.0],
})
with pytest.raises(KeyError, match="position_shares"):
ReferenceSimulator().run(bad_positions, daily)
+92
View File
@@ -0,0 +1,92 @@
"""Offline tests for baostock-backed universe helpers."""
from __future__ import annotations
import pandas as pd
import data.universe as universe
class _FakeResult:
def __init__(self, rows, fields=None):
self.rows = rows
self.fields = fields or ["code", "name", "date"]
self._idx = -1
def next(self):
self._idx += 1
return self._idx < len(self.rows)
def get_row_data(self):
return self.rows[self._idx]
def test_index_constituent_helpers_normalize_dotted_codes(monkeypatch):
calls: list[str] = []
monkeypatch.setattr(universe.bs, "login", lambda: calls.append("login"))
monkeypatch.setattr(universe.bs, "logout", lambda: calls.append("logout"))
monkeypatch.setattr(
universe.bs,
"query_hs300_stocks",
lambda: _FakeResult([
["sh.600000", "浦发银行", "2024-01-12"],
["sz.000001", "平安银行", "2024-01-12"],
]),
)
monkeypatch.setattr(
universe.bs,
"query_zz500_stocks",
lambda: _FakeResult([
["sh.600006", "东风汽车", "2024-01-12"],
]),
)
hs300 = universe.get_hs300_stocks()
zz500 = universe.get_zz500_stocks()
assert calls == ["login", "logout", "login", "logout"]
assert hs300["code"].tolist() == ["sh600000", "sz000001"]
assert zz500["code"].tolist() == ["sh600006"]
assert hs300["name"].tolist() == ["浦发银行", "平安银行"]
def test_get_all_stocks_walks_back_and_filters_to_listed_a_shares(monkeypatch):
fields = ["code", "tradeStatus", "code_name"]
responses = [
_FakeResult([], fields=fields),
_FakeResult(
[
["sh.600000", "1", "浦发银行"],
["sh.688001", "1", "华兴源创"],
["sz.000001", "1", "平安银行"],
["sz.300750", "1", "宁德时代"],
["sz.399001", "1", "深证成指"],
["sz.200001", "1", "深物业B"],
["bj.430047", "1", "北交所样本"],
],
fields=fields,
),
]
query_days: list[str] = []
monkeypatch.setattr(universe.bs, "login", lambda: None)
monkeypatch.setattr(universe.bs, "logout", lambda: None)
def fake_query_all_stock(day):
query_days.append(day)
return responses.pop(0)
monkeypatch.setattr(universe.bs, "query_all_stock", fake_query_all_stock)
result = universe.get_all_stocks("2024-01-07")
assert query_days == ["2024-01-07", "2024-01-06"]
assert result.columns.tolist() == ["code", "name"]
assert result["code"].tolist() == [
"sh600000",
"sh688001",
"sz000001",
"sz300750",
]
assert result["name"].tolist() == ["浦发银行", "华兴源创", "平安银行", "宁德时代"]
+508
View File
@@ -0,0 +1,508 @@
"""Verbose offline checks for the daily research workflow."""
from __future__ import annotations
from pathlib import Path
import numpy as np
import pandas as pd
from pipeline.alpha.compute import compute_alpha
from pipeline.combo.combine import combine_alphas
from pipeline.common.schema import (
ALPHA_COLUMNS,
COMBO_COLUMNS,
FILL_COLUMNS,
PNL_COLUMNS,
POSITION_COLUMNS,
)
from pipeline.portfolio.constraints import (
PriceLimitConstraint,
SuspensionConstraint,
VolumeCapConstraint,
)
from pipeline.portfolio.construct import construct_positions
from pipeline.portfolio.research import evaluate_portfolio
from pipeline.portfolio.simulator import ReferenceSimulator
from tests.helpers import (
GENERATED_SYMBOLS,
generated_sessions,
make_generated_alpha_weights,
make_generated_combo_weights,
make_generated_daily_bars,
)
FIXTURE_PATH = Path(__file__).parent / "fixtures" / "daily_bars_real_2024_01_sample.pq"
def _assert_sorted_by_symbol_date(frame: pd.DataFrame) -> None:
expected = frame.sort_values(["symbol_id", "date"]).reset_index(drop=True)
pd.testing.assert_frame_equal(frame.reset_index(drop=True), expected)
def _assert_metric_dict_is_finite(metrics: dict[str, float]) -> None:
for key in (
"cumulative_return",
"sharpe_annual",
"turnover_annual",
"max_drawdown",
"hit_rate",
"n_dates",
):
assert key in metrics
assert np.isfinite(metrics[key])
assert "ic" not in metrics
assert "rank_ic" not in metrics
assert "ir" not in metrics
def test_tiny_workflow_golden_outputs_are_stable(tmp_path):
dates = pd.to_datetime(["2024-01-02", "2024-01-03", "2024-01-04"])
daily_bars = pd.DataFrame([
{
"symbol_id": "sh600000",
"symbol_name": "A",
"date": dates[0],
"open": 10.0,
"high": 10.0,
"low": 10.0,
"close": 10.0,
"preclose": 10.0,
"volume": 1_000_000.0,
"amount": 10_000_000.0,
"vwap": 10.0,
"turn": 1.0,
"pctChg": 0.0,
"tradestatus": 1,
"isST": 0,
"peTTM": 1.0,
"pbMRQ": 1.0,
"psTTM": 1.0,
"pcfNcfTTM": 1.0,
},
{
"symbol_id": "sz000001",
"symbol_name": "B",
"date": dates[0],
"open": 20.0,
"high": 20.0,
"low": 20.0,
"close": 20.0,
"preclose": 20.0,
"volume": 1_000_000.0,
"amount": 20_000_000.0,
"vwap": 20.0,
"turn": 1.0,
"pctChg": 0.0,
"tradestatus": 1,
"isST": 0,
"peTTM": 1.0,
"pbMRQ": 1.0,
"psTTM": 1.0,
"pcfNcfTTM": 1.0,
},
{
"symbol_id": "sh600000",
"symbol_name": "A",
"date": dates[1],
"open": 10.0,
"high": 12.0,
"low": 10.0,
"close": 12.0,
"preclose": 10.0,
"volume": 1_000_000.0,
"amount": 10_000_000.0,
"vwap": 10.0,
"turn": 1.0,
"pctChg": 20.0,
"tradestatus": 1,
"isST": 0,
"peTTM": 1.0,
"pbMRQ": 1.0,
"psTTM": 1.0,
"pcfNcfTTM": 1.0,
},
{
"symbol_id": "sz000001",
"symbol_name": "B",
"date": dates[1],
"open": 20.0,
"high": 20.0,
"low": 18.0,
"close": 18.0,
"preclose": 20.0,
"volume": 1_000_000.0,
"amount": 20_000_000.0,
"vwap": 20.0,
"turn": 1.0,
"pctChg": -10.0,
"tradestatus": 1,
"isST": 0,
"peTTM": 1.0,
"pbMRQ": 1.0,
"psTTM": 1.0,
"pcfNcfTTM": 1.0,
},
{
"symbol_id": "sh600000",
"symbol_name": "A",
"date": dates[2],
"open": 12.0,
"high": 13.0,
"low": 12.0,
"close": 13.0,
"preclose": 12.0,
"volume": 1_000_000.0,
"amount": 12_000_000.0,
"vwap": 12.0,
"turn": 1.0,
"pctChg": 8.33,
"tradestatus": 1,
"isST": 0,
"peTTM": 1.0,
"pbMRQ": 1.0,
"psTTM": 1.0,
"pcfNcfTTM": 1.0,
},
{
"symbol_id": "sz000001",
"symbol_name": "B",
"date": dates[2],
"open": 18.0,
"high": 21.0,
"low": 18.0,
"close": 21.0,
"preclose": 18.0,
"volume": 1_000_000.0,
"amount": 18_000_000.0,
"vwap": 18.0,
"turn": 1.0,
"pctChg": 16.67,
"tradestatus": 1,
"isST": 0,
"peTTM": 1.0,
"pbMRQ": 1.0,
"psTTM": 1.0,
"pcfNcfTTM": 1.0,
},
])
alpha = pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sh600000", "sz000001"],
"date": [dates[0], dates[0], dates[1], dates[1]],
"alpha_name": ["gold_alpha"] * 4,
"weight": [1.0, -1.0, -1.0, 1.0],
})
alpha_path = tmp_path / "gold_alpha.pq"
alpha.to_parquet(alpha_path, index=False)
combo = combine_alphas([str(alpha_path)], "gold_combo")
positions = construct_positions(combo, daily_bars, booksize=20_000.0, portfolio_name="gold_port")
fills, pnl = ReferenceSimulator().run(positions, daily_bars)
expected_combo = pd.DataFrame({
"symbol_id": ["sh600000", "sh600000", "sz000001", "sz000001"],
"date": [dates[0], dates[1], dates[0], dates[1]],
"combo_name": ["gold_combo"] * 4,
"weight": [1.0, -1.0, -1.0, 1.0],
})
expected_positions = pd.DataFrame({
"symbol_id": ["sh600000", "sh600000", "sz000001", "sz000001"],
"date": [dates[0], dates[1], dates[0], dates[1]],
"portfolio_name": ["gold_port"] * 4,
"target_weight": [0.5, -0.5, -0.5, 0.5],
"target_value": [10000.0, -10000.0, -10000.0, 10000.0],
"target_shares": [1000.0, -10000.0 / 12.0, -500.0, 10000.0 / 18.0],
"position_shares": [1000, -833, -500, 556],
"position_value": [10000.0, -9996.0, -10000.0, 10008.0],
"price": [10.0, 12.0, 20.0, 18.0],
})
expected_fills = pd.DataFrame({
"symbol_id": ["sh600000", "sz000001", "sh600000", "sz000001"],
"date": [dates[1], dates[1], dates[2], dates[2]],
"portfolio_name": ["gold_port"] * 4,
"prev_shares": [0, 0, 1000, -500],
"target_shares": [1000, -500, -833, 556],
"traded_shares": [1000, -500, -1833, 1056],
"realized_shares": [1000, -500, -833, 556],
"blocked": [0, 0, 0, 0],
"trade_cost": [0.0, 0.0, 0.0, 0.0],
})
expected_pnl = pd.DataFrame({
"date": [dates[1], dates[2]],
"portfolio_name": ["gold_port", "gold_port"],
"gross_exposure": [21000.0, 22505.0],
"net_exposure": [3000.0, 847.0],
"pnl": [3000.0, 835.0],
"cost": [0.0, 0.0],
"turnover": [1.0, 2.0502],
"n_positions": [2, 2],
})
pd.testing.assert_frame_equal(combo, expected_combo)
pd.testing.assert_frame_equal(positions, expected_positions)
pd.testing.assert_frame_equal(fills, expected_fills)
pd.testing.assert_frame_equal(pnl, expected_pnl)
def test_generated_alpha_combo_portfolio_execution_workflow(tmp_path):
daily_bars = make_generated_daily_bars()
computed_alpha = compute_alpha(
data=daily_bars,
alpha_name="generated_reversal_3d",
alpha_type="reversal",
lookback=3,
)
assert list(computed_alpha.columns) == ALPHA_COLUMNS
assert not computed_alpha.empty
assert set(computed_alpha["symbol_id"]).issubset(set(GENERATED_SYMBOLS))
assert computed_alpha["date"].min() > daily_bars["date"].min()
assert computed_alpha["weight"].notna().all()
assert computed_alpha["weight"].abs().sum() > 0.0
assert {"ic", "rank_ic", "ir"}.isdisjoint(computed_alpha.columns)
_assert_sorted_by_symbol_date(computed_alpha)
alpha_a = make_generated_alpha_weights("alpha_a", zero_date_index=2)
alpha_b = make_generated_alpha_weights(
"alpha_b",
scale=0.5,
offset=0.25,
zero_date_index=2,
)
alpha_a_path = tmp_path / "alpha_a.pq"
alpha_b_path = tmp_path / "alpha_b.pq"
alpha_a.to_parquet(alpha_a_path, index=False)
alpha_b.to_parquet(alpha_b_path, index=False)
identity_combo = combine_alphas([str(alpha_a_path)], "identity_combo")
assert list(identity_combo.columns) == COMBO_COLUMNS
assert (identity_combo["combo_name"] == "identity_combo").all()
pd.testing.assert_frame_equal(
identity_combo[["symbol_id", "date", "weight"]],
alpha_a[["symbol_id", "date", "weight"]],
)
equal_combo = combine_alphas([str(alpha_a_path), str(alpha_b_path)], "equal_combo")
expected_equal_weights = (
pd.concat([alpha_a, alpha_b], ignore_index=True)
.groupby(["symbol_id", "date"], as_index=False)["weight"]
.mean()
.sort_values(["symbol_id", "date"])
.reset_index(drop=True)
)
pd.testing.assert_frame_equal(
equal_combo[["symbol_id", "date", "weight"]],
expected_equal_weights,
)
portfolio_weights = make_generated_combo_weights("workflow_combo", zero_date_index=2)
positions = construct_positions(
weights_df=portfolio_weights,
data_df=daily_bars,
booksize=2_000_000.0,
portfolio_name="workflow_portfolio",
)
assert list(positions.columns) == POSITION_COLUMNS
assert not positions.empty
assert (positions["portfolio_name"] == "workflow_portfolio").all()
assert pd.api.types.is_integer_dtype(positions["position_shares"])
assert np.allclose(
positions["position_value"],
positions["position_shares"].astype(float) * positions["price"].fillna(0.0),
)
target_gross_by_date = positions.groupby("date")["target_weight"].apply(lambda s: s.abs().sum())
nonzero_target_dates = target_gross_by_date[target_gross_by_date > 0.0]
assert np.allclose(nonzero_target_dates, 1.0)
nonzero_share_counts = positions.loc[positions["position_shares"] != 0, "position_shares"].abs()
assert (nonzero_share_counts >= 100).all()
zero_gross_date = generated_sessions(10)[2]
previous_date = generated_sessions(10)[1]
zero_gross_positions = positions[positions["date"] == zero_gross_date].set_index("symbol_id")
previous_positions = positions[positions["date"] == previous_date].set_index("symbol_id")
common_symbols = zero_gross_positions.index.intersection(previous_positions.index)
assert not common_symbols.empty
assert (zero_gross_positions.loc[common_symbols, "target_weight"] == 0.0).all()
pd.testing.assert_series_equal(
zero_gross_positions.loc[common_symbols, "position_shares"],
previous_positions.loc[common_symbols, "position_shares"],
check_names=False,
)
simulator = ReferenceSimulator(
constraints=[
SuspensionConstraint(),
PriceLimitConstraint(),
VolumeCapConstraint(max_frac=0.02),
],
cost_bps=5,
slippage_bps=5,
)
fills, pnl = simulator.run(positions, daily_bars)
assert list(fills.columns) == FILL_COLUMNS
assert list(pnl.columns) == PNL_COLUMNS
assert not fills.empty
assert not pnl.empty
assert (fills["realized_shares"] == fills["prev_shares"] + fills["traded_shares"]).all()
assert fills["blocked"].sum() > 0
fill_prices = fills.merge(
daily_bars[["symbol_id", "date", "open"]],
on=["symbol_id", "date"],
how="left",
validate="many_to_one",
)
expected_trade_cost = (
fill_prices["traded_shares"].abs()
* fill_prices["open"].fillna(0.0)
* 10
/ 10_000
)
assert np.allclose(fill_prices["trade_cost"], expected_trade_cost)
cost_by_date = fills.groupby("date")["trade_cost"].sum()
assert np.allclose(
pnl.set_index("date")["cost"],
cost_by_date.reindex(pnl["date"], fill_value=0.0),
)
booksize_used_by_simulator = positions.groupby("date")["target_value"].apply(lambda s: s.abs().sum()).max()
traded_value_by_date = (
fill_prices.assign(traded_value=fill_prices["traded_shares"].abs() * fill_prices["open"])
.groupby("date")["traded_value"]
.sum()
)
assert np.allclose(
pnl.set_index("date")["turnover"],
traded_value_by_date.reindex(pnl["date"], fill_value=0.0) / booksize_used_by_simulator,
)
metrics = evaluate_portfolio(positions, daily_bars)
_assert_metric_dict_is_finite(metrics)
def test_generated_workflow_outputs_keep_parquet_schema_contracts(tmp_path):
daily_bars = make_generated_daily_bars(n_sessions=10, include_missing=False)
alpha = compute_alpha(
data=daily_bars,
alpha_name="schema_reversal",
alpha_type="reversal",
lookback=3,
)
alpha_path = tmp_path / "schema_reversal.pq"
alpha.to_parquet(alpha_path, index=False)
combo = combine_alphas([str(alpha_path)], "schema_combo")
positions = construct_positions(
weights_df=combo,
data_df=daily_bars,
booksize=1_000_000.0,
portfolio_name="schema_portfolio",
)
fills, pnl = ReferenceSimulator(cost_bps=5, slippage_bps=5).run(positions, daily_bars)
assert list(alpha.columns) == ALPHA_COLUMNS
assert pd.api.types.is_object_dtype(alpha["symbol_id"])
assert pd.api.types.is_datetime64_any_dtype(alpha["date"])
assert pd.api.types.is_object_dtype(alpha["alpha_name"])
assert pd.api.types.is_float_dtype(alpha["weight"])
assert not alpha.isna().any().any()
assert np.isfinite(alpha["weight"]).all()
assert list(combo.columns) == COMBO_COLUMNS
assert pd.api.types.is_object_dtype(combo["symbol_id"])
assert pd.api.types.is_datetime64_any_dtype(combo["date"])
assert pd.api.types.is_object_dtype(combo["combo_name"])
assert pd.api.types.is_float_dtype(combo["weight"])
assert not combo.isna().any().any()
assert np.isfinite(combo["weight"]).all()
assert list(positions.columns) == POSITION_COLUMNS
assert pd.api.types.is_integer_dtype(positions["position_shares"])
assert pd.api.types.is_datetime64_any_dtype(positions["date"])
assert not positions.isna().any().any()
position_numeric_columns = [
"target_weight",
"target_value",
"target_shares",
"position_value",
"price",
]
assert np.isfinite(positions[position_numeric_columns]).all().all()
assert list(fills.columns) == FILL_COLUMNS
assert pd.api.types.is_integer_dtype(fills["prev_shares"])
assert pd.api.types.is_integer_dtype(fills["target_shares"])
assert pd.api.types.is_integer_dtype(fills["traded_shares"])
assert pd.api.types.is_integer_dtype(fills["realized_shares"])
assert pd.api.types.is_integer_dtype(fills["blocked"])
assert not fills.isna().any().any()
assert np.isfinite(fills["trade_cost"]).all()
assert list(pnl.columns) == PNL_COLUMNS
assert pd.api.types.is_integer_dtype(pnl["n_positions"])
assert not pnl.isna().any().any()
pnl_numeric_columns = [
"gross_exposure",
"net_exposure",
"pnl",
"cost",
"turnover",
]
assert np.isfinite(pnl[pnl_numeric_columns]).all().all()
def test_frozen_real_fixture_runs_high_level_workflow(tmp_path):
real_daily_bars = pd.read_parquet(FIXTURE_PATH)
assert real_daily_bars.shape == (36, 19)
assert set(real_daily_bars["symbol_id"]) == set(GENERATED_SYMBOLS)
assert real_daily_bars["date"].min() == pd.Timestamp("2024-01-02")
assert real_daily_bars["date"].max() == pd.Timestamp("2024-01-12")
assert real_daily_bars.groupby("date")["symbol_id"].nunique().eq(4).all()
reversal_alpha = compute_alpha(
data=real_daily_bars,
alpha_name="real_reversal_3d",
alpha_type="reversal",
lookback=3,
)
reversal_vol_alpha = compute_alpha(
data=real_daily_bars,
alpha_name="real_reversal_vol_3d",
alpha_type="reversal_vol",
lookback=3,
vol_window=3,
)
reversal_path = tmp_path / "real_reversal.pq"
reversal_vol_path = tmp_path / "real_reversal_vol.pq"
reversal_alpha.to_parquet(reversal_path, index=False)
reversal_vol_alpha.to_parquet(reversal_vol_path, index=False)
combo = combine_alphas([str(reversal_path), str(reversal_vol_path)], "real_equal_combo")
positions = construct_positions(
weights_df=combo,
data_df=real_daily_bars,
booksize=1_000_000.0,
portfolio_name="real_fixture_portfolio",
)
fills, pnl = ReferenceSimulator(cost_bps=5, slippage_bps=5).run(positions, real_daily_bars)
metrics = evaluate_portfolio(positions, real_daily_bars)
assert not reversal_alpha.empty
assert not reversal_vol_alpha.empty
assert not combo.empty
assert not positions.empty
assert not fills.empty
assert not pnl.empty
assert np.isfinite(combo["weight"]).all()
assert np.isfinite(positions["target_weight"]).all()
assert np.isfinite(pnl[["gross_exposure", "net_exposure", "pnl", "cost", "turnover"]]).all().all()
_assert_metric_dict_is_finite(metrics)
Generated
+118 -1
View File
@@ -299,6 +299,7 @@ backtrader = [
[package.dev-dependencies]
dev = [
{ name = "coverage" },
{ name = "pytest" },
]
@@ -315,7 +316,10 @@ requires-dist = [
provides-extras = ["backtrader"]
[package.metadata.requires-dev]
dev = [{ name = "pytest", specifier = ">=7.0.0" }]
dev = [
{ name = "coverage", specifier = ">=7.14.1" },
{ name = "pytest", specifier = ">=7.0.0" },
]
[[package]]
name = "click"
@@ -498,6 +502,119 @@ wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0c/58/bd257695f39d05594ca4ad60df5bcb7e32247f9951fd09a9b8edb82d1daa/contourpy-1.3.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:3d1a3799d62d45c18bafd41c5fa05120b96a28079f2393af559b843d1a966a77", size = 225315, upload-time = "2025-07-26T12:02:58.801Z" },
]
[[package]]
name = "coverage"
version = "7.14.1"
source = { registry = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/" }
sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/54/fd/0ab2772530e946e1be1abd0bc09e647ec9b02e88f0867857601fefca8953/coverage-7.14.1.tar.gz", hash = "sha256:30c08f7d90415aa98b3c990385dea2939b0da55f38515e5b369b83655f8523be", size = 920132, upload-time = "2026-05-26T20:41:36.783Z" }
wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/92/69/0d2ef01ff4b8fcecd4cba920d11e92fa4f96ae412441d3b56a90a258e69b/coverage-7.14.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:3e3680291c4a1d0dadfa84a2c459576a4af5133abb617905714339a0c73138cf", size = 219722, upload-time = "2026-05-26T20:38:14.002Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f8/ae/9afdeaa31b9d9ce98124b6abf8bb49119bf71aecae04f8567c189d91299f/coverage-7.14.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:a5274669f37f2343635a347b91a60777621341ab3378e9c6ac9335eee704bddf", size = 220240, upload-time = "2026-05-26T20:38:17.424Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/51/69/c998589871df7ea7dba865cc5ee32b5a3e1d47ba6c68ef91104c7c46fa5e/coverage-7.14.1-cp310-cp310-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:cfe5a5fec635799ef33428f1e5e61bafa45a92a96190ba731561ba558ccc214d", size = 246981, upload-time = "2026-05-26T20:38:19.266Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fc/10/1c7d04c13040dac531d21b712bbe08f902e6dd9b58f5d77875c4d030f8f2/coverage-7.14.1-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:62a9f70b52e0b5a95cfef4a5c5641b06983cadc5e538a3feeb5c00211f523ac2", size = 248812, upload-time = "2026-05-26T20:38:20.75Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c1/65/2a38a4607ef27cadcfbcee034dba5830ae2569f90144a0f4c7dbf47d30b0/coverage-7.14.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c18ebc343e15be53049b3a2dce38fe82d58f37e20ab9094b3a39c0aa4f6bb47", size = 250675, upload-time = "2026-05-26T20:38:22.159Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c9/a2/a446ed9752a4a59b79e0fb6cbb319f6facb2183045c0725462625e66f87e/coverage-7.14.1-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b84ffdf877644e7096aa936991efeed873f7f3df57b9cd001312b7668ab08550", size = 252590, upload-time = "2026-05-26T20:38:23.63Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/9e/fd/e81fbd7ba752365546e9842b1cbdaad3d6919d2a522c590aef16a281ec5e/coverage-7.14.1-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e854312c4103f2ad4c0dc023b69b77ebfd2c89db5f86c4c94dc2353f9a92167e", size = 247691, upload-time = "2026-05-26T20:38:25.057Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/53/35/f3c26fdaae9ea937d154ca4d372e5ea0a4167ff70d36c6074ac2eacb2f83/coverage-7.14.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:c643734307300234fafa36bf2a040a7235f8f177ea1fd6ec1423aea6fb7b929f", size = 248716, upload-time = "2026-05-26T20:38:26.406Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2e/14/940b6c49551fd343e8507ee2b0ba7af5d0aa04ed5bf768285cb7c72a9884/coverage-7.14.1-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:84ac9499e48700399a5dd0ea7085b5091961fec52c68d66b4ec0d3cf7f4441b1", size = 246721, upload-time = "2026-05-26T20:38:28.282Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/aa/2c/40fc0634186c28292a662dff578866b3913983d6c375a3c2a74020938719/coverage-7.14.1-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:7f02d09f70776579b926d889a4c9c235070a1f47c40458aeaca563fae5acfdb5", size = 250533, upload-time = "2026-05-26T20:38:29.753Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/de/e3/2c26bf1e811f9df991ff2a9bdddebdd13ee0665d564df7d05979f9146297/coverage-7.14.1-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:ce66d8e46da2bb5ee313a745cbd2e391d319176c1f7a9451bfcd3a2fb920859b", size = 246990, upload-time = "2026-05-26T20:38:31.516Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a8/b0/060260ef56bd92363ebdce0c7095ce422b06e69aae71828efeca473ab1ca/coverage-7.14.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:c912c259304cfb5ee584481cfb7ce1ff932b4d61e6c9140b8f19cb7b5ed82332", size = 247593, upload-time = "2026-05-26T20:38:33.065Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/63/f3/501502046efeb0d6d94b5ca54941d95f1184183dd6bdb7f283985783bb4a/coverage-7.14.1-cp310-cp310-win32.whl", hash = "sha256:1238cb94638e610e972c60dac68e813f868dc7d6e982535270558443058d9d59", size = 222330, upload-time = "2026-05-26T20:38:35.36Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a0/5d/1bf99f2c558f128faf7906817ccbdb576ba815d3b41ce2ac1719b70a3663/coverage-7.14.1-cp310-cp310-win_amd64.whl", hash = "sha256:fc459e5d73be2d6332fcfe8dbf3d8994671fe33c700f4565988ecfa511547253", size = 223261, upload-time = "2026-05-26T20:38:37.196Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7d/d7/477ad149490e6cb849f28abea1dabb9c823cea72e7500c81b4240ce619c0/coverage-7.14.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:478b5bcd63c2e1357c5c7e16c070690df7b07f676b1c114d7b93e533c664309f", size = 219848, upload-time = "2026-05-26T20:38:38.715Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/91/82/a5eb47257c50601bb7b9a9d2857c67b7a3a85ad74180eb2c98bb1fbe0ce5/coverage-7.14.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a24a81f9715ee42ef59a316cc11611c98fe23920f7c81861315c9f3ff4a230f4", size = 220354, upload-time = "2026-05-26T20:38:40.232Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/43/8b/78419b5391a5cb706b6544390507e469d83ffc9a8248b02c4011aceb9365/coverage-7.14.1-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:196a13319ad88d6d8ef5ab489ec4f44ddde2143c0c7d5b27786f6c3ffd56a7e1", size = 250771, upload-time = "2026-05-26T20:38:41.782Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/77/63/e77aaacd491182210d639636b7a8bba23ffffa9b82aa3762da9431855fa9/coverage-7.14.1-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:3d452fd08b5c72c5167c93e6867b5c08500bd40f2a21e1e854a500550b6cc36f", size = 252683, upload-time = "2026-05-26T20:38:43.305Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/65/1c/a022e3cfbec2ac241640003cb3a817e161d9c7f5aa9b49173756cdc03204/coverage-7.14.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:23bf7fa51ac02e07fc7c96849b82946da47ae862dc8f86d183b2a4864fc38129", size = 254791, upload-time = "2026-05-26T20:38:45.361Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/61/d6/967e408aca4c1ceb88cb0cc677169110ae7f5995fb5eaf5fb1f5a1bb8f5d/coverage-7.14.1-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bcaa50684dcaadfa599ac48f81103c756d791cfd85c97203d2217c593d48b860", size = 256748, upload-time = "2026-05-26T20:38:46.91Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b8/be/869188f7fe28638078ec479331ace6dc5f7b40b7153eb616f47ab79404d8/coverage-7.14.1-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4ea1c034f95c9b056e856b794630b17f9fa3d57e4800ff1e503d3be0f9c9078c", size = 250907, upload-time = "2026-05-26T20:38:48.493Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/07/aa/adb7d3b4278d690e68703abcd76ab1b948242e3668d921711551b78f9ddb/coverage-7.14.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c7e057326434e441306226fbeb5d1aaf14a2637efe97ba668306635835f32ad7", size = 252483, upload-time = "2026-05-26T20:38:50.074Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/43/61/331c74103c62dcb0c4b9b3a0de9a61aca016208b0a90f109592a9f9ecc28/coverage-7.14.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:59baf88468dbc8d63b1887afd92bda52e40bb1561696e5819670601403810cec", size = 250545, upload-time = "2026-05-26T20:38:51.613Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f6/b6/c5dae3c104d89be04828f61810e6b3473825482e4c288cc4ed04553e08ae/coverage-7.14.1-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:d34d75f892b3ab73ba11cab5442cce7b3e168fd64162b16f0e1e0d09c508edef", size = 254310, upload-time = "2026-05-26T20:38:53.503Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ad/a1/2b9d5863e3b83c01ad8199e3c597802fbb3a9dc90b058885804c20296d31/coverage-7.14.1-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:3a56abc20a472baf0304c455721bc601477440d28ecfde8a03dde79ede07e0df", size = 250266, upload-time = "2026-05-26T20:38:55.414Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7f/5e/0e511fbdb269359be26fe678a1c3fa1f2aa2a01573cc3f54268c8d6d4797/coverage-7.14.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:6a3cb83d1552c0cd1b4906655b6a33fd4a8473229633a901c6b73bf86914dee9", size = 251174, upload-time = "2026-05-26T20:38:57.141Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/85/10/e55307b622b3dd9671cb321824502dc10f93e72f2802b9946159a8edadeb/coverage-7.14.1-cp311-cp311-win32.whl", hash = "sha256:10274a1fbeb8ec5d72966e17bb198a3104257aca4ac09d98667c5f8aca8c8548", size = 222354, upload-time = "2026-05-26T20:38:58.727Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/71/cf/107421693cfb71e4f1ca5bf70443f64d4161878068d07a3e51c7ad21d17b/coverage-7.14.1-cp311-cp311-win_amd64.whl", hash = "sha256:87ebdf787d4888e3f3f2d523eadc6e18c6d18c6d0eb173801a189641627fb37e", size = 223290, upload-time = "2026-05-26T20:39:00.413Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b8/1d/3e3644585eb29e9dafefb19555078529a4d7cce12bd21929664eea989277/coverage-7.14.1-cp311-cp311-win_arm64.whl", hash = "sha256:dd34767fa19848d35659ffc0a75314f58c7af3f1cd87ec521e8292a1238398a3", size = 221953, upload-time = "2026-05-26T20:39:02.159Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/3d/b7/bdbb725ba02c5b42825b200c940f38b7a54fcad24627b7192f78f8110d76/coverage-7.14.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a06c76364a9360e33d6d23769aefdf7f66f38e2ffb60ceb1baaa4989d83b695c", size = 220022, upload-time = "2026-05-26T20:39:03.702Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/72/81/fdc0898a55c6219223291ec1a1fe89966ef212ce82276aa0899df84b5de0/coverage-7.14.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fad54e871165f6ec2f536063ac74c3104508a12963e64072ba44bd822de52b0c", size = 220379, upload-time = "2026-05-26T20:39:05.381Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/de/72/de048c4a25e13bce59ac6a339351c10bdf2515e07459afcdaf04dc3143a2/coverage-7.14.1-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:84b535f00655ecafe1d929d1fb00ed5d6fa3051ea643ab2c161a3887b86f294b", size = 251888, upload-time = "2026-05-26T20:39:07.367Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/28/30/300c343f68beb9d4cbb64ec81e58c5b6b80b56927f72d2b38654ac26e013/coverage-7.14.1-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:6b6b0853b895fe0e98cbfc580d1ec3393d9302b4b1e96a77b3f5c91fdab899e6", size = 254624, upload-time = "2026-05-26T20:39:09.037Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b1/ed/7b25642496e8170b6bac14adce00537c6e5fa2d586159401a4de3e8b49e6/coverage-7.14.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:442cc9c952b2df400cda54bb04ab87330cf2cd08a8692cbbea36773531eb6f37", size = 255739, upload-time = "2026-05-26T20:39:10.889Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7f/a2/abd210b8c4e29c24e4624916db97bb519097a91034aaeb767f937e7da794/coverage-7.14.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8270544c361ed405a27a060dbc9ed2c124b084d96dfdc2d9a2510482aef981ad", size = 257998, upload-time = "2026-05-26T20:39:12.722Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7f/24/7c50beed3792fe62f6ce0545c6686ce83379719e2c0276179333d97eae92/coverage-7.14.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:48b283b1dd6372e8de2a7a9a4c4d5dc06f4d4fd209b876f3c88a7a205a0c8f84", size = 252296, upload-time = "2026-05-26T20:39:14.259Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/15/05/0f874628ebcbfc77ead559ff210281ef06a97db08481832e7dd39274a135/coverage-7.14.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5b0c99ba93a07d56f6df340bb79be53202a082b2fdb81bfe6190b741a3470d54", size = 253658, upload-time = "2026-05-26T20:39:15.923Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/99/6f/ca6ad067364b337ef997802115e7ecad2abd2248b05471464b0dea02b4d4/coverage-7.14.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:e471bc5769ff073b058cfadb0d736b56ce067c8560eabeb0da88462df98c23e7", size = 251803, upload-time = "2026-05-26T20:39:17.537Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c0/30/b9b4d377cd9f40baf228068f5a81faf8450c6228503011bd499708483a50/coverage-7.14.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:f497a1ea81d4cd7c10ddcaa685135b9aabd291af3d55775a9ddf3cb7a364cdd9", size = 255873, upload-time = "2026-05-26T20:39:19.414Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/3c/21/7c721a9e5e6bb88547d30a787aefb97512d3f54c1324c7488d9b3743f7f9/coverage-7.14.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:2222be86d0b54f5dd5a38f45f17f315f737245e857bf0bdedc70734f84a13c02", size = 251372, upload-time = "2026-05-26T20:39:21.169Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/9d/8c/f8ae5a2200130e1503cd7661a6cd3b2b7bacef98277fbf3571fb13f8b766/coverage-7.14.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:85e85586565842f6932abebd4c18bcb1074223dc0b3576e7d173ca710622813a", size = 253245, upload-time = "2026-05-26T20:39:23.097Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/34/62/70a9024672a5f6910517d9628c52c9afbdd3cf8f46426af52bb148a56fff/coverage-7.14.1-cp312-cp312-win32.whl", hash = "sha256:4a28fd227808366b196a75476dced2eb35b351d6766ba9c858dc93319e87f4f1", size = 222567, upload-time = "2026-05-26T20:39:24.868Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f6/81/8b7cd386839b039ebe1855733b9f9449a8dec5d79564018234f185a7fa70/coverage-7.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:54acdb6674a4661768d7bf7db32dfb9f46ab1d764f8aba6df75ce1a6a088724e", size = 223372, upload-time = "2026-05-26T20:39:26.603Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ae/ba/b44d472022f620d289d95fa830143235c0c36461c6f2437ea8d51e5481ed/coverage-7.14.1-cp312-cp312-win_arm64.whl", hash = "sha256:99cd41ff91afd94896fea3bc002706b6ae4ce95727d06e4a0f39c0a8d8bd8b1a", size = 221989, upload-time = "2026-05-26T20:39:28.242Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8a/9e/5f6d56327c62b185225d145191c607e07515294a0aa6338e58805cd4a5ac/coverage-7.14.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:be9f2c802dcfce3f71298303aa5dad0dce440a76c52f2f60dacd8656dab78793", size = 220044, upload-time = "2026-05-26T20:39:29.902Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/75/92/e82aca356744cbbc0f77a0b623e38918c1872361963413a3bab5d0340393/coverage-7.14.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:6223a72fd0e4c7156353ec0f08a5f93623e1d3034d0e2683b9bb8ea674131b1d", size = 220412, upload-time = "2026-05-26T20:39:31.561Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/27/c9/385bde0bf7ed0f4bf3a7ee5367060a86b5d218718cfd6fb943c0f836b34f/coverage-7.14.1-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:7279d2110a28cebc738b6459ecda2771735a4c18465fbbd36b3288fe5ed92247", size = 251412, upload-time = "2026-05-26T20:39:33.337Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/51/8c/23faf6a2343a0d17f960a4bd56c43bc7eb4cf312f774dd6ceebd82c7d8fc/coverage-7.14.1-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:9eeb3fcbc13ba40dfbdb22d01d196a28e9cef9ed4c29b60061a1e0e823a9929d", size = 254008, upload-time = "2026-05-26T20:39:35.009Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/42/06/36f4aa9ca8a815e6036156e80706a67828bb97bd826948244f6996dda957/coverage-7.14.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f0cfc27c539f07cf5c0a4cfe211d0b6cae039f8f40526dbaa71944e64b50a7b", size = 255241, upload-time = "2026-05-26T20:39:36.71Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ca/79/95266316352f90f6b1c6736bb413302edfde2453fb32422d3911642691b3/coverage-7.14.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:221c70f316241a78e77e607c227cefc8808d4e08f28d99c04f35694690e940be", size = 257373, upload-time = "2026-05-26T20:39:38.412Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e3/9c/58316d1f66c488b5fca8a0eb3e98348807813efa8a0d0833b9021be27488/coverage-7.14.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:da028256b04ec30e5e0114b6f76172938c313991f0a2d3d894271315cf5d5e43", size = 251635, upload-time = "2026-05-26T20:39:40.268Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ef/5a/ca2398a568e16fed7bb713e84ba3603a7164fb65779abe645c565ec890d5/coverage-7.14.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:76a085d7005236a767e3426148b2c407e53ad61695c562f8a81da2d373324901", size = 253373, upload-time = "2026-05-26T20:39:42.145Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/6e/2c/0396562c32deaebe7be51d865b3a41e9a87d7561acafe1a28f53b07e019a/coverage-7.14.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:b553d04b5e778a8e56d57eb134aff42a92718ecba45e79c4764ecfa40efd92ff", size = 251341, upload-time = "2026-05-26T20:39:43.907Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fd/8f/a94f9221184c9cae1ee115820e3798e48b6b17777a9f19e46fb9a0c8dc74/coverage-7.14.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:46f714d2fb8ae2f4f29f23ada7f1e79b759fff5a70f94a1dac23af204c3ec9e4", size = 255497, upload-time = "2026-05-26T20:39:46.166Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/71/69/505d70e47db1eaebcd002c39759707621ef184cd6b1ae084d9f41293f323/coverage-7.14.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:1896f5e19ff3f0431c7ce2172adc54890fd97f86b59ced8ca1649145d9ffe35d", size = 251159, upload-time = "2026-05-26T20:39:48.03Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e0/aa/58681c383aa33a9d2ed40a02d7a22fbf780d1fa4d575396365777828198c/coverage-7.14.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:62fd185ef9df3c33d1c8178c5af105f762afbad96038de9a4ae100aa6297ca33", size = 252934, upload-time = "2026-05-26T20:39:49.872Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/eb/fd/11c928cd6bdffc7074bb5965c173d9ebf517fb00205e1da524b98d29ef92/coverage-7.14.1-cp313-cp313-win32.whl", hash = "sha256:ab4af6352741a604c431c6072fce5bee33bf0f20dc7a56618d6bf6bb89e9810c", size = 222584, upload-time = "2026-05-26T20:39:51.68Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/6f/92/fb416fc26d340dcba19518c418d6048e913186e17243982c5e435e41fa7a/coverage-7.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:7af486dabe8954d03b087f0021540897afe084f04e16ff5579e08cc46f871416", size = 223394, upload-time = "2026-05-26T20:39:53.472Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/73/c6/02d56e3867972f77d5036de924643f26c056e848f00452cafb4dbc3c29b4/coverage-7.14.1-cp313-cp313-win_arm64.whl", hash = "sha256:2224f89ffd0c5605ccce1ed7a584da162bc7c55f601ab1c946bc9de31a486b42", size = 222015, upload-time = "2026-05-26T20:39:55.374Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/4d/9e/fcc77914050df73f7662fa1f00902774c79c075a8388ab334074574bf77e/coverage-7.14.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:de286598cc65d2b489411174b1faec2f5a7775fb3201fd925db2a76b4030f37d", size = 220733, upload-time = "2026-05-26T20:39:57.189Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f7/67/2963cbdaf5cbadec44efa3a1e39eaa1f02df4079585f05387607a221e126/coverage-7.14.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:042c46ded7c288aeb07cf14a28b6c1e10b78fcba40171c3fa1e939377eeef0b5", size = 221086, upload-time = "2026-05-26T20:39:59.019Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c8/c5/8701645574e11881f2f47d8930f98bc48b5d43b25eb5b4430dfc4a2f9f48/coverage-7.14.1-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:f4ddbe407477f04c45115d1a4e5bc480f753553b534d338d4c3358b1cdd0ea52", size = 262381, upload-time = "2026-05-26T20:40:00.822Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7c/28/7a64d73598263e0c5abd5084211a8474488d31b3c552ff531c719dfcff62/coverage-7.14.1-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:d13e6725992e2d2fd7d81d4f5241952d13740121dfd501da09201be39b2c003a", size = 264458, upload-time = "2026-05-26T20:40:02.506Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fa/d8/4969179db9f7eb4df218e69540adf829d1c835f59452513d065d15446802/coverage-7.14.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f747dc8edcfe740130f28f32f3995e955494285717e86ee25af51db2219df08a", size = 266884, upload-time = "2026-05-26T20:40:04.421Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a6/78/a45d5794dbc9bafd97afc96a4377c86c7820d78b6cf51b89bc1d4e919275/coverage-7.14.1-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ced2f09ef276fd58611a1ef502164ad266d2b75174e5a40cabbdb4033f9f6cf2", size = 268022, upload-time = "2026-05-26T20:40:06.298Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/21/cb/4f5e354e9e3e67af96bd4e57113e6db6b22298c7168b13eec408a549903d/coverage-7.14.1-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b84800013769a78ccb9ef4659402e26d06867e337b61ec365f77ad008adea80e", size = 261631, upload-time = "2026-05-26T20:40:08.226Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ec/49/eced49af4cb996d5d8b7e94e736175c513e4facd3398507b89892b4326d8/coverage-7.14.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:ea8cd6ca0ee9f616aaef3afc6882e32c2cbf18b00d96313ffd76af650574034d", size = 264443, upload-time = "2026-05-26T20:40:10.137Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f1/d8/5603a88a7c5913a6b54f6cb1a8c46f7b39cbb30f27cd3f492908da09b2d7/coverage-7.14.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:aa5e304a873fabddc11e484e9b6b738bd38bd7bed17b09aa84eecf5332e8b8bb", size = 262069, upload-time = "2026-05-26T20:40:11.999Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f0/59/2ae3cb79da554a06c8619d6c88ea19dd1e4aed4b834b6a83bb1fa243bdc5/coverage-7.14.1-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:5a1c5215be81035e629d5bc756650634d0bf31991038db7a0eccb90f025ce16d", size = 265780, upload-time = "2026-05-26T20:40:13.858Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/af/5f/b130c1dc999031f2648bd25317fbce505ad8d5562079b4ed81e736a84967/coverage-7.14.1-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:79058c47dae6788504b5effb319961bcd72d7240551464b91d474bc0ed186d69", size = 260970, upload-time = "2026-05-26T20:40:16.142Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/87/d1/ec13ccddeb48ec963bdfa72a11224bac2584bd045ba13beca82f8113e9c7/coverage-7.14.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:370c5afae3fa0658e11694a32b24c2778f6bc2d17718121f94ee185e69f26b54", size = 263157, upload-time = "2026-05-26T20:40:18.382Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/cf/c2/cd91ead503045161092d3845f7bb95ea2f25131ce96d3e314dd835d91b9c/coverage-7.14.1-cp313-cp313t-win32.whl", hash = "sha256:3758dd0a7f1fa57365ef2e781df0f0731d38b6e3772259d13dae4bd8a958d4b1", size = 223259, upload-time = "2026-05-26T20:40:20.381Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/71/9f/1e28d97e6bd2c76b07f38b7c02870f1371255ff6717f54eca578fcbbdd0e/coverage-7.14.1-cp313-cp313t-win_amd64.whl", hash = "sha256:6ff665fb023a77386fe11685190cee1f60a7d635994a30d9b0a061533d470fce", size = 224320, upload-time = "2026-05-26T20:40:22.316Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a9/e0/d936e908f0e1efa55e52b91e01b52f1055cef5e1ab2718493390ed8e2fb8/coverage-7.14.1-cp313-cp313t-win_arm64.whl", hash = "sha256:17a5a241e5997621a956a7f402a7433ef4221e5152809b785bec79e2323799f1", size = 222577, upload-time = "2026-05-26T20:40:24.894Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/d6/34/fc2f101b151af3799a101f0550b0454aa008afdc0add677394ec4aa8ea10/coverage-7.14.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d5ed429d0b8edaac649e889b4ffcedb6c80b06629a3f93050e3dddfb99235bee", size = 220091, upload-time = "2026-05-26T20:40:27.249Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/3d/a7/1ebae2ab5b961b5c79bb09fe7b3ac99edb190d8be4a8c510b2cf66f46468/coverage-7.14.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:8011224a62280e50dab346960c03cf47aca1a1e09e608c0fb33fd6e0cc8e9500", size = 220421, upload-time = "2026-05-26T20:40:30.084Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/5e/90/92aca9cf0acc95123c96cd1eb1f08917897a7f5dee01e15738922971ec31/coverage-7.14.1-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:12c42ec1e14f553c4f817e989365982e646e27211f10a0f717855b94a79c8906", size = 251466, upload-time = "2026-05-26T20:40:32.542Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/26/2b/78048cbe3b999f6cbf9cc0d90abba6a88a3e0863a8c1c6cbc762f3f8802f/coverage-7.14.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:06144cd511cf2624873a035c5069cf297144f6e77a73ee3d7a55b605ec5efb42", size = 253973, upload-time = "2026-05-26T20:40:34.473Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8e/21/c2e33b29d1cfde484a19d437afc343c6cd30b08d78cbbf9f5aff14e57b2b/coverage-7.14.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a311d8e1da24be5c1ccf85cbfb06315dbaa1703d5a1eab3f6432c72b837917c8", size = 255318, upload-time = "2026-05-26T20:40:38.154Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8e/ee/aad2f108d63b769121005302f16bf66db8625c88ceaba466942e09a2607e/coverage-7.14.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c79cead5b5bc584d9c71451cb984d0e3a84e0c0937379c8efcbf27c8d661b851", size = 257633, upload-time = "2026-05-26T20:40:40.164Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c2/f8/11a2c29b4fd76d9849f81d0bb812ec0017a9396df3217214e38934a8c837/coverage-7.14.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:dcbf65f1f66a26cdd88c35cf68fb4729c5d1cd2e88added72420541dfb212034", size = 251488, upload-time = "2026-05-26T20:40:42.631Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c9/b8/9a5820de4b8ac2b71d85e3b5fb49108d7469c665f0e2ad0dd7569023e305/coverage-7.14.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fd86572566fb40189a8260446158235159bc7a82dfbc87a3b39cf4fb57fcec1c", size = 253329, upload-time = "2026-05-26T20:40:45.208Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/6b/ff/f33e4823667e27548e8fd8df44217515303f9808d0ff29817db56f87d990/coverage-7.14.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:7771b601718fdde84832c3a434ca9bbf4ae9adbc49d84198b4110700c3c77c36", size = 251291, upload-time = "2026-05-26T20:40:47.502Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/68/9b/489db0ebb209054766b90a9014a45f6d26eb724c02ec21311c3733b5a644/coverage-7.14.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:39b21e212c55af06fa375e3dbf90a8a8e38792f3a910c580066d23563830ddd5", size = 255564, upload-time = "2026-05-26T20:40:49.372Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/27/b5/16bc2d4c2409b23c7737edb68c83bc89e345f378050549fe1d75ac7d34d5/coverage-7.14.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:f2302660e32562a532b442480121aef8aa61a5bdb20b30bf0adab29f10a5a4b4", size = 251107, upload-time = "2026-05-26T20:40:51.677Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7d/0c/2629997469a00cd069d588a41c9dc887610f2775ae89d250c4791e65272a/coverage-7.14.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:03a6f93c1ec3b7f2e77b5dbcc5573a2c21f12529a5c6bbe0f16f72303cc2fa4d", size = 252764, upload-time = "2026-05-26T20:40:54.267Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/d2/ee/f78d63c8f079e0d7211c7e2401fa17e311514534ba61bae03e4b287ce4ab/coverage-7.14.1-cp314-cp314-win32.whl", hash = "sha256:8a3ce026d73290f42f08dafecbd82c193a74df280461fbf97300fec51fd133ee", size = 222837, upload-time = "2026-05-26T20:40:56.496Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/dc/b9/be539854f93a70dfbeec69117f33ec70dc42ff0b65b5b07ab8d40d04228e/coverage-7.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:114c95ef29302423b87d159075805f4ab973254a2638a5d7d046c94887cc87d7", size = 223650, upload-time = "2026-05-26T20:40:58.351Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fe/9e/24e2842fef40f35ac82ba3a7719c8023d011bf3bf652d0675316a9d088a1/coverage-7.14.1-cp314-cp314-win_arm64.whl", hash = "sha256:a07891c3f4805442b31b71e84ba3cf29ed1aa9a428284e06deeb4b23e5b46343", size = 222218, upload-time = "2026-05-26T20:41:00.321Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0a/1d/ac0a9df5fe31c1e8bdd658074905fc12844a05c1a7e3fdb8417e97c31e23/coverage-7.14.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:1101a5ebb083aecb625ebb6209d4105b58f647b093cb2dc8122d7b33f743cfe1", size = 220822, upload-time = "2026-05-26T20:41:02.281Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/32/cf/f964fd9aff20323f9f1a726c97135f8a76bcd87b92dad141a456a43f3c64/coverage-7.14.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:851b9e1e4e8a4608e77c79714b2e77c0970d2ed7202a05e92ae407817481887b", size = 221084, upload-time = "2026-05-26T20:41:04.593Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/d8/5e/7e5ef2aba844de2b80d678619fcf0841b42e3f37f16411226f3fe4c1016f/coverage-7.14.1-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:d5b89cdfb2ee051b71e8c3c70bd81a9eff81100f736a269136fe1a68efe00474", size = 262454, upload-time = "2026-05-26T20:41:06.641Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/64/62/75809bded87015cc4935524218a2a8ed8dd1a8498bfed30a2f4f7a4b4d34/coverage-7.14.1-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:0177614a0370f227888b4e436a7c55686d6a9f90eb1ade2b624ba685a1686e86", size = 264578, upload-time = "2026-05-26T20:41:08.556Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f3/42/d33392dc14633525012d2d504fa1a33b05538bf535f5c1d64675e5754b78/coverage-7.14.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2d69af5dea2de76fc485a83032a630523f985198b7e25be901ec60181587b01e", size = 266981, upload-time = "2026-05-26T20:41:10.824Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2a/49/0157c4428c2aca7f1e09d5565930586fd5ae36f1655f08b0daa7cf1fcae1/coverage-7.14.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:35ab22d91de736e8966b980dc355cbcdd2c6dbbcfe275f9a2991bc8a91b3df65", size = 268112, upload-time = "2026-05-26T20:41:12.966Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/96/26/86b9ce71f4092b1ed325ce1421698081df1286b833400b6836912834d6e0/coverage-7.14.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:357d4e32935c36588aaba057d734fa32428c360c9fc2e4442afbf1b646beee6e", size = 261558, upload-time = "2026-05-26T20:41:15Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/20/4c/c311210c5472cf5401d8422b0d7812cdd520f24417673afabda6c323faca/coverage-7.14.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:51bd64741cc6fa065abd300ede1afe5a5291ece9c31da8b24884deda48bcc3f8", size = 264447, upload-time = "2026-05-26T20:41:17.369Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fb/71/59513f8710ed3e6b0ac0a050a5b7e977bb9c9e880354863b5d00d8809256/coverage-7.14.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:9132cd363a68a4c3daa7c8704a654b1e39d3360f6f5b8ddd470608a945236c07", size = 262048, upload-time = "2026-05-26T20:41:19.309Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/84/8d/bceed32dc494f5bbf50f775cd2e78ca814953942b5ea28d3c1c3ac316f14/coverage-7.14.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:07c6290b1697b862c0478eab545eec949a0d0e4d6d03497f446d706da3b4f2de", size = 265781, upload-time = "2026-05-26T20:41:21.559Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e7/c5/9348fe40dbfd4991aaf78df2c6c3098bfb2cc834d1fd362a64b4efef855a/coverage-7.14.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:5ea0c297e27133853b4d8a3eb799bff5a2dbd9f2f41537a240d337ac9b4df890", size = 260896, upload-time = "2026-05-26T20:41:23.428Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ca/92/1ea0f03929da7cf87206b1fa24f4c8e9c158be0455481af29ec0a1f3503f/coverage-7.14.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:01b7733daad0237daa01ef80fe2dfceffc911e6a17fa7b55d14aa8214eaaaecd", size = 263214, upload-time = "2026-05-26T20:41:25.419Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f6/a9/b2493c054c0e01a643266742ab45e15744e60743f9260cd930c7142b1124/coverage-7.14.1-cp314-cp314t-win32.whl", hash = "sha256:6adc5a36984624a70bf11d7184e20fa0a49aa7c47ffab43804106a1a695ea22e", size = 223624, upload-time = "2026-05-26T20:41:27.795Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fc/bd/3e1e6a57fccd2d7c83fcdf338e93ba98eb85c6e877dd34731ac585375490/coverage-7.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:ddf799247318f34dbcd2efa8c95a8d0642674e926bb1774cf9b63dfd2a389d1c", size = 224728, upload-time = "2026-05-26T20:41:30.098Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/bb/d7/31066cf1d2f0c6c797fce911bcfa01dd35642dc6da992a950256097c5860/coverage-7.14.1-cp314-cp314t-win_arm64.whl", hash = "sha256:145986fe66647eb489f18d9a997567a3fd358584c4b5a808769113abc07466af", size = 222752, upload-time = "2026-05-26T20:41:32.123Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8a/3c/1a983b9a745d7f83d53f057bcc5bf79ba6a2bbc08266b3f0c7d6fe630c9b/coverage-7.14.1-py3-none-any.whl", hash = "sha256:a252f21c27e38347e60111a3266b03827422a7d5525951aceee313aa68bab1d2", size = 211815, upload-time = "2026-05-26T20:41:34.078Z" },
]
[[package]]
name = "curl-cffi"
version = "0.15.0"