Add daily derived data pipeline

This commit is contained in:
Yuxuan Yan
2026-06-16 15:55:30 +08:00
parent 83a006bbe4
commit 8d908477e2
19 changed files with 897 additions and 231 deletions
+7 -4
View File
@@ -16,17 +16,20 @@ The default layout is:
data/minute_bars/{universe}/frequency=5m/month=YYYY-MM/*.pq
```
Feature plugins can aggregate those bars to daily `symbol_id,date` feature
Derived-data plugins can aggregate those bars to daily `symbol_id,date` numeric
files, for example:
```bash
uv run python cli.py feature compute \
uv run python cli.py derived compute \
--minute-path data/minute_bars/sh600000 \
--daily-path data/daily_bars/sh600000 \
--feature-type minute_daily_summary \
--feature-name minute_summary
--derived-type minute_daily_summary \
--derived-name minute_summary
```
The legacy `feature compute` command delegates to the same derived-data
registry and remains available for existing scripts.
## Daily vs Minute Reconciliation
Baostock's daily raw bars and 5-minute raw bars are close, but they should not