Yuxuan Yan
71 lines
2.4 KiB
Markdown
71 lines
2.4 KiB
Markdown
# Minute Bar Data Notes
|
|
|
|
The minute-bar path downloads raw Baostock intraday bars and stores them as a
|
|
Hive-partitioned dataset:
|
|
|
|
```bash
|
|
uv run python cli.py data download-minute \
|
|
--universe sh600000 \
|
|
--start-date 2024-01-02 --end-date 2024-01-05 \
|
|
--frequency 5
|
|
```
|
|
|
|
The default layout is:
|
|
|
|
```text
|
|
data/minute_bars/{universe}/frequency=5m/month=YYYY-MM/*.pq
|
|
```
|
|
|
|
Derived-data plugins can aggregate those bars to daily `symbol_id,date` numeric
|
|
files, for example:
|
|
|
|
```bash
|
|
uv run python cli.py derived compute \
|
|
--minute-path data/minute_bars/sh600000 \
|
|
--daily-path data/daily_bars/sh600000 \
|
|
--derived-type minute_daily_summary \
|
|
--derived-name minute_summary
|
|
```
|
|
|
|
The legacy `feature compute` command delegates to the same derived-data
|
|
registry and remains available for existing scripts.
|
|
|
|
## Daily vs Minute Reconciliation
|
|
|
|
Baostock's daily raw bars and 5-minute raw bars are close, but they should not
|
|
be treated as perfectly reconstructible from each other.
|
|
|
|
When checking consistency, compare daily raw bars (`data download --adjust none`)
|
|
against minute bars on the same raw price scale. The minute aggregation should
|
|
use:
|
|
|
|
- `open`: first minute open
|
|
- `high`: max minute high
|
|
- `low`: min minute low
|
|
- `close`: last minute close
|
|
- `volume`: sum minute volume
|
|
- `amount`: sum minute amount
|
|
- `vwap`: `sum(amount) / sum(volume)`
|
|
|
|
In a sanity check for `sh600000` from `2024-01-02` through `2024-01-05`, Baostock
|
|
returned 4 daily rows and 192 5-minute bars, exactly 48 bars per day. Open, low,
|
|
and close matched daily exactly on all 4 days. High matched on 3 of 4 days; on
|
|
`2024-01-04`, the daily high was `6.67` while the max 5-minute high was `6.66`.
|
|
Minute-summed volume and amount were higher than daily by roughly `0.16%` to
|
|
`1.23%`. VWAP remained very close, with max relative difference around
|
|
`0.0043%`.
|
|
|
|
This appears to be a source-level Baostock reconciliation caveat, not a parser
|
|
or ordering issue: the minute bars covered the regular `09:35:00` through
|
|
`15:00:00` range and sorted correctly by timestamp.
|
|
|
|
Practical guidance:
|
|
|
|
- Use tolerance-based daily-vs-minute checks; do not require exact equality for
|
|
high, volume, or amount.
|
|
- Expect open/close alignment to be a stronger sanity check than exact volume
|
|
reconstruction.
|
|
- Use minute-derived values as separate daily features, not as replacements for
|
|
the canonical daily bar dataset unless a strategy explicitly wants that
|
|
source convention.
|