Compare commits

..

8 Commits

Author SHA1 Message Date
Yuxuan Yan c91415aff8 Document JoinQuant cost model findings 2026-07-06 15:35:41 +08:00
Yuxuan Yan efa9d24c73 Harden JoinQuant browser smoke automation 2026-07-04 19:27:48 +08:00
Yuxuan Yan 5fada75d23 Add JoinQuant env browser login helper 2026-07-04 18:37:34 +08:00
Yuxuan Yan d05931f41a Add JoinQuant simulated trading automation 2026-07-04 18:12:01 +08:00
Yuxuan Yan 39a93259e1 Add JoinQuant browser backtest automation 2026-07-04 18:08:31 +08:00
Yuxuan Yan 5388359dc8 Automate local JoinQuant smoke prep 2026-07-04 17:55:19 +08:00
Yuxuan Yan c200508f9e Align JoinQuant targets to execution dates 2026-07-04 17:50:55 +08:00
Yuxuan Yan f25db279bf Add JoinQuant comparison plugin 2026-07-04 17:43:09 +08:00
18 changed files with 4282 additions and 1 deletions
+2
View File
@@ -20,6 +20,7 @@ from pipeline.alpha.cli import alpha
from pipeline.features.cli import feature from pipeline.features.cli import feature
from pipeline.combo.cli import combo from pipeline.combo.cli import combo
from pipeline.portfolio.cli import portfolio from pipeline.portfolio.cli import portfolio
from plugins.joinquant.cli import joinquant
from tools.pqcat import pqcat from tools.pqcat import pqcat
from tools.alphaview import alphaview from tools.alphaview import alphaview
@@ -48,6 +49,7 @@ cli.add_command(alpha)
cli.add_command(feature) cli.add_command(feature)
cli.add_command(combo) cli.add_command(combo)
cli.add_command(portfolio) cli.add_command(portfolio)
cli.add_command(joinquant)
cli.add_command(pqcat) cli.add_command(pqcat)
cli.add_command(alphaview) cli.add_command(alphaview)
+352
View File
@@ -0,0 +1,352 @@
# JoinQuant Comparison Plugin
## Why a Plugin
JoinQuant is an external execution and simulation reference. Keeping this code
under `plugins/joinquant/` prevents vendor-specific assumptions from entering
`pipeline/portfolio/`, where the internal reference simulator remains the
canonical implementation.
## What It Validates
The comparison is for system correctness:
- date alignment
- internal to JoinQuant symbol mapping
- target position generation
- once-per-day open execution timing
- lot rounding and filled shares
- position carry
- trading cost
- PnL accounting
- blocked trades from suspension, limit-up, and limit-down conditions
## What It Does Not Validate
It does not validate alpha quality, IC, IR, forecast skill, or whether the
strategy is economically useful. Differences can be expected when JoinQuant
uses different fee, slippage, cash, corporate-action, or internal rounding
rules.
## Historical Backtest Workflow
```bash
# 1. Build internal portfolio targets.
uv run python cli.py portfolio build ...
# 2. Export JoinQuant-compatible frozen targets.
uv run python cli.py joinquant export-targets \
--positions-path portfolio/run1.pq \
--portfolio-name run1 \
--mode target_shares \
--execution-calendar-path data/daily_bars/<universe> \
--out-dir plugins_output/joinquant/targets
# 3. Generate and copy the wrapper strategy and target files into JoinQuant.
uv run python cli.py joinquant write-wrapper \
--portfolio-name run1 \
--mode target_shares \
--out-path plugins_output/joinquant/wrapper_strategy_run1.py
# 4. Run the JoinQuant backtest or simulated trading job.
# 5. Export JoinQuant fills, positions, and daily PnL to CSV.
# 6. Ingest JoinQuant output.
uv run python cli.py joinquant ingest \
--portfolio-name run1 \
--fills-csv path/to/jq_fills.csv \
--positions-csv path/to/jq_positions.csv \
--pnl-csv path/to/jq_pnl.csv
# 7. Reconcile.
uv run python cli.py joinquant reconcile \
--portfolio-name run1 \
--targets-dir plugins_output/joinquant/targets/run1 \
--our-fills-path fills/run1.pq \
--our-positions-path portfolio/run1.pq \
--our-pnl-path pnl/run1.pq \
--jq-fills-path plugins_output/joinquant/ingested/run1/fills.pq \
--jq-positions-path plugins_output/joinquant/ingested/run1/positions.pq \
--jq-pnl-path plugins_output/joinquant/ingested/run1/pnl.pq
```
## Forward-Testing Workflow
After the T-1 close and after the data update:
```bash
uv run python cli.py portfolio build ...
uv run python cli.py joinquant export-targets \
--positions-path portfolio/run1.pq \
--portfolio-name run1 \
--mode target_shares \
--execution-calendar-path data/daily_bars/<universe> \
--start-date T \
--end-date T
```
Before the T open, upload or expose the frozen target file to JoinQuant. During
the T open, the JoinQuant wrapper reads that file and submits orders, while the
internal simulator should run against the same frozen target. After T close or
after JoinQuant results are available, ingest the JoinQuant CSV files and run
`joinquant reconcile`.
Forward target files must be frozen before execution. Do not regenerate a
target file after observing open or close data for the same trading date. The
exporter writes a snapshot JSON with a SHA-256 hash for this reason and refuses
to overwrite existing target/snapshot files unless `--force` is passed.
For comparisons against the internal simulator, pass
`--execution-calendar-path` to `joinquant export-targets`. The positions file is
dated by construction/signal date, while the simulator executes at the next
available open. The calendar option shifts exported target files to that next
trading date, so JoinQuant reads the same target on the same execution session.
For JoinQuant 模拟盘, the browser automation has two operational phases:
```bash
# Before T open: upload frozen target(s), save strategy, and start/restart 模拟盘.
uv run python cli.py joinquant write-browser-config \
--out-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
--strategy-url "https://www.joinquant.com/<your 模拟盘 page>" \
--flow sim-trade
uv run python cli.py joinquant run-browser-sim \
--manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
--config-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json \
--headed
# After T close: download/export JoinQuant fills, positions, and pnl, then
# ingest/reconcile. The same run-browser-sim command can do this if the config
# includes download actions, otherwise use the ingest/reconcile commands.
```
The default simulated-trading template includes selectors for saving the
strategy and clicking simulated-trading controls such as `模拟盘`, `模拟交易`,
`启动`, and `重启`. These selectors are intentionally configurable because the
JoinQuant web UI can differ by account and page version.
## Target-Shares Mode
`target_shares` is the default and preferred correctness mode. The exported
`target_shares` field comes from the internal `position_shares` column produced
by `portfolio build`, because the internal simulator executes that discretized
integer book. The generated wrapper calls:
```python
order_target(jq_symbol, target_shares)
```
This mode makes filled shares, position carry, and blocked trades easiest to
compare.
## Target-Value Mode
`target_value` mode exports `target_value` and `target_weight` from the
portfolio file. The generated wrapper calls:
```python
order_target_value(jq_symbol, target_value)
```
This can be useful for portfolio-level comparisons, but JoinQuant may apply its
own rounding, cash, and lot rules. Differences are often classified as
`JOINQUANT_INTERNAL_ROUNDING`, `LOT_ROUNDING`, or `CASH_CONSTRAINT` depending
on the observed output.
## Symbol Mapping
Internal symbols are converted as follows:
```text
sh600000 -> 600000.XSHG
sh688001 -> 688001.XSHG
sz000001 -> 000001.XSHE
sz300001 -> 300001.XSHE
```
Reverse mapping is also supported. Invalid exchanges or unsupported A-share
prefixes raise `ValueError` instead of silently guessing.
## Wrapper Strategy Usage
Generate a configured wrapper:
```bash
uv run python cli.py joinquant write-wrapper \
--portfolio-name run1 \
--mode target_shares \
--out-path plugins_output/joinquant/wrapper_strategy_run1.py
```
Copy the generated file and daily CSV target files into JoinQuant. The default
loader uses JoinQuant `read_file`, which works for uploaded files. If your
JoinQuant runtime allows HTTP or another storage backend, replace only
`_read_target_file()` in the generated strategy.
The wrapper is long-only by default:
```python
ALLOW_SHORT = False
```
Negative targets are clipped to zero and logged. Use `--allow-short` only if
the target JoinQuant account supports the required shorting mechanics.
## Ingesting JoinQuant Outputs
The ingest command accepts permissive CSV column names and writes strict plugin
schemas:
```text
plugins_output/joinquant/ingested/{portfolio_name}/fills.pq
plugins_output/joinquant/ingested/{portfolio_name}/positions.pq
plugins_output/joinquant/ingested/{portfolio_name}/pnl.pq
```
Missing cost fields default to zero. Missing blocked status defaults to zero.
Symbols and dates are normalized.
## Reading Reconciliation Reports
The reconcile command writes:
```text
plugins_output/joinquant/reconcile/{portfolio_name}/daily_reconcile.pq
plugins_output/joinquant/reconcile/{portfolio_name}/summary.csv
plugins_output/joinquant/reconcile/{portfolio_name}/summary.md
```
`daily_reconcile.pq` is per-symbol and includes target shares, internal filled
shares, JoinQuant filled shares, realized positions, trade prices, costs, PnL,
and a `diff_reason`. `summary.csv` is the daily portfolio-level view for gross
exposure, net exposure, cash, total value, PnL, cumulative PnL, turnover, and
cost.
Difference reasons include:
```text
MATCH SYMBOL_MAPPING PRICE_MISMATCH LOT_ROUNDING SUSPENSION LIMIT_UP_BLOCK
LIMIT_DOWN_BLOCK VOLUME_OR_LIQUIDITY COST_MODEL CASH_CONSTRAINT
SHORT_NOT_SUPPORTED CORPORATE_ACTION JOINQUANT_INTERNAL_ROUNDING
MISSING_IN_OUR_SYSTEM MISSING_IN_JOINQUANT UNKNOWN
```
Default tolerances are exact share matching, `1e-4` relative trade-price
tolerance, and value tolerance `max(1 yuan, 1e-6 * booksize)`. PnL tolerance is
configurable with `--pnl-tolerance`.
## Minimal Example
Create a 5-stock equal-weight or fixed-share test portfolio:
```text
sh600000, sz000001, sh600519, sz002594, sz300750
```
Build positions for a small date range, export `target_shares`, upload the CSV
files and wrapper to JoinQuant, run the JoinQuant backtest, export fills,
positions, and PnL, then run ingest and reconcile. Start with one or two days
before expanding the sample.
For the first one-stock long-only smoke test, the local side can be prepared in
one command:
```bash
uv run python cli.py joinquant prepare-smoke \
--out-dir /tmp/chinese-equity-quant-realdata
```
The command downloads a tiny public daily-bar sample, builds a fixed-share
`sh600000` long-only position file, simulates it internally, exports aligned
JoinQuant target files, writes a configured wrapper strategy, and creates
`joinquant_smoke_manifest.json` with all output paths.
## Browser Backtest Automation
JoinQuant's public `jqdatasdk` is a data SDK. It supports authenticated data
calls such as `auth(username, password)` and `get_price(...)`, but cloud
strategy upload, backtest execution, and result export are web-application
workflows. The plugin therefore automates those remote steps through Playwright
with a saved browser session.
Install the optional browser runner in the uv environment:
```bash
uv sync --extra joinquant-browser
uv run playwright install chromium
```
Save a reusable login state. This opens a browser; log in normally, including
any CAPTCHA or 2FA, then press Enter in the terminal to save state:
```bash
uv run python cli.py joinquant browser-login \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json
```
Create a selector/action config for a historical backtest:
```bash
uv run python cli.py joinquant write-browser-config \
--out-path /tmp/chinese-equity-quant-realdata/joinquant_browser_config.json \
--strategy-url "https://www.joinquant.com/<your strategy page>" \
--flow backtest
```
If selectors need tuning, capture the logged-in page:
```bash
uv run python cli.py joinquant browser-snapshot \
--url "https://www.joinquant.com/<your strategy page>" \
--out-dir /tmp/chinese-equity-quant-realdata/browser_snapshot
```
Then run the remote backtest automation:
```bash
uv run python cli.py joinquant run-browser-backtest \
--manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
--config-path /tmp/chinese-equity-quant-realdata/joinquant_browser_config.json \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json \
--headed
```
The config is declarative: actions can navigate, paste the generated wrapper,
upload all target CSV files, fill dates, click run, wait for completion,
download result CSVs, and take screenshots. When the configured downloads
produce fills, positions, and PnL CSVs, the runner automatically calls
`joinquant ingest` and `joinquant reconcile`.
For forward testing / 模拟盘, create the config with `--flow sim-trade` and run:
```bash
uv run python cli.py joinquant run-browser-sim \
--manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
--config-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json \
--headed
```
Do not store raw JoinQuant passwords in this repository. The browser state file
is created with `0600` permissions and should live under `~/.config`, outside
the repo.
## Recommended First Sanity Checks
1. One liquid stock with a fixed target share count.
2. A 10-stock equal-weight long-only portfolio.
3. A forced suspension, limit-up, and limit-down sample.
4. A short target in long-only mode to confirm `SHORT_NOT_SUPPORTED`.
5. A 5-day reversal portfolio after the mechanical checks pass.
## Known Limitations
- JoinQuant internal execution details may differ from the reference simulator.
- External file loading depends on the JoinQuant environment.
- Short selling may not be supported.
- Fee, tax, slippage, and minimum-fee models may differ.
- Corporate actions may need special handling and should not be hidden.
- The internal simulator does not currently emit execution price in
`FILL_COLUMNS`; price reconciliation uses explicit price columns if supplied.
+167
View File
@@ -0,0 +1,167 @@
# JoinQuant Cost Model Findings
Generated: 2026-07-06
This report summarizes the JoinQuant trading-cost behavior observed from the
browser-automated real-data backtests and compares it with the current internal
simulator model. The JoinQuant cost formula below is inferred from rendered
transaction tables and strategy logs for this account. Treat it as an observed
platform default, not as a guaranteed external contract.
## Runs Used
### Longer Comparison Run
- Portfolio: `jq_long_one_stock_long`
- Window: `2024-01-11` to `2024-02-29`
- Booksize: `1,000,000 CNY`
- Instrument: `600000.XSHG`
- Target: buy and hold `1,000` shares
- JoinQuant rendered result: completed
- Local total PnL: `546.22 CNY`
- JoinQuant total PnL from positions tab: `575.00 CNY`
- Difference: `28.78 CNY`, or `0.002878` percentage points on the book
The first JoinQuant transaction was:
| Date | Side | Shares | Price | Turnover | Fee |
|---|---:|---:|---:|---:|---:|
| `2024-01-11` | Buy | `1,000` | `6.57` | `6,570.00` | `5.00` |
That trade hit a minimum fee. The local simulator charged `6.1415 CNY` because
the smoke runner used a flat `10 bps` cash cost on adjusted-price turnover.
### Cost Probe Run
- Portfolio: `jq_cost_probe_buy_sell`
- Window: `2024-01-11` to `2024-01-12`
- Booksize: `1,000,000 CNY`
- Instrument: `600000.XSHG`
- Targets:
- `2024-01-11`: buy `100,000` shares
- `2024-01-12`: sell to `0` shares
Observed JoinQuant transactions:
| Date | Side | Shares | Price | Turnover | Fee | Implied Fee |
|---|---:|---:|---:|---:|---:|---:|
| `2024-01-11` | Buy | `100,000` | `6.57` | `657,000.00` | `197.10` | `3.0000 bps` |
| `2024-01-12` | Sell | `-100,000` | `6.51` | `651,000.00` | `846.30` | `13.0000 bps` |
The transaction-table numbers match this formula exactly:
```text
buy fee = max(5 CNY, turnover * 0.0003)
sell fee = max(5 CNY, turnover * 0.0003) + turnover * 0.001
```
In basis points:
- Buy commission: `3 bps`
- Sell commission: `3 bps`
- Sell stamp tax: `10 bps`
- Minimum commission: `5 CNY`
No separate transfer fee was visible in this probe. If a separate transfer fee
was present as an additional charge, the observed fees would not match the
formula above exactly. It may still be folded into JoinQuant's displayed
commission field, so this finding should be read as "not separately observable"
rather than "impossible".
No slippage was visible. The wrapper submitted open-time market orders with
`run_daily(..., time="open")`, and JoinQuant filled them at the displayed open
prices.
## Evidence From Strategy Logs
The JoinQuant order log for the cost probe showed:
```text
2024-01-11 ... trade price: 6.57, amount:100000, commission: 197.1
2024-01-12 ... trade price: 6.51, amount:100000, commission: 846.3
```
The normal transaction tab showed the same fee values. However, the generated
wrapper's `JOINQUANT_FILL` log records had `trade_cost: 0.0`, even for those
same fills. That means `get_trades()` did not expose the usable commission
value through the field the wrapper currently reads.
For reconciliation, use the transaction table or JoinQuant order logs for fee
details. Do not rely on the wrapper's current `JOINQUANT_FILL.trade_cost`.
## Difference From The Internal Simulator
The current internal simulator cost model is
`SimpleProportionalCostModel` in `pipeline/portfolio/costs.py`:
```text
trade_cost = abs(traded_shares * execution_price)
* (cost_bps + slippage_bps) / 10000
```
The smoke runner used:
- `cost_bps = 5`
- `slippage_bps = 5`
- combined one-way cash cost: `10 bps`
Important differences:
- The internal simulator uses the same rate for buys and sells.
- It has no minimum commission.
- It has no sell-only stamp tax.
- Slippage is modeled as an extra cash cost.
- JoinQuant did not show slippage in the observed open-time fills.
- The local smoke download used `adjust="qfq"`, while the JoinQuant wrapper set
`set_option("use_real_price", True)`. That price-scale mismatch also affects
PnL and cost comparisons.
## Practical Implications
For a buy-only smoke test, JoinQuant may charge less than the local model when
the trade is large enough for `3 bps` to apply, but it may charge more on small
orders because of the `5 CNY` minimum.
For any test with sells, JoinQuant's default sell fee is materially higher than
the current local flat model because of the inferred `10 bps` stamp tax.
The earlier 30-day buy-and-hold discrepancy was small because only one buy was
executed. A rebalancing strategy with many sells will show a larger cost-model
difference unless the local simulator is configured to match JoinQuant.
## Recommended Follow-Ups
1. Add a JoinQuant-style cost model to the internal simulator:
```text
commission = max(min_commission, turnover * commission_bps / 10000)
stamp_tax = turnover * sell_stamp_tax_bps / 10000 for sells only
trade_cost = commission + stamp_tax
```
2. Add a CLI option or preset for `portfolio simulate`, for example
`--cost-model joinquant-stock`.
3. Update JoinQuant reconciliation to parse fee values from the transaction
table or order logs when CSV exports are unavailable.
4. Run a second local-vs-JoinQuant comparison with:
- raw or real-price local bars, not adjusted-price bars
- JoinQuant-style costs
- slippage disabled locally
That test should isolate remaining differences to data alignment, price source,
rounding, and JoinQuant internal execution behavior.
## Local Artifacts
The temporary artifacts from the investigation are:
- `/tmp/chinese-equity-quant-jq-long/comparison_report.md`
- `/tmp/chinese-equity-quant-jq-long/parsed_joinquant/daily_pnl_compare_from_positions_tab.csv`
- `/tmp/chinese-equity-quant-jq-cost-probe/jq_cost_analysis_report.md`
- `/tmp/chinese-equity-quant-jq-cost-probe/jq_cost_analysis_summary.json`
- `/tmp/chinese-equity-quant-jq-cost-probe/jq_cost_probe_transactions_parsed.csv`
- `/tmp/chinese-equity-quant-jq-cost-probe/detail_tabs/transactions.txt`
+2
View File
@@ -0,0 +1,2 @@
"""Optional plugin packages for the research pipeline."""
+103
View File
@@ -0,0 +1,103 @@
# JoinQuant Comparison Plugin
This plugin exports frozen targets from the internal A-share research pipeline,
drives a standalone JoinQuant wrapper strategy, ingests JoinQuant output files,
and reconciles them against the internal reference simulator.
The plugin validates system mechanics, not alpha quality:
- date alignment
- symbol mapping
- target position generation
- open execution timing
- lot rounding and filled shares
- position carry
- trading cost and PnL accounting
- blocked trades from suspension and price limits
## Commands
```bash
uv run python cli.py joinquant prepare-smoke \
--out-dir /tmp/chinese-equity-quant-realdata
uv sync --extra joinquant-browser
uv run playwright install chromium
uv run python cli.py joinquant browser-login \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json
uv run python cli.py joinquant write-browser-config \
--out-path /tmp/chinese-equity-quant-realdata/joinquant_browser_config.json \
--strategy-url "https://www.joinquant.com/..." \
--flow backtest
uv run python cli.py joinquant run-browser-backtest \
--manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
--config-path /tmp/chinese-equity-quant-realdata/joinquant_browser_config.json \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json
uv run python cli.py joinquant write-browser-config \
--out-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
--strategy-url "https://www.joinquant.com/<模拟盘 page>" \
--flow sim-trade
uv run python cli.py joinquant run-browser-sim \
--manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
--config-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
--storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json
uv run python cli.py joinquant export-targets \
--positions-path portfolio/run1.pq \
--portfolio-name run1 \
--mode target_shares \
--execution-calendar-path data/daily_bars/csi500 \
--start-date 2026-07-01 \
--end-date 2026-07-31 \
--out-dir plugins_output/joinquant/targets
uv run python cli.py joinquant write-wrapper \
--portfolio-name run1 \
--mode target_shares \
--out-path plugins_output/joinquant/wrapper_strategy_run1.py
uv run python cli.py joinquant ingest \
--portfolio-name run1 \
--fills-csv path/to/jq_fills.csv \
--positions-csv path/to/jq_positions.csv \
--pnl-csv path/to/jq_pnl.csv \
--out-dir plugins_output/joinquant/ingested
uv run python cli.py joinquant reconcile \
--portfolio-name run1 \
--targets-dir plugins_output/joinquant/targets/run1 \
--our-fills-path fills/run1.pq \
--our-positions-path portfolio/run1.pq \
--our-pnl-path pnl/run1.pq \
--jq-fills-path plugins_output/joinquant/ingested/run1/fills.pq \
--jq-positions-path plugins_output/joinquant/ingested/run1/positions.pq \
--jq-pnl-path plugins_output/joinquant/ingested/run1/pnl.pq \
--out-dir plugins_output/joinquant/reconcile
```
`target_shares` is the default and uses the built integer `position_shares`
from `portfolio build`, matching what the internal simulator executes.
For strict simulator-vs-JoinQuant comparison, pass `--execution-calendar-path`
so position dates are shifted to the next session open, matching the internal
simulator's next-open convention.
`prepare-smoke` automates the local side of the first sanity check: tiny real
data download, one-stock long-only position file, internal simulation, aligned
target export, wrapper generation, and a manifest with expected JoinQuant CSV
export paths.
`run-browser-backtest` automates the remote JoinQuant web run through
Playwright. It reuses a saved browser login state, executes the configured UI
actions, downloads JoinQuant CSVs when configured, and runs ingest/reconcile
automatically once all three CSVs are present.
`run-browser-sim` is the forward-test / 模拟盘 equivalent. Use a `--flow
sim-trade` config to upload the frozen next-session target file, save the
strategy, and start or restart the JoinQuant simulated-trading job. After close,
run it again with download actions or use `ingest` / `reconcile` directly on
exported CSVs.
+10
View File
@@ -0,0 +1,10 @@
"""JoinQuant comparison plugin.
This package keeps JoinQuant-specific export, ingest, and reconciliation code
outside the core portfolio modules.
"""
from plugins.joinquant.symbols import from_joinquant_symbol, to_joinquant_symbol
__all__ = ["from_joinquant_symbol", "to_joinquant_symbol"]
+710
View File
@@ -0,0 +1,710 @@
"""Browser automation for JoinQuant cloud backtest and simulated-trading runs.
JoinQuant's public ``jqdatasdk`` is a data API; cloud strategy upload/run/export
is exposed through the web application. This module keeps that automation
optional and configurable so UI selector drift does not affect the core test
suite.
"""
from __future__ import annotations
import json
import re
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from plugins.joinquant.ingest import ingest_joinquant_outputs
from plugins.joinquant.reconcile import reconcile_joinquant
DEFAULT_LOGIN_URL = "https://www.joinquant.com/user/login/index"
def _require_playwright():
try:
from playwright.sync_api import TimeoutError as PlaywrightTimeoutError
from playwright.sync_api import sync_playwright
except ImportError as exc:
raise RuntimeError(
"Playwright is required for JoinQuant browser automation. Install it "
"in this uv environment, then install Chromium:\n"
" uv sync --extra joinquant-browser\n"
" uv run playwright install chromium"
) from exc
return sync_playwright, PlaywrightTimeoutError
def _backtest_actions() -> list[dict[str, Any]]:
return [
{"type": "goto", "url": "{strategy_url}"},
{
"type": "click",
"selector": "text=不再提示",
"timeout_ms": 5000,
"force": True,
"optional": True,
"description": "Dismiss JoinQuant first-run editor guide.",
},
{
"type": "click",
"selector": "text=确定",
"timeout_ms": 5000,
"force": True,
"optional": True,
"description": "Close JoinQuant first-run editor guide.",
},
{
"type": "click",
"selector": "text=跳过",
"timeout_ms": 5000,
"force": True,
"optional": True,
"description": "Skip JoinQuant first-run editor guide.",
},
{
"type": "evaluate",
"script": (
"() => {"
"document.querySelectorAll("
"'.introjs-overlay,.introjs-helperLayer,.introjs-tooltipReferenceLayer,"
".introjs-tooltip,.introjs-disableInteraction'"
").forEach((node) => node.remove());"
"document.body.classList.remove('introjs-noscroll');"
"}"
),
"optional": True,
"description": "Remove any remaining JoinQuant intro overlay.",
},
{
"type": "set_ace_editor_file",
"selector": ".ace_editor",
"path": "{wrapper_path}",
"description": "Set generated wrapper strategy in the Ace code editor.",
},
{
"type": "set_input_files",
"selector": "input[type=file]",
"paths": "{target_csvs}",
"description": "Upload all aligned daily target CSV files.",
"timeout_ms": 5000,
"optional": True,
},
{
"type": "evaluate",
"script": (
"(arg) => {"
"const setValue = (selector, value) => {"
"const el = document.querySelector(selector);"
"if (!el) return false;"
"el.value = value;"
"el.setAttribute('value', value);"
"el.dispatchEvent(new Event('input', {bubbles: true}));"
"el.dispatchEvent(new Event('change', {bubbles: true}));"
"return true;"
"};"
"return {"
"start: setValue('#startTime', arg.start),"
"end: setValue('#endTime', arg.end),"
"capital: setValue('#daily_backtest_capital_base_box', arg.capital)"
"};"
"}"
),
"arg": {
"start": "{backtest_start_date}",
"end": "{backtest_end_date}",
"capital": "{booksize}",
},
"description": "Set JoinQuant backtest dates and starting capital.",
"optional": True,
},
{
"type": "click",
"selector": "#algo-save-button",
"timeout_ms": 5000,
"description": "Save the JoinQuant strategy code.",
"optional": True,
},
{
"type": "click",
"selector": ".bootstrap-dialog .btn-primary, .modal.in button:has-text(\"确定\")",
"timeout_ms": 5000,
"force": True,
"description": "Confirm any JoinQuant save dialog.",
"optional": True,
},
{"type": "wait_for_timeout", "timeout_ms": 2000},
{
"type": "click",
"selector": "#daily-new-backtest-button",
"description": "Start the JoinQuant backtest.",
},
{
"type": "wait_for_selector",
"selector": "text=/回测完成|运行完成|Backtest complete|Finished/i",
"timeout_ms": 600_000,
"optional": True,
},
{
"type": "download",
"selector": "text=/导出成交|下载成交|fills|trades/i",
"save_as": "{expected_joinquant_csvs.fills}",
"timeout_ms": 15000,
"optional": True,
},
{
"type": "download",
"selector": "text=/导出持仓|下载持仓|positions/i",
"save_as": "{expected_joinquant_csvs.positions}",
"timeout_ms": 15000,
"optional": True,
},
{
"type": "download",
"selector": "text=/导出收益|下载收益|pnl|收益/i",
"save_as": "{expected_joinquant_csvs.pnl}",
"timeout_ms": 15000,
"optional": True,
},
{"type": "screenshot", "path": "{run_artifact_dir}/final.png"},
]
def _sim_trade_actions() -> list[dict[str, Any]]:
return [
{"type": "goto", "url": "{strategy_url}"},
{
"type": "paste_text_file",
"selector": "textarea, .ace_text-input, .cm-content, .CodeMirror textarea",
"path": "{wrapper_path}",
"description": "Paste generated wrapper strategy into the simulated-trading strategy editor.",
},
{
"type": "set_input_files",
"selector": "input[type=file]",
"paths": "{target_csvs}",
"description": "Upload frozen target CSV files for 模拟盘.",
"optional": True,
},
{
"type": "click",
"selector": "text=/保存|Save/i",
"description": "Save the strategy code and uploaded files.",
"optional": True,
},
{
"type": "wait_for_selector",
"selector": "text=/保存成功|已保存|Saved/i",
"timeout_ms": 120_000,
"optional": True,
},
{
"type": "click",
"selector": "text=/模拟盘|模拟交易|启动模拟|运行模拟|启动|重启|Run Sim|Start|Restart/i",
"description": "Start or restart the JoinQuant simulated-trading job.",
},
{
"type": "wait_for_selector",
"selector": "text=/运行中|已启动|模拟交易运行|Started|Running/i",
"timeout_ms": 180_000,
"optional": True,
},
{"type": "screenshot", "path": "{run_artifact_dir}/sim_trade_final.png"},
]
def default_browser_config(strategy_url: str = "", *, flow: str = "backtest") -> dict[str, Any]:
"""Return a selector/action template for JoinQuant browser automation."""
if flow not in {"backtest", "sim-trade", "sim_trade"}:
raise ValueError("flow must be 'backtest' or 'sim-trade'")
normalized_flow = "sim-trade" if flow == "sim_trade" else flow
actions = _sim_trade_actions() if normalized_flow == "sim-trade" else _backtest_actions()
return {
"flow": normalized_flow,
"strategy_url": strategy_url,
"login_url": DEFAULT_LOGIN_URL,
"headless": False,
"timeout_ms": 120_000,
"notes": [
"Fill strategy_url and selectors after inspecting your JoinQuant strategy page.",
"Use `joinquant browser-snapshot` to capture HTML/screenshots for selector tuning.",
"The action list is declarative and runs in order.",
],
"actions": actions,
}
def write_browser_config_template(
path: str | Path,
*,
strategy_url: str = "",
flow: str = "backtest",
) -> Path:
"""Write a JSON config template for browser automation."""
out_path = Path(path)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text(
json.dumps(
default_browser_config(strategy_url, flow=flow),
indent=2,
ensure_ascii=False,
) + "\n",
encoding="utf-8",
)
return out_path
def load_json(path: str | Path) -> dict[str, Any]:
"""Load a JSON object from disk."""
data = json.loads(Path(path).read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise ValueError(f"Expected JSON object in {path}")
return data
def load_env_file(path: str | Path) -> dict[str, str]:
"""Load simple KEY=VALUE env files without requiring shell-safe quoting."""
values: dict[str, str] = {}
for raw in Path(path).expanduser().read_text(encoding="utf-8").splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, value = line.split("=", 1)
clean = value.strip()
if len(clean) >= 2 and clean[0] == clean[-1] and clean[0] in {"'", '"'}:
clean = clean[1:-1]
else:
clean = clean.strip("'\"")
values[key.strip()] = clean
return values
def _get_dotted(data: dict[str, Any], dotted: str) -> Any:
current: Any = data
for part in dotted.split("."):
if isinstance(current, dict) and part in current:
current = current[part]
else:
raise KeyError(dotted)
return current
def _manifest_context(manifest: dict[str, Any], artifact_dir: Path, config: dict[str, Any]) -> dict[str, Any]:
targets_dir = Path(str(manifest["targets_dir"]))
target_csvs = [str(path) for path in sorted(targets_dir.glob("*.csv"))]
if not target_csvs:
raise ValueError(f"No target CSV files found under {targets_dir}")
target_dates = [
datetime.strptime(Path(path).stem, "%Y%m%d").strftime("%Y-%m-%d")
for path in target_csvs
]
context = dict(manifest)
context.update({
"strategy_url": config.get("strategy_url", ""),
"target_csvs": target_csvs,
"target_csvs_csv": ",".join(target_csvs),
"backtest_start_date": config.get("backtest_start_date") or min(target_dates),
"backtest_end_date": config.get("backtest_end_date") or max(target_dates),
"run_artifact_dir": str(artifact_dir),
})
return context
_TOKEN_RE = re.compile(r"^\{([A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*)\}$")
_PARTIAL_TOKEN_RE = re.compile(r"\{([A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*)\}")
def resolve_template(value: Any, context: dict[str, Any]) -> Any:
"""Resolve ``{tokens}`` in config values against the manifest context."""
if isinstance(value, str):
whole = _TOKEN_RE.match(value)
if whole:
return _get_dotted(context, whole.group(1))
def replace(match: re.Match[str]) -> str:
resolved = _get_dotted(context, match.group(1))
return str(resolved)
return _PARTIAL_TOKEN_RE.sub(replace, value)
if isinstance(value, list):
return [resolve_template(item, context) for item in value]
if isinstance(value, dict):
return {key: resolve_template(val, context) for key, val in value.items()}
return value
def save_login_state(
*,
storage_state: str | Path,
login_url: str = DEFAULT_LOGIN_URL,
headless: bool = False,
wait_seconds: int = 0,
) -> Path:
"""Open a browser for manual login and save the authenticated state."""
if headless and wait_seconds <= 0:
raise ValueError("headless browser-login requires --wait-seconds > 0")
sync_playwright, _ = _require_playwright()
state_path = Path(storage_state).expanduser()
state_path.parent.mkdir(parents=True, exist_ok=True)
with sync_playwright() as pw:
browser = pw.chromium.launch(headless=headless)
context = browser.new_context()
page = context.new_page()
page.goto(login_url, wait_until="domcontentloaded")
if wait_seconds > 0:
page.wait_for_timeout(wait_seconds * 1000)
else:
input("Log in to JoinQuant in the opened browser, then press Enter here to save state...")
context.storage_state(path=str(state_path))
browser.close()
state_path.chmod(0o600)
return state_path
def save_login_state_from_env(
*,
env_path: str | Path,
storage_state: str | Path,
login_url: str = DEFAULT_LOGIN_URL,
headless: bool = True,
out_dir: str | Path | None = None,
timeout_ms: int = 120_000,
) -> dict[str, Any]:
"""Try to log in with env-file credentials and save browser state.
This handles the normal password-login form. If JoinQuant presents CAPTCHA,
slide verification, SMS verification, or 2FA, the report will mark the run
as not logged in and save a screenshot for manual diagnosis.
"""
env = load_env_file(env_path)
username = env.get("JOINQUANT_USERNAME")
password = env.get("JOINQUANT_PASSWORD")
if not username or not password:
raise ValueError("JOINQUANT_USERNAME and JOINQUANT_PASSWORD are required")
sync_playwright, _ = _require_playwright()
state_path = Path(storage_state).expanduser()
state_path.parent.mkdir(parents=True, exist_ok=True)
artifact_dir = Path(out_dir or state_path.parent / "joinquant_login_artifacts")
artifact_dir.mkdir(parents=True, exist_ok=True)
screenshot_path = artifact_dir / "login_after_submit.png"
html_path = artifact_dir / "login_after_submit.html"
with sync_playwright() as pw:
browser = pw.chromium.launch(headless=headless)
context = browser.new_context()
page = context.new_page()
page.set_default_timeout(timeout_ms)
page.goto(login_url, wait_until="networkidle", timeout=timeout_ms)
page.locator('input[name="username"], input.pwd-phone').first.fill(username)
page.locator('input[name="pwd"], input.jq-login__password').first.fill(password)
checkbox = page.locator("#agreementBox, input.agreement-box").first
if checkbox.count():
checkbox.check(force=True)
page.locator("button.btnPwdSubmit, button.login-submit").first.click()
page.wait_for_timeout(5000)
html = page.content()
html_path.write_text(html, encoding="utf-8")
page.screenshot(path=str(screenshot_path), full_page=True)
login_inputs = page.locator('input[name="username"], input[name="pwd"]').count()
current_url = page.url
logged_in = login_inputs == 0 and "/user/login" not in current_url
if logged_in:
context.storage_state(path=str(state_path))
state_path.chmod(0o600)
browser.close()
report = {
"created_at": datetime.now(timezone.utc).isoformat(),
"logged_in": bool(logged_in),
"storage_state": str(state_path) if logged_in else "",
"artifact_dir": str(artifact_dir),
"screenshot": str(screenshot_path),
"html": str(html_path),
"current_url": current_url,
"notes": (
"Logged in and saved browser state."
if logged_in
else "Login did not complete. CAPTCHA/SMS/2FA or invalid credentials may be required."
),
}
report_path = artifact_dir / "login_report.json"
report["report_path"] = str(report_path)
report_path.write_text(json.dumps(report, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
return report
def browser_snapshot(
*,
url: str,
storage_state: str | Path,
out_dir: str | Path,
headless: bool = True,
timeout_ms: int = 60_000,
) -> dict[str, Path]:
"""Save a logged-in page screenshot and HTML for selector discovery."""
sync_playwright, _ = _require_playwright()
root = Path(out_dir)
root.mkdir(parents=True, exist_ok=True)
html_path = root / "page.html"
screenshot_path = root / "page.png"
with sync_playwright() as pw:
browser = pw.chromium.launch(headless=headless)
context = browser.new_context(storage_state=str(Path(storage_state).expanduser()))
page = context.new_page()
page.set_default_timeout(timeout_ms)
page.goto(url, wait_until="networkidle")
html_path.write_text(page.content(), encoding="utf-8")
page.screenshot(path=str(screenshot_path), full_page=True)
browser.close()
return {"html": html_path, "screenshot": screenshot_path}
def _locator(page: Any, selector: str):
return page.locator(selector).first
def _action_fail(action: dict[str, Any], exc: Exception) -> None:
if action.get("optional", False):
return
raise exc
def _run_action(page: Any, action: dict[str, Any], context: dict[str, Any], timeout_default: int) -> dict[str, Any]:
resolved = resolve_template(action, context)
kind = resolved["type"]
timeout_ms = int(resolved.get("timeout_ms", timeout_default))
record: dict[str, Any] = {"type": kind, "status": "ok"}
try:
if kind == "goto":
page.goto(str(resolved["url"]), wait_until=resolved.get("wait_until", "domcontentloaded"), timeout=timeout_ms)
elif kind == "click":
_locator(page, str(resolved["selector"])).click(
timeout=timeout_ms,
force=bool(resolved.get("force", False)),
)
elif kind == "fill":
_locator(page, str(resolved["selector"])).fill(str(resolved.get("text", "")), timeout=timeout_ms)
elif kind == "press":
_locator(page, str(resolved["selector"])).press(str(resolved["key"]), timeout=timeout_ms)
elif kind == "set_input_files":
paths = resolved.get("paths", [])
if isinstance(paths, str):
paths = [path for path in paths.split(",") if path]
_locator(page, str(resolved["selector"])).set_input_files(paths, timeout=timeout_ms)
record["n_files"] = len(paths)
elif kind == "paste_text_file":
text = Path(str(resolved["path"])).read_text(encoding="utf-8")
loc = _locator(page, str(resolved["selector"]))
loc.click(timeout=timeout_ms)
page.keyboard.press("Control+A")
page.keyboard.insert_text(text)
record["n_chars"] = len(text)
elif kind == "set_ace_editor_file":
text = Path(str(resolved["path"])).read_text(encoding="utf-8")
selector = str(resolved.get("selector", ".ace_editor"))
page.wait_for_function(
"(selector) => Boolean(window.ace && document.querySelector(selector))",
arg=selector,
timeout=timeout_ms,
)
page.evaluate(
"""(arg) => {
const node = document.querySelector(arg.selector);
if (!node || !window.ace) {
throw new Error(`Ace editor not found for ${arg.selector}`);
}
const editor = window.ace.edit(node);
editor.setValue(arg.text, -1);
editor.clearSelection();
editor.focus();
return editor.getValue().length;
}""",
{"selector": selector, "text": text},
)
record["n_chars"] = len(text)
elif kind == "evaluate":
page.evaluate(str(resolved["script"]), resolved.get("arg"))
elif kind == "wait_for_selector":
page.wait_for_selector(str(resolved["selector"]), timeout=timeout_ms)
elif kind == "wait_for_timeout":
page.wait_for_timeout(int(resolved.get("timeout_ms", 1000)))
elif kind == "download":
save_as = Path(str(resolved["save_as"]))
save_as.parent.mkdir(parents=True, exist_ok=True)
with page.expect_download(timeout=timeout_ms) as download_info:
_locator(page, str(resolved["selector"])).click(timeout=timeout_ms)
download = download_info.value
download.save_as(str(save_as))
record["path"] = str(save_as)
elif kind == "screenshot":
path = Path(str(resolved["path"]))
path.parent.mkdir(parents=True, exist_ok=True)
page.screenshot(path=str(path), full_page=bool(resolved.get("full_page", True)))
record["path"] = str(path)
else:
raise ValueError(f"Unsupported browser action type: {kind}")
except Exception as exc: # pragma: no cover - exercised only with Playwright.
record["status"] = "skipped" if resolved.get("optional", False) else "failed"
record["error"] = str(exc)
_action_fail(resolved, exc)
return record
def run_browser_flow(
*,
manifest_path: str | Path,
config_path: str | Path,
storage_state: str | Path,
out_dir: str | Path | None = None,
headless: bool | None = None,
auto_reconcile: bool = True,
flow_name: str | None = None,
) -> dict[str, Any]:
"""Run configured browser automation for a JoinQuant web workflow."""
sync_playwright, _ = _require_playwright()
manifest = load_json(manifest_path)
config = load_json(config_path)
flow = flow_name or str(config.get("flow") or "browser")
artifact_dir = Path(out_dir or Path(str(manifest["joinquant_export_dir"])).parent / f"browser_{flow}")
artifact_dir.mkdir(parents=True, exist_ok=True)
context_vars = _manifest_context(manifest, artifact_dir, config)
timeout_ms = int(config.get("timeout_ms", 120_000))
actions = config.get("actions") or []
if not actions:
raise ValueError("Browser config contains no actions")
records: list[dict[str, Any]] = []
failure: Exception | None = None
failure_artifacts: dict[str, str] = {}
with sync_playwright() as pw:
browser = pw.chromium.launch(
headless=bool(config.get("headless", False) if headless is None else headless)
)
context = browser.new_context(
storage_state=str(Path(storage_state).expanduser()),
accept_downloads=True,
)
page = context.new_page()
page.set_default_timeout(timeout_ms)
try:
for action in actions:
records.append(_run_action(page, action, context_vars, timeout_ms))
except Exception as exc: # pragma: no cover - requires live browser UI.
failure = exc
screenshot_path = artifact_dir / "failure.png"
html_path = artifact_dir / "failure.html"
try:
page.screenshot(path=str(screenshot_path), full_page=True)
html_path.write_text(page.content(), encoding="utf-8")
failure_artifacts = {
"screenshot": str(screenshot_path),
"html": str(html_path),
}
except Exception as artifact_exc:
failure_artifacts = {"artifact_error": str(artifact_exc)}
records.append({
"type": "browser_flow",
"status": "failed",
"error": str(exc),
**failure_artifacts,
})
browser.close()
expected = manifest.get("expected_joinquant_csvs", {})
downloaded = {
key: str(path)
for key, value in expected.items()
if (path := Path(str(value))).exists()
}
reconcile_paths: dict[str, str] = {}
if auto_reconcile and {"fills", "positions", "pnl"}.issubset(downloaded):
ingested = ingest_joinquant_outputs(
portfolio_name=str(manifest["portfolio_name"]),
fills_csv=downloaded["fills"],
positions_csv=downloaded["positions"],
pnl_csv=downloaded["pnl"],
out_dir=Path(str(manifest["joinquant_export_dir"])).parent / "ingested",
)
reconciled = reconcile_joinquant(
portfolio_name=str(manifest["portfolio_name"]),
targets_dir=str(manifest["targets_dir"]),
our_fills_path=str(manifest["fills_path"]),
our_positions_path=str(manifest["positions_path"]),
our_pnl_path=str(manifest["pnl_path"]),
jq_fills_path=str(ingested["fills"]),
jq_positions_path=str(ingested["positions"]),
jq_pnl_path=str(ingested["pnl"]),
out_dir=Path(str(manifest["joinquant_export_dir"])).parent / "reconcile",
)
reconcile_paths = {key: str(value) for key, value in reconciled.items()}
report = {
"created_at": datetime.now(timezone.utc).isoformat(),
"manifest_path": str(manifest_path),
"config_path": str(config_path),
"flow": flow,
"status": "failed" if failure else "ok",
"storage_state": str(storage_state),
"artifact_dir": str(artifact_dir),
"actions": records,
"downloaded": downloaded,
"reconcile_paths": reconcile_paths,
}
report_path = artifact_dir / "browser_run_report.json"
report["report_path"] = str(report_path)
report_path.write_text(
json.dumps(report, indent=2, ensure_ascii=False) + "\n",
encoding="utf-8",
)
if failure is not None:
raise RuntimeError(
f"Browser flow failed; report saved to {report_path}: {failure}"
) from failure
return report
def run_browser_backtest(
*,
manifest_path: str | Path,
config_path: str | Path,
storage_state: str | Path,
out_dir: str | Path | None = None,
headless: bool | None = None,
auto_reconcile: bool = True,
) -> dict[str, Any]:
"""Run configured browser automation for a JoinQuant backtest."""
return run_browser_flow(
manifest_path=manifest_path,
config_path=config_path,
storage_state=storage_state,
out_dir=out_dir,
headless=headless,
auto_reconcile=auto_reconcile,
flow_name="backtest",
)
def run_browser_sim_trade(
*,
manifest_path: str | Path,
config_path: str | Path,
storage_state: str | Path,
out_dir: str | Path | None = None,
headless: bool | None = None,
auto_reconcile: bool = True,
) -> dict[str, Any]:
"""Run configured browser automation for JoinQuant 模拟盘."""
return run_browser_flow(
manifest_path=manifest_path,
config_path=config_path,
storage_state=storage_state,
out_dir=out_dir,
headless=headless,
auto_reconcile=auto_reconcile,
flow_name="sim-trade",
)
+426
View File
@@ -0,0 +1,426 @@
"""CLI commands for the JoinQuant comparison plugin."""
from __future__ import annotations
import click
from plugins.joinquant.browser import (
browser_snapshot,
load_env_file,
run_browser_backtest,
run_browser_sim_trade,
save_login_state,
save_login_state_from_env,
write_browser_config_template,
)
from plugins.joinquant.export_targets import export_targets
from plugins.joinquant.ingest import ingest_joinquant_outputs
from plugins.joinquant.reconcile import reconcile_joinquant
from plugins.joinquant.smoke import prepare_smoke_test
from plugins.joinquant.wrapper_strategy import write_wrapper_strategy
@click.group(name="joinquant")
def joinquant():
"""Compare internal portfolio simulation with JoinQuant output."""
@joinquant.command("export-targets")
@click.option("--positions-path", required=True, help="Portfolio positions parquet from `portfolio build`")
@click.option("--portfolio-name", required=True, help="Portfolio run to export")
@click.option(
"--mode",
"mode",
type=click.Choice(["target_shares", "target_value"]),
default="target_shares",
show_default=True,
help="JoinQuant target order mode",
)
@click.option("--start-date", default=None, help="Inclusive YYYY-MM-DD start date")
@click.option("--end-date", default=None, help="Inclusive YYYY-MM-DD end date")
@click.option(
"--execution-calendar-path",
default=None,
help="Daily data parquet/dataset used to shift position dates to next execution session",
)
@click.option("--out-dir", default="plugins_output/joinquant/targets", show_default=True)
@click.option("--force", is_flag=True, help="Overwrite frozen target/snapshot files")
def export_targets_cmd(
positions_path,
portfolio_name,
mode,
start_date,
end_date,
execution_calendar_path,
out_dir,
force,
):
"""Export frozen daily target files for JoinQuant."""
snapshots = export_targets(
positions_path=positions_path,
portfolio_name=portfolio_name,
mode=mode,
start_date=start_date,
end_date=end_date,
execution_calendar_path=execution_calendar_path,
out_dir=out_dir,
force=force,
)
click.echo(f"Exported JoinQuant targets: {len(snapshots)} day(s)")
for snapshot in snapshots:
click.echo(
f" {snapshot['date']}: {snapshot['n_symbols']} symbols, "
f"sha256={str(snapshot['file_sha256'])[:12]}"
)
@joinquant.command("ingest")
@click.option("--portfolio-name", required=True, help="Portfolio run name")
@click.option("--fills-csv", required=True, help="JoinQuant fills CSV")
@click.option("--positions-csv", required=True, help="JoinQuant positions CSV")
@click.option("--pnl-csv", required=True, help="JoinQuant daily PnL CSV")
@click.option("--out-dir", default="plugins_output/joinquant/ingested", show_default=True)
def ingest_cmd(portfolio_name, fills_csv, positions_csv, pnl_csv, out_dir):
"""Normalize JoinQuant CSV exports to parquet."""
paths = ingest_joinquant_outputs(
portfolio_name=portfolio_name,
fills_csv=fills_csv,
positions_csv=positions_csv,
pnl_csv=pnl_csv,
out_dir=out_dir,
)
click.echo(f"Saved JoinQuant fills: {paths['fills']}")
click.echo(f"Saved JoinQuant positions: {paths['positions']}")
click.echo(f"Saved JoinQuant pnl: {paths['pnl']}")
@joinquant.command("reconcile")
@click.option("--portfolio-name", required=True, help="Portfolio run name")
@click.option("--targets-dir", required=True, help="Directory containing exported daily target files")
@click.option("--our-fills-path", required=True, help="Internal simulator fills parquet")
@click.option("--our-positions-path", required=True, help="Internal portfolio positions parquet")
@click.option("--our-pnl-path", required=True, help="Internal simulator PnL parquet")
@click.option("--jq-fills-path", required=True, help="Normalized JoinQuant fills parquet")
@click.option("--jq-positions-path", required=True, help="Normalized JoinQuant positions parquet")
@click.option("--jq-pnl-path", required=True, help="Normalized JoinQuant PnL parquet")
@click.option("--out-dir", default="plugins_output/joinquant/reconcile", show_default=True)
@click.option("--share-tolerance", default=0.0, show_default=True, type=float)
@click.option("--price-rel-tolerance", default=1e-4, show_default=True, type=float)
@click.option("--pnl-tolerance", default=1.0, show_default=True, type=float)
@click.option("--booksize", default=None, type=float, help="Booksize for value tolerance inference")
def reconcile_cmd(
portfolio_name,
targets_dir,
our_fills_path,
our_positions_path,
our_pnl_path,
jq_fills_path,
jq_positions_path,
jq_pnl_path,
out_dir,
share_tolerance,
price_rel_tolerance,
pnl_tolerance,
booksize,
):
"""Write per-symbol and daily JoinQuant reconciliation reports."""
paths = reconcile_joinquant(
portfolio_name=portfolio_name,
targets_dir=targets_dir,
our_fills_path=our_fills_path,
our_positions_path=our_positions_path,
our_pnl_path=our_pnl_path,
jq_fills_path=jq_fills_path,
jq_positions_path=jq_positions_path,
jq_pnl_path=jq_pnl_path,
out_dir=out_dir,
share_tolerance=share_tolerance,
price_rel_tolerance=price_rel_tolerance,
pnl_tolerance=pnl_tolerance,
booksize=booksize,
)
click.echo(f"Saved reconciliation parquet: {paths['daily_reconcile']}")
click.echo(f"Saved reconciliation summary: {paths['summary_md']}")
click.echo(f"Saved reconciliation CSV: {paths['summary_csv']}")
@joinquant.command("write-wrapper")
@click.option("--portfolio-name", required=True, help="Portfolio run name")
@click.option(
"--mode",
"mode",
type=click.Choice(["target_shares", "target_value"]),
default="target_shares",
show_default=True,
)
@click.option("--out-path", required=True, help="Path for generated standalone strategy")
@click.option("--allow-short", is_flag=True, help="Do not clip negative targets in the generated wrapper")
@click.option(
"--embed-targets-dir",
default=None,
help="Embed CSV target files from this directory into the strategy source",
)
def write_wrapper_cmd(portfolio_name, mode, out_path, allow_short, embed_targets_dir):
"""Generate a standalone JoinQuant wrapper strategy."""
path = write_wrapper_strategy(
portfolio_name=portfolio_name,
mode=mode,
out_path=out_path,
allow_short=allow_short,
embedded_targets_dir=embed_targets_dir,
)
click.echo(f"Saved JoinQuant wrapper strategy: {path}")
@joinquant.command("prepare-smoke")
@click.option("--out-dir", required=True, help="Root directory for generated smoke-test artifacts")
@click.option(
"--universe",
default="sh600000,sz000001,sh600519,sz002594,sz300750",
show_default=True,
help="Universe for the tiny real-data download",
)
@click.option("--trade-symbol", default="sh600000", show_default=True)
@click.option("--start-date", default="2024-01-02", show_default=True)
@click.option("--end-date", default="2024-01-12", show_default=True)
@click.option("--portfolio-name", default="jq_smoke_one_stock_long", show_default=True)
@click.option("--shares", default=1000, show_default=True, type=int)
@click.option("--booksize", default=1_000_000.0, show_default=True, type=float)
@click.option("--max-signal-dates", default=3, show_default=True, type=int)
@click.option("--cost-bps", default=5.0, show_default=True, type=float)
@click.option("--slippage-bps", default=5.0, show_default=True, type=float)
@click.option("--volume-frac", default=0.02, show_default=True, type=float)
@click.option("--force", is_flag=True, help="Overwrite existing frozen target files")
def prepare_smoke_cmd(
out_dir,
universe,
trade_symbol,
start_date,
end_date,
portfolio_name,
shares,
booksize,
max_signal_dates,
cost_bps,
slippage_bps,
volume_frac,
force,
):
"""Prepare a one-command local real-data JoinQuant smoke test."""
manifest = prepare_smoke_test(
out_dir=out_dir,
universe=universe,
trade_symbol=trade_symbol,
start_date=start_date,
end_date=end_date,
portfolio_name=portfolio_name,
shares=shares,
booksize=booksize,
max_signal_dates=max_signal_dates,
cost_bps=cost_bps,
slippage_bps=slippage_bps,
volume_frac=volume_frac,
force=force,
)
click.echo(f"Prepared JoinQuant smoke manifest: {manifest['manifest_path']}")
click.echo(f"Wrapper: {manifest['wrapper_path']}")
click.echo(f"Targets: {manifest['targets_dir']}")
click.echo(f"Expected JoinQuant exports: {manifest['joinquant_export_dir']}")
summary = manifest["local_summary"]
click.echo(
f"Local simulator: {summary['n_pnl_rows']} days, "
f"PnL={summary['total_pnl']:,.2f}, cost={summary['total_cost']:,.2f}, "
f"blocked={summary['blocked_trades']}"
)
@joinquant.command("browser-login")
@click.option(
"--storage-state",
default="~/.config/chinese-equity-quant/joinquant_storage_state.json",
show_default=True,
help="Path where the authenticated browser state is stored",
)
@click.option(
"--login-url",
default="https://www.joinquant.com/user/login/index",
show_default=True,
)
@click.option("--headless", is_flag=True, help="Use a headless browser")
@click.option(
"--wait-seconds",
default=0,
show_default=True,
type=int,
help="Seconds to wait before saving state; 0 prompts for Enter after login",
)
def browser_login_cmd(storage_state, login_url, headless, wait_seconds):
"""Open JoinQuant login and save reusable browser session state."""
path = save_login_state(
storage_state=storage_state,
login_url=login_url,
headless=headless,
wait_seconds=wait_seconds,
)
click.echo(f"Saved JoinQuant browser state: {path}")
@joinquant.command("browser-login-env")
@click.option(
"--env-path",
default="~/.config/chinese-equity-quant/joinquant.env",
show_default=True,
help="Env file with JOINQUANT_USERNAME and JOINQUANT_PASSWORD",
)
@click.option(
"--storage-state",
default="~/.config/chinese-equity-quant/joinquant_storage_state.json",
show_default=True,
help="Path where the authenticated browser state is stored",
)
@click.option(
"--login-url",
default="https://www.joinquant.com/user/login/index",
show_default=True,
)
@click.option("--headed", is_flag=True, help="Run with a visible browser")
@click.option("--out-dir", default=None, help="Login artifact directory")
@click.option("--timeout-ms", default=120_000, show_default=True, type=int)
def browser_login_env_cmd(env_path, storage_state, login_url, headed, out_dir, timeout_ms):
"""Try credential-based JoinQuant login from an env file."""
env = load_env_file(env_path)
missing = [
key for key in ["JOINQUANT_USERNAME", "JOINQUANT_PASSWORD"]
if not env.get(key)
]
if missing:
raise click.ClickException(f"Missing required env keys: {', '.join(missing)}")
report = save_login_state_from_env(
env_path=env_path,
storage_state=storage_state,
login_url=login_url,
headless=not headed,
out_dir=out_dir,
timeout_ms=timeout_ms,
)
click.echo(f"Saved login report: {report['report_path']}")
if report["logged_in"]:
click.echo(f"Saved JoinQuant browser state: {report['storage_state']}")
else:
raise click.ClickException(
"JoinQuant login did not complete. Check the saved screenshot/HTML; "
"CAPTCHA/SMS/2FA may require `joinquant browser-login` once."
)
@joinquant.command("write-browser-config")
@click.option("--out-path", required=True, help="Path for JSON browser automation config")
@click.option("--strategy-url", default="", help="JoinQuant strategy edit/backtest/模拟盘 URL")
@click.option(
"--flow",
type=click.Choice(["backtest", "sim-trade"]),
default="backtest",
show_default=True,
help="Browser automation template to write",
)
def write_browser_config_cmd(out_path, strategy_url, flow):
"""Write a browser automation selector/action config template."""
path = write_browser_config_template(out_path, strategy_url=strategy_url, flow=flow)
click.echo(f"Saved JoinQuant browser config template: {path}")
@joinquant.command("browser-snapshot")
@click.option("--url", required=True, help="Logged-in JoinQuant page URL to inspect")
@click.option(
"--storage-state",
default="~/.config/chinese-equity-quant/joinquant_storage_state.json",
show_default=True,
)
@click.option("--out-dir", required=True, help="Directory for page.html and page.png")
@click.option("--headed", is_flag=True, help="Run with a visible browser")
@click.option("--timeout-ms", default=60_000, show_default=True, type=int)
def browser_snapshot_cmd(url, storage_state, out_dir, headed, timeout_ms):
"""Save a page screenshot and HTML for selector discovery."""
paths = browser_snapshot(
url=url,
storage_state=storage_state,
out_dir=out_dir,
headless=not headed,
timeout_ms=timeout_ms,
)
click.echo(f"Saved HTML: {paths['html']}")
click.echo(f"Saved screenshot: {paths['screenshot']}")
@joinquant.command("run-browser-backtest")
@click.option("--manifest-path", required=True, help="Manifest from `joinquant prepare-smoke`")
@click.option("--config-path", required=True, help="JSON browser automation config")
@click.option(
"--storage-state",
default="~/.config/chinese-equity-quant/joinquant_storage_state.json",
show_default=True,
)
@click.option("--out-dir", default=None, help="Browser-run artifact directory")
@click.option("--headed", is_flag=True, help="Run with a visible browser")
@click.option("--no-auto-reconcile", is_flag=True, help="Skip ingest/reconcile after downloads")
def run_browser_backtest_cmd(
manifest_path,
config_path,
storage_state,
out_dir,
headed,
no_auto_reconcile,
):
"""Run JoinQuant browser automation from manifest and config."""
report = run_browser_backtest(
manifest_path=manifest_path,
config_path=config_path,
storage_state=storage_state,
out_dir=out_dir,
headless=not headed,
auto_reconcile=not no_auto_reconcile,
)
click.echo(f"Saved browser run report: {report['report_path']}")
if report["downloaded"]:
click.echo("Downloaded JoinQuant CSVs:")
for key, path in report["downloaded"].items():
click.echo(f" {key}: {path}")
if report["reconcile_paths"]:
click.echo(f"Saved reconciliation summary: {report['reconcile_paths']['summary_md']}")
@joinquant.command("run-browser-sim")
@click.option("--manifest-path", required=True, help="Manifest from target preparation")
@click.option("--config-path", required=True, help="JSON simulated-trading browser config")
@click.option(
"--storage-state",
default="~/.config/chinese-equity-quant/joinquant_storage_state.json",
show_default=True,
)
@click.option("--out-dir", default=None, help="Browser-run artifact directory")
@click.option("--headed", is_flag=True, help="Run with a visible browser")
@click.option("--no-auto-reconcile", is_flag=True, help="Skip ingest/reconcile after downloads")
def run_browser_sim_cmd(
manifest_path,
config_path,
storage_state,
out_dir,
headed,
no_auto_reconcile,
):
"""Run JoinQuant 模拟盘 browser automation from manifest and config."""
report = run_browser_sim_trade(
manifest_path=manifest_path,
config_path=config_path,
storage_state=storage_state,
out_dir=out_dir,
headless=not headed,
auto_reconcile=not no_auto_reconcile,
)
click.echo(f"Saved browser sim-trade report: {report['report_path']}")
if report["downloaded"]:
click.echo("Downloaded JoinQuant CSVs:")
for key, path in report["downloaded"].items():
click.echo(f" {key}: {path}")
if report["reconcile_paths"]:
click.echo(f"Saved reconciliation summary: {report['reconcile_paths']['summary_md']}")
+248
View File
@@ -0,0 +1,248 @@
"""Export portfolio positions as frozen JoinQuant target files."""
from __future__ import annotations
import hashlib
import json
import uuid
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterable, Literal
import pandas as pd
from pipeline.common.schema import POSITION_COLUMNS
from plugins.joinquant.schema import JOINQUANT_TARGET_COLUMNS
from plugins.joinquant.symbols import to_joinquant_symbol
ExportMode = Literal["target_shares", "target_value"]
def _date_text(value: object) -> str:
return pd.Timestamp(value).strftime("%Y-%m-%d")
def _date_file_stem(date_text: str) -> str:
return pd.Timestamp(date_text).strftime("%Y%m%d")
def _snapshot_root_for(targets_root: Path) -> Path:
if targets_root.name == "targets":
return targets_root.parent / "snapshots"
return targets_root / "snapshots"
def _sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as fh:
for chunk in iter(lambda: fh.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _check_position_columns(df: pd.DataFrame) -> None:
missing = [col for col in POSITION_COLUMNS if col not in df.columns]
if missing:
raise ValueError(f"Positions input missing required columns: {missing}")
def _filter_dates(
df: pd.DataFrame,
start_date: str | None,
end_date: str | None,
*,
date_column: str = "date",
) -> pd.DataFrame:
out = df.copy()
out[date_column] = pd.to_datetime(out[date_column]).dt.normalize()
if start_date:
out = out[out[date_column] >= pd.Timestamp(start_date).normalize()]
if end_date:
out = out[out[date_column] <= pd.Timestamp(end_date).normalize()]
return out
def _read_execution_calendar(path: str | Path) -> pd.DatetimeIndex:
data = pd.read_parquet(path)
if "date" not in data.columns:
raise ValueError("execution calendar parquet must contain a 'date' column")
dates = pd.to_datetime(data["date"], errors="coerce").dropna().dt.normalize()
return pd.DatetimeIndex(sorted(dates.unique()))
def _apply_execution_calendar(df: pd.DataFrame, calendar_path: str | Path) -> pd.DataFrame:
calendar = _read_execution_calendar(calendar_path)
if calendar.empty:
raise ValueError("execution calendar contains no dates")
source_dates = pd.to_datetime(df["date"]).dt.normalize()
positions = calendar.searchsorted(source_dates, side="right")
out = df.copy()
out["source_date"] = source_dates
out["export_date"] = pd.NaT
valid = positions < len(calendar)
out.loc[valid, "export_date"] = calendar.take(positions[valid])
return out[out["export_date"].notna()].copy()
def build_target_frame(
positions: pd.DataFrame,
*,
portfolio_name: str | None = None,
mode: ExportMode = "target_shares",
start_date: str | None = None,
end_date: str | None = None,
snapshot_ids: dict[str, str] | None = None,
execution_calendar_path: str | Path | None = None,
) -> pd.DataFrame:
"""Build normalized JoinQuant target rows from portfolio positions.
``target_shares`` is populated from ``position_shares`` because the core
simulator executes the discretized book, not continuous research shares.
"""
if mode not in {"target_shares", "target_value"}:
raise ValueError("mode must be 'target_shares' or 'target_value'")
_check_position_columns(positions)
df = positions.copy()
df["date"] = pd.to_datetime(df["date"]).dt.normalize()
if execution_calendar_path is not None:
df = _apply_execution_calendar(df, execution_calendar_path)
df = df.drop(columns=["date"]).rename(columns={"export_date": "date"})
df["source_date"] = pd.to_datetime(df["source_date"]).dt.strftime("%Y-%m-%d")
df = _filter_dates(df, start_date, end_date)
else:
df = _filter_dates(df, start_date, end_date)
if portfolio_name is not None:
df = df[df["portfolio_name"].astype(str) == portfolio_name]
if df.empty:
return pd.DataFrame(columns=JOINQUANT_TARGET_COLUMNS)
out = pd.DataFrame({
"date": df["date"].map(_date_text),
"portfolio_name": df["portfolio_name"].astype(str),
"symbol_id": df["symbol_id"].astype(str),
"jq_symbol": df["symbol_id"].map(to_joinquant_symbol),
"target_shares": pd.to_numeric(df["position_shares"], errors="coerce").fillna(0).astype("int64"),
"target_value": pd.to_numeric(df["target_value"], errors="coerce").fillna(0.0),
"target_weight": pd.to_numeric(df["target_weight"], errors="coerce").fillna(0.0),
"export_mode": mode,
"snapshot_id": "",
})
if snapshot_ids:
out["snapshot_id"] = out["date"].map(snapshot_ids).fillna("")
return out[JOINQUANT_TARGET_COLUMNS].sort_values(
["date", "portfolio_name", "symbol_id"]
).reset_index(drop=True)
def export_targets(
positions_path: str | Path,
*,
portfolio_name: str,
mode: ExportMode = "target_shares",
out_dir: str | Path = "plugins_output/joinquant/targets",
start_date: str | None = None,
end_date: str | None = None,
execution_calendar_path: str | Path | None = None,
force: bool = False,
) -> list[dict[str, object]]:
"""Export one daily CSV/parquet target file plus a snapshot JSON per date.
Args:
positions_path: Parquet file produced by ``portfolio build``.
portfolio_name: Portfolio run to export.
mode: ``target_shares`` or ``target_value``.
out_dir: Target root. Files are written to ``out_dir/portfolio_name``.
If the root is named ``targets``, snapshots are written to the
sibling ``snapshots`` directory.
start_date: Optional inclusive start date.
end_date: Optional inclusive end date.
execution_calendar_path: Optional daily-bar parquet dataset used to
shift each position date to the next available execution session.
This matches the internal simulator's next-open convention.
force: If false, existing target or snapshot files are treated as
frozen and cause ``FileExistsError``.
Returns:
Snapshot metadata dictionaries, one per exported date.
"""
positions_path = Path(positions_path)
targets_root = Path(out_dir)
snapshot_root = _snapshot_root_for(targets_root)
targets_portfolio_dir = targets_root / portfolio_name
snapshots_portfolio_dir = snapshot_root / portfolio_name
targets_portfolio_dir.mkdir(parents=True, exist_ok=True)
snapshots_portfolio_dir.mkdir(parents=True, exist_ok=True)
positions = pd.read_parquet(positions_path)
filtered = positions.copy()
filtered["date"] = pd.to_datetime(filtered["date"]).dt.normalize()
if execution_calendar_path is not None:
filtered = _apply_execution_calendar(filtered, execution_calendar_path)
filtered = filtered.drop(columns=["date"]).rename(columns={"export_date": "date"})
filtered["source_date"] = pd.to_datetime(filtered["source_date"]).dt.strftime("%Y-%m-%d")
filtered = _filter_dates(filtered, start_date, end_date)
else:
filtered = _filter_dates(filtered, start_date, end_date)
filtered = filtered[filtered["portfolio_name"].astype(str) == portfolio_name]
if filtered.empty:
return []
date_texts = sorted(filtered["date"].map(_date_text).unique())
snapshot_ids = {
date_text: f"jq-{portfolio_name}-{date_text}-{uuid.uuid4().hex[:12]}"
for date_text in date_texts
}
targets = build_target_frame(
filtered,
portfolio_name=portfolio_name,
mode=mode,
snapshot_ids=snapshot_ids,
execution_calendar_path=None,
)
snapshots: list[dict[str, object]] = []
for date_text, daily in targets.groupby("date", sort=True):
stem = _date_file_stem(date_text)
csv_path = targets_portfolio_dir / f"{stem}.csv"
parquet_path = targets_portfolio_dir / f"{stem}.parquet"
snapshot_path = snapshots_portfolio_dir / f"{stem}.json"
existing: Iterable[Path] = (csv_path, parquet_path, snapshot_path)
if not force:
conflicts = [str(path) for path in existing if path.exists()]
if conflicts:
raise FileExistsError(
"Frozen JoinQuant target already exists; use --force to overwrite: "
+ ", ".join(conflicts)
)
daily = daily[JOINQUANT_TARGET_COLUMNS].reset_index(drop=True)
daily.to_csv(csv_path, index=False)
daily.to_parquet(parquet_path, index=False)
file_hash = _sha256_file(csv_path)
snapshot = {
"snapshot_id": snapshot_ids[date_text],
"portfolio_name": portfolio_name,
"date": date_text,
"export_mode": mode,
"source_positions_path": str(positions_path),
"execution_calendar_path": str(execution_calendar_path) if execution_calendar_path else None,
"created_at": datetime.now(timezone.utc).isoformat(),
"n_symbols": int(len(daily)),
"file_sha256": file_hash,
"notes": "Frozen JoinQuant target file.",
"target_csv_path": str(csv_path),
"target_parquet_path": str(parquet_path),
}
snapshot_path.write_text(
json.dumps(snapshot, indent=2, ensure_ascii=False) + "\n",
encoding="utf-8",
)
snapshots.append(snapshot)
return snapshots
+225
View File
@@ -0,0 +1,225 @@
"""Normalize JoinQuant CSV exports into plugin parquet schemas."""
from __future__ import annotations
import re
from pathlib import Path
import numpy as np
import pandas as pd
from plugins.joinquant.schema import (
JOINQUANT_FILL_COLUMNS,
JOINQUANT_PNL_COLUMNS,
JOINQUANT_POSITION_COLUMNS,
)
from plugins.joinquant.symbols import normalize_symbol_pair, to_joinquant_symbol
def _clean_name(name: object) -> str:
text = str(name).strip().lower()
text = re.sub(r"[\s\-.()/]+", "_", text)
return re.sub(r"[^0-9a-z_]+", "", text).strip("_")
def _clean_columns(df: pd.DataFrame) -> pd.DataFrame:
out = df.copy()
out.columns = [_clean_name(col) for col in out.columns]
return out
def _pick(df: pd.DataFrame, candidates: list[str]) -> str | None:
for candidate in candidates:
clean = _clean_name(candidate)
if clean in df.columns:
return clean
return None
def _series_or_default(df: pd.DataFrame, candidates: list[str], default: object) -> pd.Series:
col = _pick(df, candidates)
if col is None:
return pd.Series([default] * len(df), index=df.index)
return df[col]
def _date_series(df: pd.DataFrame) -> pd.Series:
values = _series_or_default(
df,
["date", "trade_date", "datetime", "time", "created_at"],
pd.NaT,
)
parsed = pd.to_datetime(values, errors="coerce").dt.normalize()
return parsed.dt.strftime("%Y-%m-%d").fillna("")
def _numeric(values: pd.Series, default: float = 0.0) -> pd.Series:
return pd.to_numeric(values, errors="coerce").replace([np.inf, -np.inf], np.nan).fillna(default)
def _text(values: pd.Series, default: str = "") -> pd.Series:
return values.fillna(default).astype(str)
def _portfolio_series(df: pd.DataFrame, portfolio_name: str) -> pd.Series:
return _text(_series_or_default(df, ["portfolio_name", "portfolio", "strategy"], portfolio_name), portfolio_name)
def _symbol_frame(df: pd.DataFrame) -> pd.DataFrame:
internal_col = _pick(df, ["symbol_id", "internal_symbol"])
jq_col = _pick(df, ["jq_symbol", "security", "stock", "symbol", "code", "order_book_id"])
symbol_ids: list[str] = []
jq_symbols: list[str] = []
for idx in df.index:
internal = df.at[idx, internal_col] if internal_col else None
jq_value = df.at[idx, jq_col] if jq_col else None
value = internal if internal is not None and str(internal).strip() else jq_value
if value is None or not str(value).strip():
symbol_ids.append("")
jq_symbols.append("")
continue
symbol_id, jq_symbol = normalize_symbol_pair(value)
if jq_value is not None and str(jq_value).strip():
try:
_, jq_symbol = normalize_symbol_pair(jq_value)
except ValueError:
jq_symbol = to_joinquant_symbol(symbol_id)
symbol_ids.append(symbol_id)
jq_symbols.append(jq_symbol)
return pd.DataFrame({"symbol_id": symbol_ids, "jq_symbol": jq_symbols}, index=df.index)
def _signed_shares(shares: pd.Series, side: pd.Series) -> pd.Series:
signed = _numeric(shares, 0.0)
side_text = side.fillna("").astype(str).str.lower()
sell = side_text.str.contains("sell|short|close|reduce|-", regex=True)
buy = side_text.str.contains("buy|long|open|add|\\+", regex=True)
signed = signed.abs()
signed = signed.mask(sell, -signed)
signed = signed.mask(~(sell | buy), _numeric(shares, 0.0))
return signed
def normalize_fills_csv(path: str | Path, portfolio_name: str) -> pd.DataFrame:
"""Read a JoinQuant fills CSV and return ``JOINQUANT_FILL_COLUMNS``."""
raw = _clean_columns(pd.read_csv(path))
symbols = _symbol_frame(raw)
side = _text(_series_or_default(raw, ["side", "action", "direction"], ""))
requested = _signed_shares(
_series_or_default(raw, ["requested_shares", "target_shares", "amount", "order_amount"], 0),
side,
)
filled = _signed_shares(
_series_or_default(raw, ["filled_shares", "filled", "filled_amount", "deal_amount", "traded_shares"], 0),
side,
)
price = _numeric(_series_or_default(raw, ["fill_price", "price", "avg_cost", "avg_price"], np.nan), np.nan)
trade_value = _numeric(
_series_or_default(raw, ["trade_value", "value", "filled_value", "turnover"], np.nan),
np.nan,
)
trade_value = trade_value.fillna((filled * price).abs()).fillna(0.0)
out = pd.DataFrame({
"date": _date_series(raw),
"portfolio_name": _portfolio_series(raw, portfolio_name),
"symbol_id": symbols["symbol_id"],
"jq_symbol": symbols["jq_symbol"],
"order_id": _text(_series_or_default(raw, ["order_id", "id"], "")),
"side": side,
"requested_shares": requested.astype(float),
"filled_shares": filled.astype(float),
"fill_price": price.astype(float),
"trade_value": trade_value.astype(float),
"trade_cost": _numeric(_series_or_default(raw, ["trade_cost", "cost", "commission", "fee"], 0.0), 0.0),
"blocked": _numeric(_series_or_default(raw, ["blocked", "is_blocked"], 0), 0).astype("int64"),
"raw_status": _text(_series_or_default(raw, ["raw_status", "status", "order_status"], "")),
})
return out[JOINQUANT_FILL_COLUMNS]
def normalize_positions_csv(path: str | Path, portfolio_name: str) -> pd.DataFrame:
"""Read a JoinQuant positions CSV and return ``JOINQUANT_POSITION_COLUMNS``."""
raw = _clean_columns(pd.read_csv(path))
symbols = _symbol_frame(raw)
out = pd.DataFrame({
"date": _date_series(raw),
"portfolio_name": _portfolio_series(raw, portfolio_name),
"symbol_id": symbols["symbol_id"],
"jq_symbol": symbols["jq_symbol"],
"position_shares": _numeric(
_series_or_default(raw, ["position_shares", "shares", "amount", "quantity", "total_amount"], 0),
0,
),
"position_value": _numeric(
_series_or_default(raw, ["position_value", "market_value", "value"], 0.0),
0.0,
),
"cash": _numeric(_series_or_default(raw, ["cash", "available_cash"], np.nan), np.nan),
"total_value": _numeric(
_series_or_default(raw, ["total_value", "portfolio_value", "total_asset"], np.nan),
np.nan,
),
})
return out[JOINQUANT_POSITION_COLUMNS]
def normalize_pnl_csv(path: str | Path, portfolio_name: str) -> pd.DataFrame:
"""Read a JoinQuant daily PnL CSV and return ``JOINQUANT_PNL_COLUMNS``."""
raw = _clean_columns(pd.read_csv(path))
total_value = _numeric(
_series_or_default(raw, ["total_value", "portfolio_value", "total_asset"], np.nan),
np.nan,
)
pnl = _numeric(_series_or_default(raw, ["pnl", "daily_pnl", "profit", "returns_value"], np.nan), np.nan)
if pnl.isna().all() and total_value.notna().any():
pnl = total_value.diff().fillna(0.0)
out = pd.DataFrame({
"date": _date_series(raw),
"portfolio_name": _portfolio_series(raw, portfolio_name),
"gross_exposure": _numeric(
_series_or_default(raw, ["gross_exposure", "gross", "positions_value", "market_value"], np.nan),
np.nan,
),
"net_exposure": _numeric(
_series_or_default(raw, ["net_exposure", "net"], np.nan),
np.nan,
),
"cash": _numeric(_series_or_default(raw, ["cash", "available_cash"], np.nan), np.nan),
"total_value": total_value,
"pnl": pnl.fillna(0.0),
"cost": _numeric(_series_or_default(raw, ["cost", "trade_cost", "commission", "fee"], 0.0), 0.0),
"turnover": _numeric(_series_or_default(raw, ["turnover"], 0.0), 0.0),
})
return out[JOINQUANT_PNL_COLUMNS]
def ingest_joinquant_outputs(
*,
portfolio_name: str,
fills_csv: str | Path,
positions_csv: str | Path,
pnl_csv: str | Path,
out_dir: str | Path = "plugins_output/joinquant/ingested",
) -> dict[str, Path]:
"""Normalize JoinQuant CSV exports and write parquet outputs."""
out_root = Path(out_dir) / portfolio_name
out_root.mkdir(parents=True, exist_ok=True)
fills = normalize_fills_csv(fills_csv, portfolio_name)
positions = normalize_positions_csv(positions_csv, portfolio_name)
pnl = normalize_pnl_csv(pnl_csv, portfolio_name)
paths = {
"fills": out_root / "fills.pq",
"positions": out_root / "positions.pq",
"pnl": out_root / "pnl.pq",
}
fills.to_parquet(paths["fills"], index=False)
positions.to_parquet(paths["positions"], index=False)
pnl.to_parquet(paths["pnl"], index=False)
return paths
+607
View File
@@ -0,0 +1,607 @@
"""Reconcile internal simulator output against normalized JoinQuant output."""
from __future__ import annotations
from pathlib import Path
import numpy as np
import pandas as pd
from plugins.joinquant.schema import (
JOINQUANT_FILL_COLUMNS,
JOINQUANT_PNL_COLUMNS,
JOINQUANT_POSITION_COLUMNS,
JOINQUANT_TARGET_COLUMNS,
RECONCILE_COLUMNS,
)
from plugins.joinquant.symbols import to_joinquant_symbol
def _date_text(value: object) -> str:
if pd.isna(value):
return ""
return pd.Timestamp(value).strftime("%Y-%m-%d")
def _read_parquet(path: str | Path | None) -> pd.DataFrame:
if path is None:
return pd.DataFrame()
return pd.read_parquet(path)
def _normalize_common(df: pd.DataFrame, portfolio_name: str) -> pd.DataFrame:
out = df.copy()
if "date" in out.columns:
out["date"] = out["date"].map(_date_text)
else:
out["date"] = ""
if "portfolio_name" not in out.columns:
out["portfolio_name"] = portfolio_name
out["portfolio_name"] = out["portfolio_name"].fillna(portfolio_name).astype(str)
if "symbol_id" in out.columns:
out["symbol_id"] = out["symbol_id"].fillna("").astype(str)
if "jq_symbol" not in out.columns and "symbol_id" in out.columns:
out["jq_symbol"] = out["symbol_id"].map(
lambda s: to_joinquant_symbol(s) if s else ""
)
elif "jq_symbol" in out.columns:
out["jq_symbol"] = out["jq_symbol"].fillna("").astype(str)
return out
def _numeric(df: pd.DataFrame, column: str, default: float = 0.0) -> pd.Series:
if column not in df.columns:
return pd.Series([default] * len(df), index=df.index, dtype=float)
return pd.to_numeric(df[column], errors="coerce").replace([np.inf, -np.inf], np.nan)
def _weighted_price(group: pd.DataFrame, price_col: str, shares_col: str) -> float:
prices = pd.to_numeric(group[price_col], errors="coerce")
shares = pd.to_numeric(group[shares_col], errors="coerce").abs()
valid = prices.notna() & shares.notna() & (shares > 0)
if not valid.any():
return np.nan
return float(np.average(prices[valid], weights=shares[valid]))
def _load_targets(targets_dir: str | Path, portfolio_name: str) -> pd.DataFrame:
root = Path(targets_dir)
if not root.exists():
return pd.DataFrame(columns=JOINQUANT_TARGET_COLUMNS)
files_by_stem: dict[str, Path] = {}
for path in sorted(root.glob("*.csv")):
files_by_stem[path.stem] = path
for path in sorted(root.glob("*.parquet")):
files_by_stem[path.stem] = path
frames: list[pd.DataFrame] = []
for path in files_by_stem.values():
if path.suffix == ".parquet":
frame = pd.read_parquet(path)
else:
frame = pd.read_csv(path)
frames.append(frame)
if not frames:
return pd.DataFrame(columns=JOINQUANT_TARGET_COLUMNS)
targets = pd.concat(frames, ignore_index=True)
targets = _normalize_common(targets, portfolio_name)
targets = targets[targets["portfolio_name"].astype(str) == portfolio_name]
if "target_shares" not in targets.columns:
targets["target_shares"] = 0
return targets.reindex(columns=JOINQUANT_TARGET_COLUMNS)
def _aggregate_targets(targets: pd.DataFrame, portfolio_name: str) -> pd.DataFrame:
if targets.empty:
return pd.DataFrame(columns=["date", "portfolio_name", "symbol_id", "jq_symbol", "target_shares"])
targets = _normalize_common(targets, portfolio_name)
targets["target_shares"] = _numeric(targets, "target_shares", 0.0)
grouped = (
targets.groupby(["date", "portfolio_name", "symbol_id"], as_index=False)
.agg(jq_symbol=("jq_symbol", "last"), target_shares=("target_shares", "last"))
)
return grouped
def _aggregate_our_fills(fills: pd.DataFrame, portfolio_name: str) -> pd.DataFrame:
if fills.empty:
return pd.DataFrame(columns=[
"date", "portfolio_name", "symbol_id", "our_filled_shares",
"our_position_shares", "our_cost", "our_trade_price", "our_blocked",
"our_target_shares",
])
fills = _normalize_common(fills, portfolio_name)
fills["traded_shares"] = _numeric(fills, "traded_shares", 0.0)
fills["realized_shares"] = _numeric(fills, "realized_shares", np.nan)
fills["trade_cost"] = _numeric(fills, "trade_cost", 0.0).fillna(0.0)
fills["target_shares"] = _numeric(fills, "target_shares", np.nan)
fills["blocked"] = _numeric(fills, "blocked", 0.0).fillna(0.0)
price_col = next(
(col for col in ["trade_price", "fill_price", "execution_price", "price"] if col in fills.columns),
None,
)
rows: list[dict[str, object]] = []
for key, group in fills.groupby(["date", "portfolio_name", "symbol_id"], sort=False):
row = {
"date": key[0],
"portfolio_name": key[1],
"symbol_id": key[2],
"our_filled_shares": float(group["traded_shares"].sum()),
"our_position_shares": float(group["realized_shares"].dropna().iloc[-1])
if group["realized_shares"].notna().any() else np.nan,
"our_cost": float(group["trade_cost"].sum()),
"our_trade_price": _weighted_price(group, price_col, "traded_shares")
if price_col else np.nan,
"our_blocked": int(group["blocked"].max()),
"our_target_shares": float(group["target_shares"].dropna().iloc[-1])
if group["target_shares"].notna().any() else np.nan,
}
rows.append(row)
return pd.DataFrame(rows)
def _aggregate_our_positions(positions: pd.DataFrame, portfolio_name: str) -> pd.DataFrame:
if positions.empty:
return pd.DataFrame(columns=[
"date", "portfolio_name", "symbol_id", "jq_symbol",
"our_position_fallback", "our_position_price",
])
positions = _normalize_common(positions, portfolio_name)
positions["position_shares"] = _numeric(positions, "position_shares", np.nan)
positions["price"] = _numeric(positions, "price", np.nan)
return (
positions.groupby(["date", "portfolio_name", "symbol_id"], as_index=False)
.agg(
jq_symbol=("jq_symbol", "last"),
our_position_fallback=("position_shares", "last"),
our_position_price=("price", "last"),
)
)
def _aggregate_jq_fills(fills: pd.DataFrame, portfolio_name: str) -> pd.DataFrame:
if fills.empty:
return pd.DataFrame(columns=[
"date", "portfolio_name", "symbol_id", "jq_filled_shares",
"jq_trade_price", "jq_cost", "jq_blocked", "jq_requested_shares",
"raw_status",
])
fills = _normalize_common(fills, portfolio_name)
for col in JOINQUANT_FILL_COLUMNS:
if col not in fills.columns:
fills[col] = np.nan
fills["filled_shares"] = _numeric(fills, "filled_shares", 0.0).fillna(0.0)
fills["requested_shares"] = _numeric(fills, "requested_shares", np.nan)
fills["fill_price"] = _numeric(fills, "fill_price", np.nan)
fills["trade_cost"] = _numeric(fills, "trade_cost", 0.0).fillna(0.0)
fills["blocked"] = _numeric(fills, "blocked", 0.0).fillna(0.0)
fills["raw_status"] = fills["raw_status"].fillna("").astype(str)
rows: list[dict[str, object]] = []
for key, group in fills.groupby(["date", "portfolio_name", "symbol_id"], sort=False):
row = {
"date": key[0],
"portfolio_name": key[1],
"symbol_id": key[2],
"jq_filled_shares": float(group["filled_shares"].sum()),
"jq_trade_price": _weighted_price(group, "fill_price", "filled_shares"),
"jq_cost": float(group["trade_cost"].sum()),
"jq_blocked": int(group["blocked"].max()),
"jq_requested_shares": float(group["requested_shares"].dropna().iloc[-1])
if group["requested_shares"].notna().any() else np.nan,
"raw_status": ";".join([s for s in group["raw_status"].astype(str) if s]),
}
rows.append(row)
return pd.DataFrame(rows)
def _aggregate_jq_positions(positions: pd.DataFrame, portfolio_name: str) -> pd.DataFrame:
if positions.empty:
return pd.DataFrame(columns=[
"date", "portfolio_name", "symbol_id", "jq_symbol", "jq_position_shares",
])
positions = _normalize_common(positions, portfolio_name)
for col in JOINQUANT_POSITION_COLUMNS:
if col not in positions.columns:
positions[col] = np.nan
positions["position_shares"] = _numeric(positions, "position_shares", np.nan)
return (
positions.groupby(["date", "portfolio_name", "symbol_id"], as_index=False)
.agg(jq_symbol=("jq_symbol", "last"), jq_position_shares=("position_shares", "last"))
)
def _portfolio_frame(df: pd.DataFrame, portfolio_name: str, prefix: str) -> pd.DataFrame:
if df.empty:
return pd.DataFrame(columns=["date", "portfolio_name"])
out = _normalize_common(df, portfolio_name)
for col in JOINQUANT_PNL_COLUMNS:
if col not in out.columns:
out[col] = np.nan
keep = ["date", "portfolio_name", "gross_exposure", "net_exposure", "cash", "total_value", "pnl", "cost", "turnover"]
out = out[keep].copy()
for col in keep[2:]:
out[col] = pd.to_numeric(out[col], errors="coerce")
out = out.groupby(["date", "portfolio_name"], as_index=False).last()
return out.rename(columns={col: f"{prefix}_{col}" for col in keep[2:]})
def _infer_booksize(targets: pd.DataFrame, our_pnl: pd.DataFrame, jq_pnl: pd.DataFrame) -> float:
candidates: list[float] = []
if "target_value" in targets.columns:
gross = (
pd.to_numeric(targets["target_value"], errors="coerce")
.abs()
.replace([np.inf, -np.inf], np.nan)
.dropna()
.sum()
)
if gross > 0:
candidates.append(float(gross))
for df in (our_pnl, jq_pnl):
if "gross_exposure" in df.columns:
val = pd.to_numeric(df["gross_exposure"], errors="coerce").max()
if pd.notna(val) and val > 0:
candidates.append(float(val))
return max(candidates) if candidates else 1.0
def _status_reason(raw_status: object) -> str | None:
text = str(raw_status or "").lower()
if "suspend" in text or "halt" in text:
return "SUSPENSION"
if "limit_up" in text or "limit up" in text or "up_limit" in text:
return "LIMIT_UP_BLOCK"
if "limit_down" in text or "limit down" in text or "down_limit" in text:
return "LIMIT_DOWN_BLOCK"
if "volume" in text or "liquid" in text:
return "VOLUME_OR_LIQUIDITY"
if "cash" in text or "margin" in text:
return "CASH_CONSTRAINT"
return None
def _classify_symbol_row(
row: pd.Series,
*,
share_tol: float,
price_rel_tol: float,
value_tol: float,
pnl_tol: float,
) -> str:
target = row.get("target_shares", np.nan)
filled_diff = abs(row.get("filled_share_diff", 0.0))
position_diff = abs(row.get("position_share_diff", 0.0))
our_present = bool(row.get("_our_present", False))
jq_present = bool(row.get("_jq_present", False))
if pd.notna(target) and target < 0 and (filled_diff > share_tol or position_diff > share_tol or not jq_present):
return "SHORT_NOT_SUPPORTED"
if not jq_present and (our_present or pd.notna(target)):
return "MISSING_IN_JOINQUANT"
if not our_present and jq_present:
return "MISSING_IN_OUR_SYSTEM"
status_reason = _status_reason(row.get("raw_status", ""))
if (filled_diff > share_tol or position_diff > share_tol) and status_reason:
return status_reason
if filled_diff > share_tol or position_diff > share_tol:
return "UNKNOWN"
our_price = row.get("our_trade_price", np.nan)
jq_price = row.get("jq_trade_price", np.nan)
if pd.notna(our_price) and pd.notna(jq_price):
denom = max(abs(float(our_price)), abs(float(jq_price)), 1.0)
if abs(float(our_price) - float(jq_price)) > price_rel_tol * denom:
return "PRICE_MISMATCH"
if abs(row.get("cost_diff", 0.0)) > value_tol:
return "COST_MODEL"
if abs(row.get("pnl_diff", 0.0)) > pnl_tol:
return "UNKNOWN"
return "MATCH"
def _classify_portfolio_row(row: pd.Series, value_tol: float, pnl_tol: float) -> str:
our_present = bool(row.get("_our_present", False))
jq_present = bool(row.get("_jq_present", False))
if not jq_present and our_present:
return "MISSING_IN_JOINQUANT"
if not our_present and jq_present:
return "MISSING_IN_OUR_SYSTEM"
if abs(row.get("cost_diff", 0.0)) > value_tol:
return "COST_MODEL"
if abs(row.get("pnl_diff", 0.0)) > pnl_tol:
return "UNKNOWN"
return "MATCH"
def _build_symbol_reconcile(
*,
portfolio_name: str,
targets: pd.DataFrame,
our_fills: pd.DataFrame,
our_positions: pd.DataFrame,
our_pnl: pd.DataFrame,
jq_fills: pd.DataFrame,
jq_positions: pd.DataFrame,
jq_pnl: pd.DataFrame,
share_tol: float,
price_rel_tol: float,
value_tol: float,
pnl_tol: float,
) -> pd.DataFrame:
target_agg = _aggregate_targets(targets, portfolio_name)
our_fill_agg = _aggregate_our_fills(our_fills, portfolio_name)
our_pos_agg = _aggregate_our_positions(our_positions, portfolio_name)
jq_fill_agg = _aggregate_jq_fills(jq_fills, portfolio_name)
jq_pos_agg = _aggregate_jq_positions(jq_positions, portfolio_name)
key_cols = ["date", "portfolio_name", "symbol_id"]
keys = []
for frame in [target_agg, our_fill_agg, our_pos_agg, jq_fill_agg, jq_pos_agg]:
if not frame.empty:
keys.append(frame[key_cols])
if not keys:
return pd.DataFrame(columns=RECONCILE_COLUMNS)
base = pd.concat(keys, ignore_index=True).drop_duplicates()
result = base.merge(target_agg, on=key_cols, how="left")
result = result.merge(our_fill_agg, on=key_cols, how="left")
result = result.merge(our_pos_agg, on=key_cols, how="left", suffixes=("", "_ourpos"))
result = result.merge(jq_fill_agg, on=key_cols, how="left")
result = result.merge(jq_pos_agg, on=key_cols, how="left", suffixes=("", "_jqpos"))
jq_symbol_cols = [col for col in result.columns if col.startswith("jq_symbol")]
jq_symbol_values = result[jq_symbol_cols].copy() if jq_symbol_cols else pd.DataFrame(index=result.index)
result["jq_symbol"] = ""
for col in jq_symbol_values.columns:
values = jq_symbol_values[col].fillna("").astype(str)
result["jq_symbol"] = result["jq_symbol"].mask(
result["jq_symbol"].eq("") & values.ne(""),
values,
)
result["jq_symbol"] = result.apply(
lambda row: row["jq_symbol"] or to_joinquant_symbol(row["symbol_id"]),
axis=1,
)
result["_our_present"] = (
result[["our_filled_shares", "our_position_shares", "our_position_fallback"]]
.notna()
.any(axis=1)
)
result["_jq_present"] = (
result[["jq_filled_shares", "jq_position_shares"]].notna().any(axis=1)
)
target_shares = pd.to_numeric(result["target_shares"], errors="coerce")
our_target = pd.to_numeric(result["our_target_shares"], errors="coerce")
jq_target = pd.to_numeric(result["jq_requested_shares"], errors="coerce")
result["target_shares"] = target_shares.where(target_shares.notna(), our_target)
result["target_shares"] = result["target_shares"].where(
result["target_shares"].notna(),
jq_target,
)
our_position = pd.to_numeric(result["our_position_shares"], errors="coerce")
our_position_fallback = pd.to_numeric(result["our_position_fallback"], errors="coerce")
result["our_position_shares"] = our_position.where(
our_position.notna(),
our_position_fallback,
)
for col in [
"target_shares", "our_filled_shares", "jq_filled_shares",
"our_position_shares", "jq_position_shares", "our_cost", "jq_cost",
]:
result[col] = pd.to_numeric(result[col], errors="coerce").fillna(0.0)
result["filled_share_diff"] = result["our_filled_shares"] - result["jq_filled_shares"]
result["position_share_diff"] = result["our_position_shares"] - result["jq_position_shares"]
result["trade_price_diff"] = np.where(
result["our_trade_price"].notna() & result["jq_trade_price"].notna(),
result["our_trade_price"] - result["jq_trade_price"],
np.nan,
)
result["cost_diff"] = result["our_cost"] - result["jq_cost"]
our_daily = _portfolio_frame(our_pnl, portfolio_name, "our")
jq_daily = _portfolio_frame(jq_pnl, portfolio_name, "jq")
pnl_daily = our_daily.merge(jq_daily, on=["date", "portfolio_name"], how="outer")
if not pnl_daily.empty:
pnl_daily["our_pnl"] = pd.to_numeric(pnl_daily.get("our_pnl"), errors="coerce").fillna(0.0)
pnl_daily["jq_pnl"] = pd.to_numeric(pnl_daily.get("jq_pnl"), errors="coerce").fillna(0.0)
pnl_daily["pnl_diff"] = pnl_daily["our_pnl"] - pnl_daily["jq_pnl"]
result = result.merge(
pnl_daily[["date", "portfolio_name", "our_pnl", "jq_pnl", "pnl_diff"]],
on=["date", "portfolio_name"],
how="left",
)
else:
result["our_pnl"] = 0.0
result["jq_pnl"] = 0.0
result["pnl_diff"] = 0.0
for col in ["our_pnl", "jq_pnl", "pnl_diff"]:
result[col] = pd.to_numeric(result[col], errors="coerce").fillna(0.0)
result["raw_status"] = result.get("raw_status", "").fillna("")
result["diff_reason"] = result.apply(
_classify_symbol_row,
axis=1,
share_tol=share_tol,
price_rel_tol=price_rel_tol,
value_tol=value_tol,
pnl_tol=pnl_tol,
)
return result[RECONCILE_COLUMNS].sort_values(
["date", "portfolio_name", "symbol_id"]
).reset_index(drop=True)
def _build_portfolio_summary(
*,
portfolio_name: str,
our_pnl: pd.DataFrame,
jq_pnl: pd.DataFrame,
value_tol: float,
pnl_tol: float,
) -> pd.DataFrame:
our = _portfolio_frame(our_pnl, portfolio_name, "our")
jq = _portfolio_frame(jq_pnl, portfolio_name, "jq")
if our.empty and jq.empty:
return pd.DataFrame(columns=[
"date", "portfolio_name", "diff_reason",
"our_pnl", "jq_pnl", "pnl_diff",
])
summary = our.merge(jq, on=["date", "portfolio_name"], how="outer")
summary["_our_present"] = summary.filter(regex=r"^our_").notna().any(axis=1)
summary["_jq_present"] = summary.filter(regex=r"^jq_").notna().any(axis=1)
metrics = ["gross_exposure", "net_exposure", "cash", "total_value", "pnl", "cost", "turnover"]
for metric in metrics:
our_col = f"our_{metric}"
jq_col = f"jq_{metric}"
if our_col not in summary.columns:
summary[our_col] = np.nan
if jq_col not in summary.columns:
summary[jq_col] = np.nan
summary[f"{metric}_diff"] = (
pd.to_numeric(summary[our_col], errors="coerce").fillna(0.0)
- pd.to_numeric(summary[jq_col], errors="coerce").fillna(0.0)
)
summary["our_cumulative_pnl"] = (
pd.to_numeric(summary["our_pnl"], errors="coerce").fillna(0.0).cumsum()
)
summary["jq_cumulative_pnl"] = (
pd.to_numeric(summary["jq_pnl"], errors="coerce").fillna(0.0).cumsum()
)
summary["cumulative_pnl_diff"] = summary["our_cumulative_pnl"] - summary["jq_cumulative_pnl"]
summary["diff_reason"] = summary.apply(
_classify_portfolio_row,
axis=1,
value_tol=value_tol,
pnl_tol=pnl_tol,
)
summary = summary.drop(columns=["_our_present", "_jq_present"])
return summary.sort_values(["date", "portfolio_name"]).reset_index(drop=True)
def _write_summary_md(
path: Path,
*,
portfolio_name: str,
symbol_report: pd.DataFrame,
portfolio_summary: pd.DataFrame,
) -> None:
symbol_counts = (
symbol_report["diff_reason"].value_counts().sort_index()
if not symbol_report.empty else pd.Series(dtype=int)
)
portfolio_counts = (
portfolio_summary["diff_reason"].value_counts().sort_index()
if not portfolio_summary.empty else pd.Series(dtype=int)
)
lines = [
"# JoinQuant Reconciliation Summary",
"",
f"Portfolio: `{portfolio_name}`",
"",
"## Per-symbol Difference Counts",
"",
]
if symbol_counts.empty:
lines.append("No per-symbol rows were produced.")
else:
for reason, count in symbol_counts.items():
lines.append(f"- {reason}: {int(count)}")
lines.extend(["", "## Daily Portfolio Difference Counts", ""])
if portfolio_counts.empty:
lines.append("No daily portfolio rows were produced.")
else:
for reason, count in portfolio_counts.items():
lines.append(f"- {reason}: {int(count)}")
if not portfolio_summary.empty:
lines.extend(["", "## Daily Portfolio Preview", ""])
preview_cols = [
"date", "diff_reason", "our_pnl", "jq_pnl", "pnl_diff",
"our_cost", "jq_cost", "cost_diff",
]
preview_cols = [col for col in preview_cols if col in portfolio_summary.columns]
lines.append(",".join(preview_cols))
for row in portfolio_summary[preview_cols].head(20).itertuples(index=False):
lines.append(",".join(str(value) for value in row))
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
def reconcile_joinquant(
*,
portfolio_name: str,
targets_dir: str | Path,
our_fills_path: str | Path,
our_positions_path: str | Path,
our_pnl_path: str | Path,
jq_fills_path: str | Path,
jq_positions_path: str | Path,
jq_pnl_path: str | Path,
out_dir: str | Path = "plugins_output/joinquant/reconcile",
share_tolerance: float = 0.0,
price_rel_tolerance: float = 1e-4,
pnl_tolerance: float = 1.0,
booksize: float | None = None,
) -> dict[str, Path]:
"""Reconcile JoinQuant output against internal simulator output."""
targets = _load_targets(targets_dir, portfolio_name)
our_fills = _read_parquet(our_fills_path)
our_positions = _read_parquet(our_positions_path)
our_pnl = _read_parquet(our_pnl_path)
jq_fills = _read_parquet(jq_fills_path)
jq_positions = _read_parquet(jq_positions_path)
jq_pnl = _read_parquet(jq_pnl_path)
inferred_booksize = booksize or _infer_booksize(targets, our_pnl, jq_pnl)
value_tol = max(1.0, 1e-6 * float(inferred_booksize))
symbol_report = _build_symbol_reconcile(
portfolio_name=portfolio_name,
targets=targets,
our_fills=our_fills,
our_positions=our_positions,
our_pnl=our_pnl,
jq_fills=jq_fills,
jq_positions=jq_positions,
jq_pnl=jq_pnl,
share_tol=share_tolerance,
price_rel_tol=price_rel_tolerance,
value_tol=value_tol,
pnl_tol=pnl_tolerance,
)
portfolio_summary = _build_portfolio_summary(
portfolio_name=portfolio_name,
our_pnl=our_pnl,
jq_pnl=jq_pnl,
value_tol=value_tol,
pnl_tol=pnl_tolerance,
)
root = Path(out_dir) / portfolio_name
root.mkdir(parents=True, exist_ok=True)
paths = {
"daily_reconcile": root / "daily_reconcile.pq",
"summary_csv": root / "summary.csv",
"summary_md": root / "summary.md",
}
symbol_report.to_parquet(paths["daily_reconcile"], index=False)
portfolio_summary.to_csv(paths["summary_csv"], index=False)
_write_summary_md(
paths["summary_md"],
portfolio_name=portfolio_name,
symbol_report=symbol_report,
portfolio_summary=portfolio_summary,
)
return paths
+99
View File
@@ -0,0 +1,99 @@
"""Column contracts for the JoinQuant comparison plugin."""
from typing import Final
JOINQUANT_TARGET_COLUMNS: Final[list[str]] = [
"date",
"portfolio_name",
"symbol_id",
"jq_symbol",
"target_shares",
"target_value",
"target_weight",
"export_mode",
"snapshot_id",
]
JOINQUANT_FILL_COLUMNS: Final[list[str]] = [
"date",
"portfolio_name",
"symbol_id",
"jq_symbol",
"order_id",
"side",
"requested_shares",
"filled_shares",
"fill_price",
"trade_value",
"trade_cost",
"blocked",
"raw_status",
]
JOINQUANT_POSITION_COLUMNS: Final[list[str]] = [
"date",
"portfolio_name",
"symbol_id",
"jq_symbol",
"position_shares",
"position_value",
"cash",
"total_value",
]
JOINQUANT_PNL_COLUMNS: Final[list[str]] = [
"date",
"portfolio_name",
"gross_exposure",
"net_exposure",
"cash",
"total_value",
"pnl",
"cost",
"turnover",
]
RECONCILE_COLUMNS: Final[list[str]] = [
"date",
"portfolio_name",
"symbol_id",
"jq_symbol",
"target_shares",
"our_filled_shares",
"jq_filled_shares",
"filled_share_diff",
"our_position_shares",
"jq_position_shares",
"position_share_diff",
"our_trade_price",
"jq_trade_price",
"trade_price_diff",
"our_cost",
"jq_cost",
"cost_diff",
"our_pnl",
"jq_pnl",
"pnl_diff",
"diff_reason",
]
DIFF_REASONS: Final[list[str]] = [
"MATCH",
"SYMBOL_MAPPING",
"PRICE_MISMATCH",
"LOT_ROUNDING",
"SUSPENSION",
"LIMIT_UP_BLOCK",
"LIMIT_DOWN_BLOCK",
"VOLUME_OR_LIQUIDITY",
"COST_MODEL",
"CASH_CONSTRAINT",
"SHORT_NOT_SUPPORTED",
"CORPORATE_ACTION",
"JOINQUANT_INTERNAL_ROUNDING",
"MISSING_IN_OUR_SYSTEM",
"MISSING_IN_JOINQUANT",
"UNKNOWN",
]
+191
View File
@@ -0,0 +1,191 @@
"""End-to-end local smoke preparation for JoinQuant comparison."""
from __future__ import annotations
import json
from datetime import datetime, timezone
from pathlib import Path
import pandas as pd
from pipeline.common.schema import POSITION_COLUMNS
from pipeline.data.downloader import download_universe
from pipeline.portfolio.constraints import get_constraint
from pipeline.portfolio.simulator import ReferenceSimulator
from plugins.joinquant.export_targets import export_targets
from plugins.joinquant.wrapper_strategy import write_wrapper_strategy
def build_fixed_share_positions(
data: pd.DataFrame,
*,
trade_symbol: str,
portfolio_name: str,
shares: int,
booksize: float,
max_signal_dates: int | None = None,
) -> pd.DataFrame:
"""Create a deterministic long-only fixed-share position book.
The final available data date is excluded because the internal simulator
executes each signal date at the next available open.
"""
data = data.copy()
data["date"] = pd.to_datetime(data["date"]).dt.normalize()
symbol_data = (
data[data["symbol_id"].astype(str) == trade_symbol]
.sort_values("date")
.reset_index(drop=True)
)
if len(symbol_data) < 2:
raise ValueError(f"Need at least two daily bars for {trade_symbol}")
signal_data = symbol_data.iloc[:-1].copy()
if max_signal_dates is not None and max_signal_dates > 0:
signal_data = signal_data.tail(max_signal_dates)
if signal_data.empty:
raise ValueError("No signal dates available after excluding final data date")
rows: list[dict[str, object]] = []
for row in signal_data.itertuples(index=False):
price = float(row.close)
target_value = float(shares * price)
rows.append({
"symbol_id": trade_symbol,
"date": pd.Timestamp(row.date),
"portfolio_name": portfolio_name,
"target_weight": target_value / float(booksize),
"target_value": target_value,
"target_shares": float(shares),
"position_shares": int(shares),
"position_value": target_value,
"price": price,
})
return pd.DataFrame(rows, columns=POSITION_COLUMNS)
def prepare_smoke_test(
*,
out_dir: str | Path,
universe: str = "sh600000,sz000001,sh600519,sz002594,sz300750",
trade_symbol: str = "sh600000",
start_date: str = "2024-01-02",
end_date: str = "2024-01-12",
portfolio_name: str = "jq_smoke_one_stock_long",
shares: int = 1000,
booksize: float = 1_000_000.0,
max_signal_dates: int = 3,
cost_bps: float = 5.0,
slippage_bps: float = 5.0,
volume_frac: float = 0.02,
force: bool = False,
) -> dict[str, object]:
"""Run the local side of a tiny real-data JoinQuant smoke test."""
root = Path(out_dir)
root.mkdir(parents=True, exist_ok=True)
stats = download_universe(
universe=universe,
start_date=start_date,
end_date=end_date,
output_dir=root / "daily_bars",
max_symbols=0,
chunk_size=100,
adjust="qfq",
)
data_path = Path(stats["dataset_path"])
data = pd.read_parquet(data_path)
positions = build_fixed_share_positions(
data,
trade_symbol=trade_symbol,
portfolio_name=portfolio_name,
shares=shares,
booksize=booksize,
max_signal_dates=max_signal_dates,
)
portfolio_dir = root / "portfolio"
portfolio_dir.mkdir(parents=True, exist_ok=True)
positions_path = portfolio_dir / f"{portfolio_name}.pq"
positions.to_parquet(positions_path, index=False)
constraints = [
get_constraint("suspension"),
get_constraint("price_limit"),
get_constraint("volume_cap", max_frac=volume_frac),
]
sim = ReferenceSimulator(
constraints=constraints,
cost_bps=cost_bps,
slippage_bps=slippage_bps,
)
fills, pnl = sim.run(positions, data)
execution_dir = root / "execution"
fills_dir = execution_dir / "fills"
pnl_dir = execution_dir / "pnl"
fills_dir.mkdir(parents=True, exist_ok=True)
pnl_dir.mkdir(parents=True, exist_ok=True)
fills_path = fills_dir / f"{portfolio_name}.pq"
pnl_path = pnl_dir / f"{portfolio_name}.pq"
fills.to_parquet(fills_path, index=False)
pnl.to_parquet(pnl_path, index=False)
target_root = root / "plugins_output" / "joinquant" / "targets_aligned"
snapshots = export_targets(
positions_path=positions_path,
portfolio_name=portfolio_name,
mode="target_shares",
out_dir=target_root,
execution_calendar_path=data_path,
force=force,
)
wrapper_path = root / "plugins_output" / "joinquant" / f"wrapper_strategy_{portfolio_name}.py"
write_wrapper_strategy(
portfolio_name=portfolio_name,
mode="target_shares",
out_path=wrapper_path,
)
export_dir = root / "joinquant_exports"
export_dir.mkdir(parents=True, exist_ok=True)
manifest = {
"created_at": datetime.now(timezone.utc).isoformat(),
"portfolio_name": portfolio_name,
"universe": universe,
"trade_symbol": trade_symbol,
"start_date": start_date,
"end_date": end_date,
"shares": shares,
"booksize": booksize,
"data_path": str(data_path),
"positions_path": str(positions_path),
"fills_path": str(fills_path),
"pnl_path": str(pnl_path),
"targets_dir": str(target_root / portfolio_name),
"wrapper_path": str(wrapper_path),
"joinquant_export_dir": str(export_dir),
"expected_joinquant_csvs": {
"fills": str(export_dir / "jq_fills.csv"),
"positions": str(export_dir / "jq_positions.csv"),
"pnl": str(export_dir / "jq_pnl.csv"),
},
"target_snapshots": snapshots,
"local_summary": {
"n_data_rows": int(len(data)),
"n_position_rows": int(len(positions)),
"n_fill_rows": int(len(fills)),
"n_pnl_rows": int(len(pnl)),
"total_pnl": float(pnl["pnl"].sum()) if len(pnl) else 0.0,
"total_cost": float(pnl["cost"].sum()) if len(pnl) else 0.0,
"blocked_trades": int(fills["blocked"].sum()) if len(fills) else 0,
},
}
manifest_path = root / "joinquant_smoke_manifest.json"
manifest["manifest_path"] = str(manifest_path)
manifest_path.write_text(
json.dumps(manifest, indent=2, ensure_ascii=False) + "\n",
encoding="utf-8",
)
return manifest
+94
View File
@@ -0,0 +1,94 @@
"""Symbol conversion between internal A-share ids and JoinQuant ids."""
from __future__ import annotations
import re
_INTERNAL_RE = re.compile(r"^(?P<prefix>sh|sz)(?P<code>\d{6})$", re.IGNORECASE)
_JOINQUANT_RE = re.compile(
r"^(?P<code>\d{6})\.(?P<exchange>XSHG|XSHE)$", re.IGNORECASE
)
_BARE_RE = re.compile(r"^\d{6}$")
def _validate_internal(prefix: str, code: str) -> None:
if prefix == "sh" and code.startswith("6"):
return
if prefix == "sz" and code.startswith(("0", "3")):
return
raise ValueError(f"Unsupported A-share symbol: {prefix}{code}")
def _validate_joinquant(code: str, exchange: str) -> None:
if exchange == "XSHG" and code.startswith("6"):
return
if exchange == "XSHE" and code.startswith(("0", "3")):
return
raise ValueError(f"Unsupported JoinQuant A-share symbol: {code}.{exchange}")
def to_joinquant_symbol(symbol_id: str) -> str:
"""Convert an internal symbol like ``sh600000`` to ``600000.XSHG``.
Args:
symbol_id: Internal A-share id with ``sh`` or ``sz`` prefix.
Returns:
JoinQuant security id.
Raises:
ValueError: If the symbol is malformed or outside supported A-share
Shanghai/Shenzhen equity prefixes.
"""
text = str(symbol_id).strip().lower()
match = _INTERNAL_RE.match(text)
if not match:
raise ValueError(f"Invalid internal symbol: {symbol_id!r}")
prefix = match.group("prefix").lower()
code = match.group("code")
_validate_internal(prefix, code)
exchange = "XSHG" if prefix == "sh" else "XSHE"
return f"{code}.{exchange}"
def from_joinquant_symbol(jq_symbol: str) -> str:
"""Convert a JoinQuant symbol like ``600000.XSHG`` to ``sh600000``."""
text = str(jq_symbol).strip().upper()
match = _JOINQUANT_RE.match(text)
if not match:
raise ValueError(f"Invalid JoinQuant symbol: {jq_symbol!r}")
code = match.group("code")
exchange = match.group("exchange").upper()
_validate_joinquant(code, exchange)
prefix = "sh" if exchange == "XSHG" else "sz"
return f"{prefix}{code}"
def normalize_symbol_pair(value: object) -> tuple[str, str]:
"""Return ``(symbol_id, jq_symbol)`` from any supported symbol spelling."""
text = str(value).strip()
if not text or text.lower() == "nan":
raise ValueError("Missing symbol")
if _INTERNAL_RE.match(text):
symbol_id = text.lower()
return symbol_id, to_joinquant_symbol(symbol_id)
if _JOINQUANT_RE.match(text):
jq_symbol = text.upper()
return from_joinquant_symbol(jq_symbol), jq_symbol
if _BARE_RE.match(text):
if text.startswith("6"):
symbol_id = f"sh{text}"
elif text.startswith(("0", "3")):
symbol_id = f"sz{text}"
else:
raise ValueError(f"Unsupported bare A-share code: {text}")
return symbol_id, to_joinquant_symbol(symbol_id)
raise ValueError(f"Unsupported symbol: {value!r}")
+271
View File
@@ -0,0 +1,271 @@
"""Generate a standalone JoinQuant wrapper strategy.
The module also defines default JoinQuant strategy hooks by executing the same
template with ``run1`` / ``target_shares`` defaults. That means this file can be
copied directly into JoinQuant for a quick smoke test, while the CLI can still
write a configured standalone file for a real run.
"""
from __future__ import annotations
import json
from pathlib import Path
from string import Template
from typing import Literal
WrapperMode = Literal["target_shares", "target_value"]
_WRAPPER_TEMPLATE = Template(
r'''# Standalone JoinQuant target wrapper generated by chinese-equity-quant.
# Copy this file and the exported daily CSV target files into JoinQuant.
#
# The file loader is isolated in _read_target_file(). The default implementation
# uses JoinQuant's read_file API for uploaded files. If your JoinQuant runtime
# allows HTTP or another storage backend, replace only that function.
import csv
import io
import json
PORTFOLIO_NAME = "${portfolio_name}"
TARGET_MODE = "${mode}"
ALLOW_SHORT = ${allow_short}
TARGET_FILE_PREFIX = "" # Optional uploaded-file prefix, for example "run1/"
_EMBEDDED_TARGETS = ${embedded_targets}
def initialize(context):
set_benchmark("000300.XSHG")
set_option("use_real_price", True)
g.portfolio_name = PORTFOLIO_NAME
g.target_mode = TARGET_MODE
g.targets_by_date = {}
run_daily(load_targets, time="before_open")
run_daily(rebalance_at_open, time="open")
run_daily(record_after_close, time="after_close")
def _today_text(context):
return context.current_dt.strftime("%Y-%m-%d")
def _today_file_name(context):
return context.current_dt.strftime("%Y%m%d") + ".csv"
def _read_target_file(file_name):
${embedded_target_read}
data = read_file(TARGET_FILE_PREFIX + file_name)
if isinstance(data, bytes):
data = data.decode("utf-8")
return data
def _load_target_rows(context):
file_name = _today_file_name(context)
text = _read_target_file(file_name)
rows = list(csv.DictReader(io.StringIO(text)))
clean_rows = []
for row in rows:
if row.get("portfolio_name") and row["portfolio_name"] != PORTFOLIO_NAME:
continue
jq_symbol = row.get("jq_symbol") or row.get("security") or row.get("symbol")
if not jq_symbol:
log.warn("Skipping target row with no jq_symbol: %s" % row)
continue
if TARGET_MODE == "target_shares":
target = int(float(row.get("target_shares") or 0))
elif TARGET_MODE == "target_value":
target = float(row.get("target_value") or 0.0)
else:
raise ValueError("Unsupported TARGET_MODE: %s" % TARGET_MODE)
if not ALLOW_SHORT and target < 0:
log.warn(
"SHORT_NOT_SUPPORTED clipping %s target from %s to 0" %
(jq_symbol, target)
)
target = 0
clean_rows.append({"jq_symbol": jq_symbol, "target": target, "raw": row})
return clean_rows
def load_targets(context):
date_text = _today_text(context)
try:
rows = _load_target_rows(context)
except Exception as exc:
log.error("Failed to load JoinQuant target file for %s: %s" % (date_text, exc))
rows = []
g.targets_by_date[date_text] = rows
log.info("JOINQUANT_TARGET_LOAD|%s" % json.dumps({
"date": date_text,
"portfolio_name": PORTFOLIO_NAME,
"target_mode": TARGET_MODE,
"n_targets": len(rows),
}, sort_keys=True))
def rebalance_at_open(context):
date_text = _today_text(context)
rows = g.targets_by_date.get(date_text, [])
target_symbols = set()
for row in rows:
security = row["jq_symbol"]
target_symbols.add(security)
if TARGET_MODE == "target_shares":
order_target(security, int(row["target"]))
else:
order_target_value(security, float(row["target"]))
log.info("JOINQUANT_ORDER_SUBMIT|%s" % json.dumps({
"date": date_text,
"portfolio_name": PORTFOLIO_NAME,
"jq_symbol": security,
"target_mode": TARGET_MODE,
"target": row["target"],
}, sort_keys=True))
for security in list(context.portfolio.positions.keys()):
if security not in target_symbols:
order_target(security, 0)
log.info("JOINQUANT_ORDER_CLOSE|%s" % json.dumps({
"date": date_text,
"portfolio_name": PORTFOLIO_NAME,
"jq_symbol": security,
}, sort_keys=True))
def _position_records(context):
records = []
cash = float(context.portfolio.available_cash)
total_value = float(context.portfolio.total_value)
for security, position in context.portfolio.positions.items():
records.append({
"date": _today_text(context),
"portfolio_name": PORTFOLIO_NAME,
"jq_symbol": security,
"position_shares": int(position.total_amount),
"position_value": float(position.value),
"cash": cash,
"total_value": total_value,
})
return records
def _trade_records(context):
records = []
try:
trades = get_trades()
except Exception:
trades = {}
for trade_id, trade in trades.items():
amount = int(getattr(trade, "amount", 0))
price = float(getattr(trade, "price", 0.0))
security = getattr(trade, "security", "")
side = "buy" if amount >= 0 else "sell"
records.append({
"date": _today_text(context),
"portfolio_name": PORTFOLIO_NAME,
"jq_symbol": security,
"order_id": str(getattr(trade, "order_id", trade_id)),
"side": side,
"filled_shares": amount,
"fill_price": price,
"trade_value": abs(amount * price),
"trade_cost": float(getattr(trade, "commission", 0.0)),
"raw_status": "filled",
})
return records
def record_after_close(context):
date_text = _today_text(context)
for record in _trade_records(context):
log.info("JOINQUANT_FILL|%s" % json.dumps(record, sort_keys=True))
for record in _position_records(context):
log.info("JOINQUANT_POSITION|%s" % json.dumps(record, sort_keys=True))
log.info("JOINQUANT_PNL|%s" % json.dumps({
"date": date_text,
"portfolio_name": PORTFOLIO_NAME,
"cash": float(context.portfolio.available_cash),
"total_value": float(context.portfolio.total_value),
}, sort_keys=True))
'''
)
# Make this module itself usable as a JoinQuant strategy with defaults.
exec(_WRAPPER_TEMPLATE.substitute(
portfolio_name="run1",
mode="target_shares",
allow_short="False",
embedded_targets="{}",
embedded_target_read="",
))
def render_wrapper_strategy(
*,
portfolio_name: str,
mode: WrapperMode = "target_shares",
allow_short: bool = False,
embedded_targets: dict[str, str] | None = None,
) -> str:
"""Render the standalone JoinQuant wrapper strategy source."""
if mode not in {"target_shares", "target_value"}:
raise ValueError("mode must be 'target_shares' or 'target_value'")
embedded_targets = embedded_targets or {}
embedded_target_read = ""
if embedded_targets:
embedded_target_read = (
" if file_name in _EMBEDDED_TARGETS:\n"
" return _EMBEDDED_TARGETS[file_name]\n"
)
return _WRAPPER_TEMPLATE.substitute(
portfolio_name=portfolio_name,
mode=mode,
allow_short="True" if allow_short else "False",
embedded_targets=json.dumps(embedded_targets, ensure_ascii=False, indent=4),
embedded_target_read=embedded_target_read,
)
def _load_embedded_targets(targets_dir: str | Path | None) -> dict[str, str]:
if targets_dir is None:
return {}
root = Path(targets_dir)
targets = {
path.name: path.read_text(encoding="utf-8")
for path in sorted(root.glob("*.csv"))
}
if not targets:
raise ValueError(f"No CSV target files found under {root}")
return targets
def write_wrapper_strategy(
*,
portfolio_name: str,
mode: WrapperMode = "target_shares",
out_path: str | Path,
allow_short: bool = False,
embedded_targets_dir: str | Path | None = None,
) -> Path:
"""Write a configured standalone JoinQuant wrapper strategy."""
path = Path(out_path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
render_wrapper_strategy(
portfolio_name=portfolio_name,
mode=mode,
allow_short=allow_short,
embedded_targets=_load_embedded_targets(embedded_targets_dir),
),
encoding="utf-8",
)
return path
+3
View File
@@ -16,6 +16,9 @@ dependencies = [
backtrader = [ backtrader = [
"backtrader>=1.9.76.123", "backtrader>=1.9.76.123",
] ]
joinquant-browser = [
"playwright>=1.61.0",
]
[dependency-groups] [dependency-groups]
dev = [ dev = [
+650
View File
@@ -0,0 +1,650 @@
"""Tests for the JoinQuant comparison plugin (network-free)."""
from __future__ import annotations
import hashlib
import json
from pathlib import Path
import pandas as pd
import pytest
from click.testing import CliRunner
from cli import cli
from pipeline.common.schema import FILL_COLUMNS, PNL_COLUMNS, POSITION_COLUMNS
from plugins.joinquant.browser import (
default_browser_config,
load_env_file,
resolve_template,
write_browser_config_template,
)
from plugins.joinquant.export_targets import export_targets
from plugins.joinquant.ingest import (
ingest_joinquant_outputs,
normalize_fills_csv,
)
from plugins.joinquant.reconcile import reconcile_joinquant
from plugins.joinquant.schema import (
JOINQUANT_FILL_COLUMNS,
JOINQUANT_PNL_COLUMNS,
JOINQUANT_POSITION_COLUMNS,
JOINQUANT_TARGET_COLUMNS,
RECONCILE_COLUMNS,
)
from plugins.joinquant.smoke import build_fixed_share_positions
from plugins.joinquant.symbols import from_joinquant_symbol, to_joinquant_symbol
from plugins.joinquant.wrapper_strategy import write_wrapper_strategy
def _positions(
*,
symbol: str = "sh600000",
date: str = "2026-07-01",
shares: int = 1000,
price: float = 10.0,
portfolio_name: str = "run1",
) -> pd.DataFrame:
target_value = float(shares * price)
weight = target_value / 1_000_000.0
return pd.DataFrame([{
"symbol_id": symbol,
"date": pd.Timestamp(date),
"portfolio_name": portfolio_name,
"target_weight": weight,
"target_value": target_value,
"target_shares": float(shares) + 0.25,
"position_shares": shares,
"position_value": target_value,
"price": price,
}], columns=POSITION_COLUMNS)
def _our_fills(
*,
symbol: str = "sh600000",
date: str = "2026-07-01",
shares: int = 1000,
price: float = 10.0,
cost: float = 5.0,
portfolio_name: str = "run1",
) -> pd.DataFrame:
fills = pd.DataFrame([{
"symbol_id": symbol,
"date": pd.Timestamp(date),
"portfolio_name": portfolio_name,
"prev_shares": 0,
"target_shares": shares,
"traded_shares": shares,
"realized_shares": shares,
"blocked": 0,
"trade_cost": cost,
"trade_price": price,
}])
return fills
def _our_pnl(
*,
date: str = "2026-07-01",
pnl: float = 100.0,
cost: float = 5.0,
portfolio_name: str = "run1",
) -> pd.DataFrame:
return pd.DataFrame([{
"date": pd.Timestamp(date),
"portfolio_name": portfolio_name,
"gross_exposure": 10_000.0,
"net_exposure": 10_000.0,
"pnl": pnl,
"cost": cost,
"turnover": 1.0,
"n_positions": 1,
}], columns=PNL_COLUMNS)
def _jq_fills(
*,
symbol: str = "sh600000",
date: str = "2026-07-01",
shares: int = 1000,
price: float = 10.0,
cost: float = 5.0,
portfolio_name: str = "run1",
raw_status: str = "filled",
) -> pd.DataFrame:
return pd.DataFrame([{
"date": date,
"portfolio_name": portfolio_name,
"symbol_id": symbol,
"jq_symbol": to_joinquant_symbol(symbol),
"order_id": "ord-1",
"side": "buy" if shares >= 0 else "sell",
"requested_shares": shares,
"filled_shares": shares,
"fill_price": price,
"trade_value": abs(shares * price),
"trade_cost": cost,
"blocked": 0,
"raw_status": raw_status,
}], columns=JOINQUANT_FILL_COLUMNS)
def _jq_positions(
*,
symbol: str = "sh600000",
date: str = "2026-07-01",
shares: int = 1000,
price: float = 10.0,
portfolio_name: str = "run1",
) -> pd.DataFrame:
return pd.DataFrame([{
"date": date,
"portfolio_name": portfolio_name,
"symbol_id": symbol,
"jq_symbol": to_joinquant_symbol(symbol),
"position_shares": shares,
"position_value": shares * price,
"cash": 990_000.0,
"total_value": 1_000_000.0,
}], columns=JOINQUANT_POSITION_COLUMNS)
def _jq_pnl(
*,
date: str = "2026-07-01",
pnl: float = 100.0,
cost: float = 5.0,
portfolio_name: str = "run1",
) -> pd.DataFrame:
return pd.DataFrame([{
"date": date,
"portfolio_name": portfolio_name,
"gross_exposure": 10_000.0,
"net_exposure": 10_000.0,
"cash": 990_000.0,
"total_value": 1_000_000.0,
"pnl": pnl,
"cost": cost,
"turnover": 1.0,
}], columns=JOINQUANT_PNL_COLUMNS)
def _write_parquets(tmp_path: Path, frames: dict[str, pd.DataFrame]) -> dict[str, Path]:
paths = {}
for name, frame in frames.items():
path = tmp_path / f"{name}.pq"
frame.to_parquet(path, index=False)
paths[name] = path
return paths
def _export_targets_for(tmp_path: Path, positions: pd.DataFrame) -> tuple[Path, Path]:
positions_path = tmp_path / "positions.pq"
positions.to_parquet(positions_path, index=False)
targets_root = tmp_path / "targets"
export_targets(
positions_path,
portfolio_name="run1",
out_dir=targets_root,
mode="target_shares",
)
return positions_path, targets_root / "run1"
@pytest.mark.parametrize(
("internal", "joinquant"),
[
("sh600000", "600000.XSHG"),
("sh688001", "688001.XSHG"),
("sz000001", "000001.XSHE"),
("sz001001", "001001.XSHE"),
("sz002594", "002594.XSHE"),
("sz300001", "300001.XSHE"),
],
)
def test_symbol_mapping_both_directions(internal, joinquant):
assert to_joinquant_symbol(internal) == joinquant
assert from_joinquant_symbol(joinquant) == internal
@pytest.mark.parametrize("bad", ["600000", "bj830000", "sh000001", "sz600000", "abc"])
def test_symbol_mapping_rejects_invalid_symbols(bad):
with pytest.raises(ValueError):
to_joinquant_symbol(bad)
@pytest.mark.parametrize("bad", ["600000", "600000.XSHE", "000001.XSHG", "abc.XSHG"])
def test_reverse_symbol_mapping_rejects_invalid_symbols(bad):
with pytest.raises(ValueError):
from_joinquant_symbol(bad)
def test_export_targets_schema_snapshot_hash_and_no_overwrite(tmp_path):
positions_path = tmp_path / "positions.pq"
_positions().to_parquet(positions_path, index=False)
snapshots = export_targets(
positions_path,
portfolio_name="run1",
out_dir=tmp_path / "targets",
mode="target_shares",
)
csv_path = tmp_path / "targets" / "run1" / "20260701.csv"
parquet_path = tmp_path / "targets" / "run1" / "20260701.parquet"
snapshot_path = tmp_path / "snapshots" / "run1" / "20260701.json"
assert csv_path.exists()
assert parquet_path.exists()
assert snapshot_path.exists()
target = pd.read_csv(csv_path)
assert list(target.columns) == JOINQUANT_TARGET_COLUMNS
assert int(target.loc[0, "target_shares"]) == 1000
assert float(target.loc[0, "target_value"]) == 10_000.0
assert target.loc[0, "export_mode"] == "target_shares"
snapshot = json.loads(snapshot_path.read_text())
actual_hash = hashlib.sha256(csv_path.read_bytes()).hexdigest()
assert snapshots[0]["file_sha256"] == actual_hash
assert snapshot["file_sha256"] == actual_hash
assert snapshot["n_symbols"] == 1
with pytest.raises(FileExistsError):
export_targets(
positions_path,
portfolio_name="run1",
out_dir=tmp_path / "targets",
mode="target_shares",
)
def test_export_targets_target_value_mode_from_position_columns(tmp_path):
positions_path = tmp_path / "positions.pq"
_positions(shares=250, price=20.0).to_parquet(positions_path, index=False)
export_targets(
positions_path,
portfolio_name="run1",
out_dir=tmp_path / "targets_value",
mode="target_value",
)
target = pd.read_parquet(tmp_path / "targets_value" / "run1" / "20260701.parquet")
assert list(target.columns) == JOINQUANT_TARGET_COLUMNS
assert target.loc[0, "export_mode"] == "target_value"
assert target.loc[0, "target_value"] == 5_000.0
assert target.loc[0, "target_shares"] == 250
def test_export_targets_can_shift_to_next_execution_session(tmp_path):
positions_path = tmp_path / "positions.pq"
_positions(date="2024-01-09").to_parquet(positions_path, index=False)
calendar_path = tmp_path / "daily.pq"
pd.DataFrame({
"date": pd.to_datetime(["2024-01-09", "2024-01-10", "2024-01-11"]),
"symbol_id": ["sh600000", "sh600000", "sh600000"],
}).to_parquet(calendar_path, index=False)
snapshots = export_targets(
positions_path,
portfolio_name="run1",
out_dir=tmp_path / "targets_shifted",
mode="target_shares",
start_date="2024-01-10",
end_date="2024-01-10",
execution_calendar_path=calendar_path,
)
assert len(snapshots) == 1
assert snapshots[0]["date"] == "2024-01-10"
assert (tmp_path / "targets_shifted" / "run1" / "20240110.csv").exists()
target = pd.read_csv(tmp_path / "targets_shifted" / "run1" / "20240110.csv")
assert target.loc[0, "date"] == "2024-01-10"
def test_ingest_permissive_csv_column_mapping_and_output_schemas(tmp_path):
fills_csv = tmp_path / "jq_fills.csv"
positions_csv = tmp_path / "jq_positions.csv"
pnl_csv = tmp_path / "jq_pnl.csv"
pd.DataFrame([{
"Trade Date": "2026-07-01 09:31:00",
"Security": "600000.XSHG",
"Direction": "buy",
"Order Amount": 1000,
"Filled Amount": 1000,
"Price": 10.0,
"Status": "filled",
}]).to_csv(fills_csv, index=False)
pd.DataFrame([{
"Date": "2026-07-01",
"Security": "600000.XSHG",
"Shares": 1000,
"Market Value": 10_000.0,
"Cash": 990_000.0,
"Portfolio Value": 1_000_000.0,
}]).to_csv(positions_csv, index=False)
pd.DataFrame([{
"Date": "2026-07-01",
"Portfolio Value": 1_000_000.0,
"Daily PnL": 100.0,
"Turnover": 1.0,
}]).to_csv(pnl_csv, index=False)
fills = normalize_fills_csv(fills_csv, "run1")
assert list(fills.columns) == JOINQUANT_FILL_COLUMNS
assert fills.loc[0, "symbol_id"] == "sh600000"
assert fills.loc[0, "jq_symbol"] == "600000.XSHG"
assert fills.loc[0, "trade_cost"] == 0.0
assert fills.loc[0, "blocked"] == 0
paths = ingest_joinquant_outputs(
portfolio_name="run1",
fills_csv=fills_csv,
positions_csv=positions_csv,
pnl_csv=pnl_csv,
out_dir=tmp_path / "ingested",
)
assert list(pd.read_parquet(paths["fills"]).columns) == JOINQUANT_FILL_COLUMNS
assert list(pd.read_parquet(paths["positions"]).columns) == JOINQUANT_POSITION_COLUMNS
assert list(pd.read_parquet(paths["pnl"]).columns) == JOINQUANT_PNL_COLUMNS
def _run_reconcile_case(
tmp_path: Path,
*,
positions: pd.DataFrame | None = None,
our_fills: pd.DataFrame | None = None,
jq_fills: pd.DataFrame | None = None,
jq_positions: pd.DataFrame | None = None,
our_pnl: pd.DataFrame | None = None,
jq_pnl: pd.DataFrame | None = None,
) -> pd.DataFrame:
positions = _positions() if positions is None else positions
_, targets_dir = _export_targets_for(tmp_path, positions)
paths = _write_parquets(tmp_path, {
"our_fills": _our_fills() if our_fills is None else our_fills,
"our_positions": positions,
"our_pnl": _our_pnl() if our_pnl is None else our_pnl,
"jq_fills": _jq_fills() if jq_fills is None else jq_fills,
"jq_positions": _jq_positions() if jq_positions is None else jq_positions,
"jq_pnl": _jq_pnl() if jq_pnl is None else jq_pnl,
})
out_paths = reconcile_joinquant(
portfolio_name="run1",
targets_dir=targets_dir,
our_fills_path=paths["our_fills"],
our_positions_path=paths["our_positions"],
our_pnl_path=paths["our_pnl"],
jq_fills_path=paths["jq_fills"],
jq_positions_path=paths["jq_positions"],
jq_pnl_path=paths["jq_pnl"],
out_dir=tmp_path / "reconcile",
)
report = pd.read_parquet(out_paths["daily_reconcile"])
assert list(report.columns) == RECONCILE_COLUMNS
assert out_paths["summary_md"].exists()
assert out_paths["summary_csv"].exists()
return report
def test_reconcile_exact_match(tmp_path):
report = _run_reconcile_case(tmp_path)
assert report.loc[0, "diff_reason"] == "MATCH"
assert report.loc[0, "filled_share_diff"] == 0
assert report.loc[0, "position_share_diff"] == 0
def test_reconcile_price_mismatch(tmp_path):
report = _run_reconcile_case(tmp_path, jq_fills=_jq_fills(price=10.5))
assert report.loc[0, "diff_reason"] == "PRICE_MISMATCH"
def test_reconcile_cost_mismatch(tmp_path):
report = _run_reconcile_case(
tmp_path,
jq_fills=_jq_fills(cost=8.0),
jq_pnl=_jq_pnl(cost=8.0),
)
assert report.loc[0, "diff_reason"] == "COST_MODEL"
def test_reconcile_missing_symbol_in_joinquant(tmp_path):
empty_jq_fills = pd.DataFrame(columns=JOINQUANT_FILL_COLUMNS)
empty_jq_positions = pd.DataFrame(columns=JOINQUANT_POSITION_COLUMNS)
report = _run_reconcile_case(
tmp_path,
jq_fills=empty_jq_fills,
jq_positions=empty_jq_positions,
)
assert report.loc[0, "diff_reason"] == "MISSING_IN_JOINQUANT"
def test_reconcile_short_target_with_long_only_joinquant_output(tmp_path):
positions = _positions(shares=-100, price=10.0)
our_fills = _our_fills(shares=-100, price=10.0)
jq_fills = _jq_fills(shares=0, price=10.0, cost=0.0, raw_status="short clipped")
jq_positions = _jq_positions(shares=0, price=10.0)
report = _run_reconcile_case(
tmp_path,
positions=positions,
our_fills=our_fills,
jq_fills=jq_fills,
jq_positions=jq_positions,
)
assert report.loc[0, "diff_reason"] == "SHORT_NOT_SUPPORTED"
def test_joinquant_cli_smoke_export_ingest_reconcile_and_wrapper(tmp_path):
runner = CliRunner()
positions_path = tmp_path / "positions.pq"
_positions().to_parquet(positions_path, index=False)
result = runner.invoke(cli, [
"joinquant", "export-targets",
"--positions-path", str(positions_path),
"--portfolio-name", "run1",
"--mode", "target_shares",
"--out-dir", str(tmp_path / "targets"),
])
assert result.exit_code == 0, result.output
assert "Exported JoinQuant targets" in result.output
fills_csv = tmp_path / "jq_fills.csv"
positions_csv = tmp_path / "jq_positions.csv"
pnl_csv = tmp_path / "jq_pnl.csv"
_jq_fills().to_csv(fills_csv, index=False)
_jq_positions().to_csv(positions_csv, index=False)
_jq_pnl().to_csv(pnl_csv, index=False)
result = runner.invoke(cli, [
"joinquant", "ingest",
"--portfolio-name", "run1",
"--fills-csv", str(fills_csv),
"--positions-csv", str(positions_csv),
"--pnl-csv", str(pnl_csv),
"--out-dir", str(tmp_path / "ingested"),
])
assert result.exit_code == 0, result.output
assert "Saved JoinQuant fills" in result.output
paths = _write_parquets(tmp_path, {
"our_fills": _our_fills(),
"our_pnl": _our_pnl(),
})
result = runner.invoke(cli, [
"joinquant", "reconcile",
"--portfolio-name", "run1",
"--targets-dir", str(tmp_path / "targets" / "run1"),
"--our-fills-path", str(paths["our_fills"]),
"--our-positions-path", str(positions_path),
"--our-pnl-path", str(paths["our_pnl"]),
"--jq-fills-path", str(tmp_path / "ingested" / "run1" / "fills.pq"),
"--jq-positions-path", str(tmp_path / "ingested" / "run1" / "positions.pq"),
"--jq-pnl-path", str(tmp_path / "ingested" / "run1" / "pnl.pq"),
"--out-dir", str(tmp_path / "reconcile"),
])
assert result.exit_code == 0, result.output
assert "Saved reconciliation parquet" in result.output
wrapper_path = tmp_path / "wrapper_strategy_run1.py"
result = runner.invoke(cli, [
"joinquant", "write-wrapper",
"--portfolio-name", "run1",
"--mode", "target_shares",
"--out-path", str(wrapper_path),
])
assert result.exit_code == 0, result.output
assert "Saved JoinQuant wrapper strategy" in result.output
text = wrapper_path.read_text()
assert 'PORTFOLIO_NAME = "run1"' in text
assert 'TARGET_MODE = "target_shares"' in text
assert "ALLOW_SHORT = False" in text
def test_wrapper_strategy_generation_smoke(tmp_path):
path = write_wrapper_strategy(
portfolio_name="run2",
mode="target_value",
out_path=tmp_path / "wrapper.py",
)
text = path.read_text()
assert 'PORTFOLIO_NAME = "run2"' in text
assert 'TARGET_MODE = "target_value"' in text
assert "order_target_value" in text
def test_wrapper_strategy_can_embed_target_csvs(tmp_path):
targets_dir = tmp_path / "targets"
targets_dir.mkdir()
(targets_dir / "20260701.csv").write_text(
"date,portfolio_name,symbol_id,jq_symbol,target_shares,target_value,export_mode\n"
"2026-07-01,run1,sh600000,600000.XSHG,1000,10000,target_shares\n",
encoding="utf-8",
)
path = write_wrapper_strategy(
portfolio_name="run1",
mode="target_shares",
out_path=tmp_path / "wrapper_embedded.py",
embedded_targets_dir=targets_dir,
)
text = path.read_text()
assert '"20260701.csv"' in text
assert "600000.XSHG" in text
assert "if file_name in _EMBEDDED_TARGETS" in text
def test_build_fixed_share_positions_excludes_final_executionless_date():
data = pd.DataFrame({
"symbol_id": ["sh600000", "sh600000", "sh600000"],
"date": pd.to_datetime(["2024-01-09", "2024-01-10", "2024-01-11"]),
"close": [10.0, 10.5, 11.0],
})
positions = build_fixed_share_positions(
data,
trade_symbol="sh600000",
portfolio_name="run1",
shares=1000,
booksize=1_000_000.0,
)
assert list(positions.columns) == POSITION_COLUMNS
assert positions["date"].dt.strftime("%Y-%m-%d").tolist() == [
"2024-01-09",
"2024-01-10",
]
assert positions["position_shares"].tolist() == [1000, 1000]
assert positions["target_value"].tolist() == [10_000.0, 10_500.0]
def test_browser_config_template_and_placeholder_resolution(tmp_path):
config_path = write_browser_config_template(
tmp_path / "browser_config.json",
strategy_url="https://www.joinquant.com/example",
)
config = json.loads(config_path.read_text())
assert config["strategy_url"] == "https://www.joinquant.com/example"
assert config["actions"][0]["type"] == "goto"
context = {
"wrapper_path": "/tmp/wrapper.py",
"target_csvs": ["/tmp/20240110.csv", "/tmp/20240111.csv"],
"expected_joinquant_csvs": {"fills": "/tmp/jq_fills.csv"},
}
assert resolve_template("{wrapper_path}", context) == "/tmp/wrapper.py"
assert resolve_template("{target_csvs}", context) == [
"/tmp/20240110.csv",
"/tmp/20240111.csv",
]
assert resolve_template("save:{expected_joinquant_csvs.fills}", context) == "save:/tmp/jq_fills.csv"
script = "(arg) => { return new Event('change', {bubbles: true}); }"
assert resolve_template(script, context) == script
def test_load_env_file_handles_quotes_without_shell_sourcing(tmp_path):
env_path = tmp_path / "joinquant.env"
env_path.write_text(
"JOINQUANT_USERNAME=alice\n"
"JOINQUANT_PASSWORD=\"secret\"\n"
"JOINQUANT_STRATEGY_URL='https://example.test/path?x=1&y=2'\n"
)
env = load_env_file(env_path)
assert env["JOINQUANT_USERNAME"] == "alice"
assert env["JOINQUANT_PASSWORD"] == "secret"
assert env["JOINQUANT_STRATEGY_URL"] == "https://example.test/path?x=1&y=2"
def test_sim_trade_browser_config_template(tmp_path):
config_path = write_browser_config_template(
tmp_path / "sim_config.json",
strategy_url="https://www.joinquant.com/sim",
flow="sim-trade",
)
config = json.loads(config_path.read_text())
assert config["flow"] == "sim-trade"
selectors = " ".join(
action.get("selector", "") for action in config["actions"]
)
assert "模拟盘" in selectors
assert "模拟交易" in selectors
assert any(
action["type"] == "screenshot"
and action["path"] == "{run_artifact_dir}/sim_trade_final.png"
for action in config["actions"]
)
def test_joinquant_cli_browser_config_smoke(tmp_path):
runner = CliRunner()
config_path = tmp_path / "browser_config.json"
result = runner.invoke(cli, [
"joinquant",
"write-browser-config",
"--out-path",
str(config_path),
"--strategy-url",
"https://www.joinquant.com/example",
])
assert result.exit_code == 0, result.output
assert config_path.exists()
assert default_browser_config()["actions"]
sim_config_path = tmp_path / "sim_browser_config.json"
result = runner.invoke(cli, [
"joinquant",
"write-browser-config",
"--out-path",
str(sim_config_path),
"--strategy-url",
"https://www.joinquant.com/sim",
"--flow",
"sim-trade",
])
assert result.exit_code == 0, result.output
assert json.loads(sim_config_path.read_text())["flow"] == "sim-trade"
Generated
+122 -1
View File
@@ -296,6 +296,9 @@ dependencies = [
backtrader = [ backtrader = [
{ name = "backtrader" }, { name = "backtrader" },
] ]
joinquant-browser = [
{ name = "playwright" },
]
[package.dev-dependencies] [package.dev-dependencies]
dev = [ dev = [
@@ -311,9 +314,10 @@ requires-dist = [
{ name = "click", specifier = ">=8.0.0" }, { name = "click", specifier = ">=8.0.0" },
{ name = "matplotlib", specifier = ">=3.7.0" }, { name = "matplotlib", specifier = ">=3.7.0" },
{ name = "pandas", specifier = ">=2.0.0" }, { name = "pandas", specifier = ">=2.0.0" },
{ name = "playwright", marker = "extra == 'joinquant-browser'", specifier = ">=1.61.0" },
{ name = "pyarrow", specifier = ">=14.0.0" }, { name = "pyarrow", specifier = ">=14.0.0" },
] ]
provides-extras = ["backtrader"] provides-extras = ["backtrader", "joinquant-browser"]
[package.metadata.requires-dev] [package.metadata.requires-dev]
dev = [ dev = [
@@ -744,6 +748,92 @@ wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2c/47/c99d5268f354002ce80f8d029cd9d7d872969da1de8b93d32de4dc56d6f4/fonttools-4.63.0-py3-none-any.whl", hash = "sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d", size = 1164562, upload-time = "2026-05-14T12:04:29.092Z" }, { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2c/47/c99d5268f354002ce80f8d029cd9d7d872969da1de8b93d32de4dc56d6f4/fonttools-4.63.0-py3-none-any.whl", hash = "sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d", size = 1164562, upload-time = "2026-05-14T12:04:29.092Z" },
] ]
[[package]]
name = "greenlet"
version = "3.5.3"
source = { registry = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/" }
sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e2/f1/fbbfef6af0bad0548f09bc28948ea3c275b4edb19e17fc5ca9900a6a634d/greenlet-3.5.3.tar.gz", hash = "sha256:a61efc018fd3eb317eeca31aba90ee9e7f26f22884a79b6c6ec715bf71bb62f1", size = 200270, upload-time = "2026-06-26T19:28:24.832Z" }
wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f5/a1/1f7f0c555f5858fd2906fe9f7b0a3554fddb85cb70df7a6aaec41dc292c2/greenlet-3.5.3-cp310-cp310-macosx_11_0_universal2.whl", hash = "sha256:c180d22d325fb613956b443c3c6f4406eb70e6defc70d3974da2a7b59e06f48c", size = 285838, upload-time = "2026-06-26T18:21:05.167Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0a/29/be9f43ed61677a5759b38c8a9389248133c8c731bbfc0574ecdff66c99fc/greenlet-3.5.3-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:483d08c11181c83a6ce1a7a61df0f624a208ec40817a3bb2302714592eee4f04", size = 602342, upload-time = "2026-06-26T19:07:06.908Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b9/42/ba41c97ec36aa4b3ec25e5aa691d79561254805fad7f2f826dd6770587e2/greenlet-3.5.3-cp310-cp310-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1dae6e0091eae084317e411f047f0b7cb241c6db570f7c45fd6b900a274914ce", size = 615541, upload-time = "2026-06-26T19:10:04.909Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2e/8c/231ca675b0df779816950ca66b40b1fa14dbff4a0ed9814a9a29ec399140/greenlet-3.5.3-cp310-cp310-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0f6ff50ff8dbd51fae9b37f4101648b04ea0df19b3f50ab2beb5061e7716a5c8", size = 622473, upload-time = "2026-06-26T19:24:12.786Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f5/c7/28747042e1df8a9cd120a1ebe15529fc4be3b486e13e8d551ff307a82412/greenlet-3.5.3-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9bcd2d72ccd70a1ec68ba6ef93e7fbb4420ef9997dabc7010d893bd4015e0bec", size = 615675, upload-time = "2026-06-26T18:32:14.444Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/81/fe/dd97c483a3ff82849196ccd07851600edd3ac9de74669ca8a6022ada9ea1/greenlet-3.5.3-cp310-cp310-manylinux_2_39_riscv64.whl", hash = "sha256:37bf9c538f5ae6e63d643f88dec37c0c83bdf0e2ebc62961dedcf458822f7b71", size = 418421, upload-time = "2026-06-26T19:25:34.503Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/cc/a8/b85525a6c8fba9f009a5f7c8df1545de8fb0f0bf3e0179194ef4e500317f/greenlet-3.5.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:73f152c895e09907e0dbe24f6c2db37beb085cd63db91c3825a0fcd0064124a8", size = 1575057, upload-time = "2026-06-26T19:09:00.264Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/03/79/fb76edb218fe6735ab0edeba176c7ab80df9618f7c02ce4208979f3ae7db/greenlet-3.5.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:8bdb43e1a1d1873721acab2be99c5befd4d2044ddfd52e4d610801019880a702", size = 1641692, upload-time = "2026-06-26T18:31:41.454Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/6b/79/86fe3ee50ed55d9b3907eecd3208b5c3fe8a79515519aae98b4753c3fa1d/greenlet-3.5.3-cp310-cp310-win_amd64.whl", hash = "sha256:0909f9355a9f24845d3299f3112e266a06afb68302041989fd26bd68894933db", size = 238742, upload-time = "2026-06-26T18:20:40.758Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/51/58/5404031044f55afad7aad1aff8be3f22b1bed03e237cfeabbc7e5c8cfde0/greenlet-3.5.3-cp311-cp311-macosx_11_0_universal2.whl", hash = "sha256:aca9b4ce85b152b5524ef7d88170efdff80dc0032aa8b75f9aaf7f3479ea95b4", size = 287424, upload-time = "2026-06-26T18:20:31.469Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b4/bf/1c65e9b94a54d547068fa5b5a8a06f221f3316b48908e08668d29c77cb50/greenlet-3.5.3-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f71be4920368fe1fabeeaa53d1e3548337e2b223d9565f8ad5e392a75ba23fc", size = 606523, upload-time = "2026-06-26T19:07:08.859Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b8/c7/b66baacc95775ad511287acb0137b95574a9ce5491902372b7564799d790/greenlet-3.5.3-cp311-cp311-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4d77e67f65f98449e3fb83f795b5d0a8437aead2f874ca89c96576caf4be3af6", size = 618315, upload-time = "2026-06-26T19:10:06.055Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b0/a0/68afd1ebad40db87dac0a28ffa120726b98bf9c7c40c481b0f63c105d298/greenlet-3.5.3-cp311-cp311-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:e18619ba655ac05d78d80fc83cac4ba892bd6927b99e3b8237aee861aaacc8bb", size = 626155, upload-time = "2026-06-26T19:24:14.44Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/78/2b/28ed29463522fdbe4c15b1f63922041626a7478316b34ab4adda3f0a4aba/greenlet-3.5.3-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8540f1e6205bd13ca0ce685581037219ca54a1b41a0a15d228c6c9b8ad5903d7", size = 617381, upload-time = "2026-06-26T18:32:16.077Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/07/7f/e327d912239ec4b3b49999e3967389bcf1ee8722b9ee9194d2752ecd558a/greenlet-3.5.3-cp311-cp311-manylinux_2_39_riscv64.whl", hash = "sha256:d27c0c653a60d9535f690226474a5cc1036a8b0d7b57504d1c4f89c44a07a80c", size = 421083, upload-time = "2026-06-26T19:25:35.804Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2a/7b/ad04e9d1337fc04965dc9fc616b6a72cb65a24b800a014c011ec812f5489/greenlet-3.5.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:7ef56fe650f50575bf843acde967b9c567687f3c22340941a899b7bc56e956a8", size = 1577771, upload-time = "2026-06-26T19:09:01.537Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/d8/33/6c87ab7ba663f70ca21f3022aad1ffe56d3f3e0521e836c2415e13abcc3c/greenlet-3.5.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:5121af01cf911e70056c00d4b46d5e9b5d1415550038573d744138bacb59e6b8", size = 1644048, upload-time = "2026-06-26T18:31:42.996Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/1c/35/f0d8ee998b422cf8693b270f098e55d8d4ec8006b061b333f54f177d28d9/greenlet-3.5.3-cp311-cp311-win_amd64.whl", hash = "sha256:0f41e4a05a3c0cb31b17023eff28dd111e1d16bf7d7d00406cd7df23f31398a7", size = 239137, upload-time = "2026-06-26T18:23:21.664Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fb/96/b9820295576ef18c9edc404f10e260ae7215ceaf3781a54b720ed2627862/greenlet-3.5.3-cp311-cp311-win_arm64.whl", hash = "sha256:ec6f1af59f6b5f3fc9678e2ea062d8377d22ac644f7844cb7a292910cf12ff44", size = 237630, upload-time = "2026-06-26T18:24:00.281Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/5d/6e/4c37d51a2b7f82d2ff11bb6b5f7d766d9a011726624af255e843727627a3/greenlet-3.5.3-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:719757059f5a53fd0dde23f78cffeafcdd97b21c850ddb7ca684a3c1a1f122e2", size = 288685, upload-time = "2026-06-26T18:22:08.977Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7a/73/815dd90131c1b71ebdf53dbc7c276cafec2a1173b97559f97aba72724a87/greenlet-3.5.3-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:efa9f765dd09f9d0cdac651ffdf631ee59ec5dc6ee7a73e0c012ba9c52fbdf5b", size = 604761, upload-time = "2026-06-26T19:07:10.114Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/9f/57/079cfe76bcef36b153b25607ee91c6fcb58f17f8b23c86bbbeabe0c88d72/greenlet-3.5.3-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7faba15ac005376e02a0384504e0243be3370ce010296a44a820feb342b505ab", size = 617044, upload-time = "2026-06-26T19:10:07.25Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/fb/fb/d97dc261209c80744b7c8132693a30d70ec6e7315e632cb0a10b3fec94dd/greenlet-3.5.3-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5795cd1101371140551c645f2d408b8d3c01a5a29cf8a9bce6e759c983682d23", size = 622351, upload-time = "2026-06-26T19:24:16.32Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/37/87/b4d095775a3fb1bcafbb483fc206b27ebb785724c83051447737085dc54e/greenlet-3.5.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:87142215824be6ac05e2e8e2786eec307ccbc27c36723c3881959df654af6861", size = 614244, upload-time = "2026-06-26T18:32:17.594Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8e/ac/e5fee13cbbd0e8de312d9a146584b8a51891c68847330ef9dc8b5109d23f/greenlet-3.5.3-cp312-cp312-manylinux_2_39_riscv64.whl", hash = "sha256:af4923b3096e26a36d7e9cf24ab88083a20f97d191e3b97f253731ce9b41b28c", size = 425395, upload-time = "2026-06-26T19:25:37.144Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8a/70/7559b609683650fa2b95b8ab84b4ab0b26556a635d19675e12aa832d826d/greenlet-3.5.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:215275b1b49320987352e6c1b054acca0064f965a2c66992bed9a6f7d913f149", size = 1574210, upload-time = "2026-06-26T19:09:03.077Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ae/73/be55392074c60fc37655ca40fa6022457bfbf6718e9e342a7b0b41f96dd2/greenlet-3.5.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6b1b0eed82364b0e32c4ea0f221452d33e6bb17ae094d9f72aed9851812747ea", size = 1638627, upload-time = "2026-06-26T18:31:44.748Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/14/40/c57489acf8e37d74e2913d4eff63aa0dba17acccc4bdeef874dde2dbbec9/greenlet-3.5.3-cp312-cp312-win_amd64.whl", hash = "sha256:cde8adafa2365676f74a979744629589999093bc86e2484214f58e61df08902c", size = 239882, upload-time = "2026-06-26T18:23:27.518Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/71/fd/6fea0e3d6600f785069481ee637e09378dd4118acdfd38ad88ae2db31c98/greenlet-3.5.3-cp312-cp312-win_arm64.whl", hash = "sha256:c4e7b79d83805475f0102008843f6eb45fd3bb0b2e88c774adab5fbaab27117d", size = 238211, upload-time = "2026-06-26T18:22:37.671Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/9b/ff/a620267401db30a50cc8450ee90730e2d4a85658c055c0e760d4ed47fb13/greenlet-3.5.3-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:c8d87c2134d871df96ecdea9cec7cbaab286dadab0f56476e57aaf9e8ac11550", size = 287609, upload-time = "2026-06-26T18:21:14.724Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/d6/fa/5401ac78021c826a25b6dde0c705e0a8f29b617509f9185a31dac15fbe1b/greenlet-3.5.3-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a2d185dd1621757e70c3861cceffd5317ab4e7ed7eb09c82994828468527ade5", size = 607435, upload-time = "2026-06-26T19:07:11.412Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e9/76/1dc144a2e56e65d36405078ed774224375ea520a1870a6e46e08bb4ac7bf/greenlet-3.5.3-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1c514a468149bf8fbbab874188a3535cd8a48a3e353eb53a3d424296f8dbacd3", size = 619787, upload-time = "2026-06-26T19:10:08.396Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/57/61/2f5b1adf256d039f5dab8005de8d3d7ad2b0070a3219c0e036b3fbfeb440/greenlet-3.5.3-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:9ad04dd75458c6300b047c61b8639092433d205a25a14e310d6582a480efcca1", size = 625580, upload-time = "2026-06-26T19:24:18.344Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/bf/87/c298cee62df1de4ad7fec32abda73526cff347fd143a6ed4ac369246668a/greenlet-3.5.3-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:915f887cf2682b66419b879423a2e072634aa7b7dce6f3ada4957cfced3f1e9a", size = 616786, upload-time = "2026-06-26T18:32:19.128Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/3e/d9/ab7fc9e543e44d6879b0a6ef9a4b2188940fd180cc65d6f646883ddf7201/greenlet-3.5.3-cp313-cp313-manylinux_2_39_riscv64.whl", hash = "sha256:afaabdd554cd7ae9bbb3ca070b0d7fdfd207dbf1d16865f7233837709d354bda", size = 427933, upload-time = "2026-06-26T19:25:38.219Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/9e/2e/e6f009885ed0705ccf33fe0583c117cfd03cde77e31a596dd5785a30762b/greenlet-3.5.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:766cfd421c13e450feb340cd472a3ed9957d438727b7b4593ad7c76c5d2b0deb", size = 1574316, upload-time = "2026-06-26T19:09:04.273Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ef/fe/43fd110b01e40da0adb7c90ac7ea744bef2d43dca00de5095fd2351c2a68/greenlet-3.5.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2ecda9ec22edf38fa389369eaed8c3d37c05f3c54e69f69438dbb2cc1de1458b", size = 1638614, upload-time = "2026-06-26T18:31:46.297Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0f/7c/062447147a61f8b4337b156fe70d32a165fcf2f89d7ca6255e572806705c/greenlet-3.5.3-cp313-cp313-win_amd64.whl", hash = "sha256:c82304750f057167ff60d188df1d0cc1764ce9567eadf03e6a7443bcedd0b30b", size = 239850, upload-time = "2026-06-26T18:21:54.613Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c7/7e/220a7f5824a64a60443fc03b39dfac4ea63a7fb6d481efa27eafa928e7f4/greenlet-3.5.3-cp313-cp313-win_arm64.whl", hash = "sha256:dc133a1569ee667b2a6ef56ce551084aeefd87a5acbc4736d336d1e2edc6cfc4", size = 238141, upload-time = "2026-06-26T18:22:48.507Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c3/93/43e116ee114b28737ba7e12952a0d4e2f55944d0f84e42bc91ba7192a3c9/greenlet-3.5.3-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:fd2e02fa07485778536a036222d616ab957b1d533f36b3ed98ce725d9c9d3117", size = 288202, upload-time = "2026-06-26T18:23:49.604Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/82/2f/146d218299046a43d1f029fd544b3d110d0f175a09c715c7e8da4a4a345d/greenlet-3.5.3-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:df0a0628d1597eb0897b62f55d1343f772405fd25f3b2a796c76874b0c2e22e8", size = 654096, upload-time = "2026-06-26T19:07:12.71Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a0/cc/04738cafb3f45fa991ea44f9de94c47dcec964f5a972300988a6751f49d9/greenlet-3.5.3-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ebd933a6adabc298bab47731a130fe6bfb888bd934eee37810f151159544540d", size = 666304, upload-time = "2026-06-26T19:10:09.503Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/86/a9/73fa62893d5b84b4205544e6b673c654cc43aa5b9899bac00f04d64af73d/greenlet-3.5.3-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:8d19fe6c39ebff9259f07bcc685d3290f8fa4ea2278e51dd0008e4d6b0f2d814", size = 670657, upload-time = "2026-06-26T19:24:19.967Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ce/aa/4e0dad5e605c270c784ab911c43da6adb136ccd4d81180f763ca429a723d/greenlet-3.5.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4b9d501b40e80b70e32323c799dd9b420a5577a9601469d362ae1ffb690f3a7c", size = 663635, upload-time = "2026-06-26T18:32:20.802Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/29/7e/2ffce64929fb3cab7b65d5a0b20aaf9764e227681d731b041077fc9a525a/greenlet-3.5.3-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:962c5df2db8cb446da51edf1ca5296c389d93b99c9d8aa2ee4c7d0d8f1218260", size = 473497, upload-time = "2026-06-26T19:25:39.421Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/d1/50/13efdbea246fe3d3b735e191fec08fb50809f53cd2383ebe123d0809e44b/greenlet-3.5.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a1fad1d11e7d6aab184107baa8e4ece11ccba3ec9599cd7efa5ff4d70d43256a", size = 1621252, upload-time = "2026-06-26T19:09:05.647Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f7/22/c0a336ae4a1410fd5f5121098e5bfbf1865f64c5ef80b4b5412886c4a332/greenlet-3.5.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:fad5aec764399f1b5cc347ad250a59660f20c8f8888ea6bae1f93b769cce1154", size = 1684824, upload-time = "2026-06-26T18:31:47.738Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7a/94/91aec0030bea75c4b3244251d0de60a1f3432d1ecb53ab6c437fb5c3ba61/greenlet-3.5.3-cp314-cp314-win_amd64.whl", hash = "sha256:7669aa24cf2a1041d6f7899575b494a3ab4cf68bfcc8609b1dc0be7272db835e", size = 240754, upload-time = "2026-06-26T18:22:15.669Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e5/06/68d0983e79e02138f64b4d303c500c27ddb48e5e77f3debb80888a921eae/greenlet-3.5.3-cp314-cp314-win_arm64.whl", hash = "sha256:5b4807c4082c9d1b6d9eed56fcd041863e37f2228106eef24c30ca096e238605", size = 239549, upload-time = "2026-06-26T18:22:42.996Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/91/95/3e161213d7f1d378d15aa9e792093e9bfe01844680d04b7fd6e0107c9098/greenlet-3.5.3-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:271a8ea7c1024e8a0d7dd2be66dd66dda8a07193f41a17b9e924f7600f5b62be", size = 296389, upload-time = "2026-06-26T18:22:20.657Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/00/92/715c44721abe2b4d1ae9abde4179411868a5bff312479f54e105d372f131/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:19131729ae0ddc3c2e1ef85e650169b5e37ee32e400f215f78b94d7b0d567310", size = 653382, upload-time = "2026-06-26T19:07:14.209Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a0/83/37a10372a1090a6624cca8e74c12df1a36c2dc36429ed0255b7fb1aeee23/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1540dd8e5fc2a5aec40fbb98ef8e149fa47c89a4b4a1cf2575a14d3d1869d7a8", size = 659401, upload-time = "2026-06-26T19:10:10.876Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/cb/73/8faec206b851c22b1733545fda900829a1f3f5b1c78ae7e0fb3dba57d9f4/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b897d97759425953f69a9c0fac67f8fe333ec0ce7377ef186fb2b0c3ad5e354d", size = 659582, upload-time = "2026-06-26T19:24:21.357Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/db/e2/d1509cad4207da559cc42986ecdd8fc67ad0d1bba2bf03023c467fd5e0f3/greenlet-3.5.3-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e81fa194a1d20967877bdf9c7794db2bc99063e5be36aee710c08f04c5bb087f", size = 656969, upload-time = "2026-06-26T18:32:22.272Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b4/55/50c19e49f8045834ada71ef12f8ad048eba8517c6aa41161bed676328fae/greenlet-3.5.3-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:3236754d423955ea08e9bb5f6c04a7895f9e22c290b66aa7653fcb922d839eb0", size = 491037, upload-time = "2026-06-26T19:25:40.672Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/86/7d/eaf70de20aadca3a5884aec58362861c64ce45e7b277f47ed026926a3b89/greenlet-3.5.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:55cf4d777485d43110e47133cbba6d74a8885a87ec1227ef0267f9ee80c5aa21", size = 1617822, upload-time = "2026-06-26T19:09:06.893Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8a/f9/414d38fc400ae4350d4185eaad1827676f7cf5287b9136e0ed1cbbe20a7f/greenlet-3.5.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:12a248ba75f6a9a236375f52296c498c89ff1d8badf32deb9eca7abd5853f7da", size = 1677983, upload-time = "2026-06-26T18:31:49.396Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e4/15/7edb977e08f9bff702fe42d6c902702786ff6b9694058b4e6a2a6ac90e57/greenlet-3.5.3-cp314-cp314t-win_amd64.whl", hash = "sha256:efc6bd60ea02e085862c74a3ef64b147ffc6f1a5ea7d9f26e7a939943f68c1e3", size = 243626, upload-time = "2026-06-26T18:24:41.485Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/2c/8a/93928dce91e6b3598b5e779e8d1fd6576a504640c58e78627077f6a7a91a/greenlet-3.5.3-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:ea03f2f04367845d6b58eeed276e1e56e51f0b97d8ad5a88a7d20a91dc9056cc", size = 288860, upload-time = "2026-06-26T18:22:48.07Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/4f/ca/69db42d447a1378043e2c8f19c09cbbd1263371505053c496b49066d3d16/greenlet-3.5.3-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:78dbef602fda6d97d957eb7937f70c9ce9e9527330347f8f6b6f9e554a9e7a47", size = 659747, upload-time = "2026-06-26T19:07:15.565Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a8/0b/af7ac2ef8dd41e3da1a40dda6305c23b9a03e13ba975ec916357b50f8575/greenlet-3.5.3-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6f73857adb8fee13fa56c172bd11262f888c0c648f9fea113e777bb2c7904a81", size = 670419, upload-time = "2026-06-26T19:10:12.293Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/25/aa/952cf28c2ff949a8c971134fb43854dd7eaa737218723aaef758f8c9aead/greenlet-3.5.3-cp315-cp315-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:cefa9cef4b371f9844c6053db71f1138bc6807bab1578b0dae5149c1f1141357", size = 674261, upload-time = "2026-06-26T19:24:22.79Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/51/1e/1d51640cacbfc455dbe9f9a9f594c49e4e244f63b9971a2f4764e46cc53d/greenlet-3.5.3-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:232fec92e823addaf02d9472cf7381e24a1d046a6ced1103c5caa4c21b9dfc1d", size = 668787, upload-time = "2026-06-26T18:32:24.298Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/dc/f2/b00d6f5e63e531a93562b2ec1a4c320fbee91f580fc42e6417af69d706e5/greenlet-3.5.3-cp315-cp315-manylinux_2_39_riscv64.whl", hash = "sha256:6219b6d04dbf6ba6084d77dc609e8473060dc55f759cbf626d512122781fa128", size = 480322, upload-time = "2026-06-26T19:25:41.852Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/21/66/4030d5b0b5894500023f003bb054d9bb354dfbd1e186c3a296759172f5f5/greenlet-3.5.3-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:2421c3564da9429d5586d46ca31ebb26516b5498a802cf65c041a8e8a8980d34", size = 1626305, upload-time = "2026-06-26T19:09:08.281Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0e/50/5221371c7550108dfa3c378debc41d032aa9c78e89abb01d8011cfc93289/greenlet-3.5.3-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e0f0d160f0b2e558e6c75f7930967183255dc9735e5f5b8cae58ee09c9576d8b", size = 1688631, upload-time = "2026-06-26T18:31:51.278Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/68/5d/00d469daae3c65d2bf620b10eee82eb022127d483c6bc8c69fae6f3fbf17/greenlet-3.5.3-cp315-cp315-win_amd64.whl", hash = "sha256:dd99329bbc15ca78dcc583dba05d0b1b0bae01ab6c2174989f5aaee3e41ac930", size = 241027, upload-time = "2026-06-26T18:22:38.203Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e7/e8/883785b44c5780ed71e83d3e4437e710470be17a2e181e8b601e2da0dc4a/greenlet-3.5.3-cp315-cp315-win_arm64.whl", hash = "sha256:499fef2acede88c1864a57bb586b4bf533c81e1b82df7ab93451cdb47dfec227", size = 240085, upload-time = "2026-06-26T18:23:54.217Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/1c/da/4f4a8450962fad137c1c8981a3f1b8919d06c829993d4d476f9c525d5173/greenlet-3.5.3-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:176bc16a721fa5fc294d70b87b4dfa5fbdd251b3da5d5372735ecef9bd7d6d0c", size = 297221, upload-time = "2026-06-26T18:23:27.176Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/57/66/b3bfae3e220a9b63ea539a0eea681800c69ab1aada757eae8789f183e7ce/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:629b614d2b786e89c50440e246f33eea78f58a962d0bdbbcc809e6d13605903f", size = 657221, upload-time = "2026-06-26T19:07:16.973Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/7b/81/b6d4d73a709684fc77e7fa034d7c2fe82cffa9fc920fadcaa659c2626213/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2b2e857ae16f5f72142edf75f9f176fe7526ba19a2841df1420516f83831c9f2", size = 663226, upload-time = "2026-06-26T19:10:13.723Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/e9/39/0e0938a75115b939d42733a2a12e1d349653c9531fe6fe563e8a681f04e6/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:16d192579ed281051396dddd7f7754dac6259e6b1fb26378c87b66622f8e3f91", size = 663706, upload-time = "2026-06-26T19:24:24.312Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f5/07/e210b02b589f16e74ff48b730690e4a34ffe984219fce4f3c1a0e7ec8545/greenlet-3.5.3-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e515757e2e36bcbf1fad09a46e1557e8b1ae1797d4b44d09da7deed88ad28608", size = 660802, upload-time = "2026-06-26T18:32:26.081Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/5b/41/35d1c678cdb3c3b9e6bee691728e563cfb294202b23c7a4c3c2ccc343589/greenlet-3.5.3-cp315-cp315t-manylinux_2_39_riscv64.whl", hash = "sha256:4399eb8d041f20b68d943918bc55502a93d6fdc0a37c14da7881c04139acee9d", size = 498803, upload-time = "2026-06-26T19:25:43.063Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/eb/2e/5303eb3fa06bca089060f479707182a93e360683bc252acf846c3090d34e/greenlet-3.5.3-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:b363d46ed1ea431825fdb01471bb024fc08399bad1572a616e853c7684415adb", size = 1622157, upload-time = "2026-06-26T19:09:09.527Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/54/70/50de47a488f14df260b50ae34fb5d56016e308b098eab02c878b5223c26a/greenlet-3.5.3-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:e44da2f5bbdaabaf7d80b73dbb430c7035771e9f244e3c8b769715c9d8fa0a16", size = 1681159, upload-time = "2026-06-26T18:31:52.986Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a7/13/1055e1dda7882073eda533e2b96c62e55bbd2db7fda6d5ece992febc7071/greenlet-3.5.3-cp315-cp315t-win_amd64.whl", hash = "sha256:8ff8bed3e3baa20a3ea261ce00526f1898ad4801d4886fd2220580ee0ad8fadf", size = 244007, upload-time = "2026-06-26T18:22:04.353Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b4/0d/ca7d15afbdc397e3401134c9e1800d51d12b829661786187a4ad08fe484f/greenlet-3.5.3-cp315-cp315t-win_arm64.whl", hash = "sha256:b7068bd09f761f3f5b4d214c2bed063186b2a86148c740b3873e3f56d79bac31", size = 242586, upload-time = "2026-06-26T18:23:37.93Z" },
]
[[package]] [[package]]
name = "html5lib" name = "html5lib"
version = "1.1" version = "1.1"
@@ -1534,6 +1624,25 @@ wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/bc/60/5382c03e1970de634027cee8e1b7d39776b778b81812aaf45b694dfe9e28/pillow-12.2.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:bfa9c230d2fe991bed5318a5f119bd6780cda2915cca595393649fc118ab895e", size = 7080946, upload-time = "2026-04-01T14:46:11.734Z" }, { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/bc/60/5382c03e1970de634027cee8e1b7d39776b778b81812aaf45b694dfe9e28/pillow-12.2.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:bfa9c230d2fe991bed5318a5f119bd6780cda2915cca595393649fc118ab895e", size = 7080946, upload-time = "2026-04-01T14:46:11.734Z" },
] ]
[[package]]
name = "playwright"
version = "1.61.0"
source = { registry = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/" }
dependencies = [
{ name = "greenlet" },
{ name = "pyee" },
]
wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/44/ee/31e4e0db36588b817a10b299a0285082545fde7d36543c2abe498bb3d61a/playwright-1.61.0-py3-none-macosx_10_13_x86_64.whl", hash = "sha256:ff138c3a604f69911e9d42fd036e55c2a171e5616edf04c1e7f60a2a285540b0", size = 43421877, upload-time = "2026-06-29T10:32:48.428Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/42/35/71395dd3ecc798965be4a3ef8c443217d4abca168e7cb34536304f9489e6/playwright-1.61.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:009588c2a7e499bc5a8b425b61fa65490968bbda9cd69e0cf2cff10f8304659a", size = 42205016, upload-time = "2026-06-29T10:32:52.104Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/f4/44/323164cf5cd1647bdefce76ffce27651aadb959d089b48f53ea40918276e/playwright-1.61.0-py3-none-macosx_11_0_universal2.whl", hash = "sha256:9f7de4536088d12037c13a52b7ea34b59270b78926bb56935070597ffac6b1af", size = 43421884, upload-time = "2026-06-29T10:32:55.773Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ab/f8/a35bf179e4ba2522c1893635094a64e407572547bd61528820fc0abc87fe/playwright-1.61.0-py3-none-manylinux1_x86_64.whl", hash = "sha256:54f3b39f6eab832e33458c1dd7da0b5682aedab3b09ae731b5c59fa12fd2024e", size = 47421381, upload-time = "2026-06-29T10:32:59.903Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b7/eb/e3f922348ec17c315f98c463f72faa1181a1c3de0bfe31a8d2edf6561723/playwright-1.61.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:93454322ade8c11d5d6c211bfd91bdfb9ffb4810e3e026371bcbc4bec1b7ee4c", size = 47120545, upload-time = "2026-06-29T10:33:03.574Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c2/a6/5be4e52b40a9c0c8a073e7c5b0785c05cf5a9ea8f8a7b5b260e32d970342/playwright-1.61.0-py3-none-win32.whl", hash = "sha256:372d55a6f1248fa1dd47599686980cb8fb5bbe6fcda59eab793eb657c11d8a9b", size = 37844841, upload-time = "2026-06-29T10:33:07.361Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/6c/fd/2b78036e5fbe9d5f5645bbe08a1eac7160c51243c0093963edbcf67c35d9/playwright-1.61.0-py3-none-win_amd64.whl", hash = "sha256:35c6cc4589a5d00964a59d7b3e59641e0aac0c02f15479a7af77d20f6bc79597", size = 37844846, upload-time = "2026-06-29T10:33:10.637Z" },
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/27/0d/1b0f3c4ee4eb0514bc805b5c2f9a223e5b6de4f11a926f5235d51d0fc81b/playwright-1.61.0-py3-none-win_arm64.whl", hash = "sha256:e9fcbffcf557a8620fdedd92491eb59a32d18e23d6f3b4f6214b952be324fe51", size = 33955127, upload-time = "2026-06-29T10:33:14.008Z" },
]
[[package]] [[package]]
name = "pluggy" name = "pluggy"
version = "1.6.0" version = "1.6.0"
@@ -1618,6 +1727,18 @@ wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" }, { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" },
] ]
[[package]]
name = "pyee"
version = "13.0.1"
source = { registry = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/" }
dependencies = [
{ name = "typing-extensions" },
]
sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/8b/04/e7c1fe4dc78a6fdbfd6c337b1c3732ff543b8a397683ab38378447baa331/pyee-13.0.1.tar.gz", hash = "sha256:0b931f7c14535667ed4c7e0d531716368715e860b988770fc7eb8578d1f67fc8", size = 31655, upload-time = "2026-02-14T21:12:28.044Z" }
wheels = [
{ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/a0/c4/b4d4827c93ef43c01f599ef31453ccc1c132b353284fc6c87d535c233129/pyee-13.0.1-py3-none-any.whl", hash = "sha256:af2f8fede4171ef667dfded53f96e2ed0d6e6bd7ee3bb46437f77e3b57689228", size = 15659, upload-time = "2026-02-14T21:12:26.263Z" },
]
[[package]] [[package]]
name = "pygments" name = "pygments"
version = "2.20.0" version = "2.20.0"