chinese-equity-quant/docs/joinquant_comparison_plugin.md

# JoinQuant Comparison Plugin

## Why a Plugin

JoinQuant is an external execution and simulation reference. Keeping this code
under `plugins/joinquant/` prevents vendor-specific assumptions from entering
`pipeline/portfolio/`, where the internal reference simulator remains the
canonical implementation.

## What It Validates

The comparison is for system correctness:

- date alignment
- internal to JoinQuant symbol mapping
- target position generation
- once-per-day open execution timing
- lot rounding and filled shares
- position carry
- trading cost
- PnL accounting
- blocked trades from suspension, limit-up, and limit-down conditions

## What It Does Not Validate

It does not validate alpha quality, IC, IR, forecast skill, or whether the
strategy is economically useful. Differences can be expected when JoinQuant
uses different fee, slippage, cash, corporate-action, or internal rounding
rules.

## Historical Backtest Workflow

```bash
# 1. Build internal portfolio targets.
uv run python cli.py portfolio build ...

# 2. Export JoinQuant-compatible frozen targets.
uv run python cli.py joinquant export-targets \
  --positions-path portfolio/run1.pq \
  --portfolio-name run1 \
  --mode target_shares \
  --execution-calendar-path data/daily_bars/<universe> \
  --out-dir plugins_output/joinquant/targets

# 3. Generate and copy the wrapper strategy and target files into JoinQuant.
uv run python cli.py joinquant write-wrapper \
  --portfolio-name run1 \
  --mode target_shares \
  --out-path plugins_output/joinquant/wrapper_strategy_run1.py

# 4. Run the JoinQuant backtest or simulated trading job.
# 5. Export JoinQuant fills, positions, and daily PnL to CSV.

# 6. Ingest JoinQuant output.
uv run python cli.py joinquant ingest \
  --portfolio-name run1 \
  --fills-csv path/to/jq_fills.csv \
  --positions-csv path/to/jq_positions.csv \
  --pnl-csv path/to/jq_pnl.csv

# 7. Reconcile.
uv run python cli.py joinquant reconcile \
  --portfolio-name run1 \
  --targets-dir plugins_output/joinquant/targets/run1 \
  --our-fills-path fills/run1.pq \
  --our-positions-path portfolio/run1.pq \
  --our-pnl-path pnl/run1.pq \
  --jq-fills-path plugins_output/joinquant/ingested/run1/fills.pq \
  --jq-positions-path plugins_output/joinquant/ingested/run1/positions.pq \
  --jq-pnl-path plugins_output/joinquant/ingested/run1/pnl.pq
```

## Forward-Testing Workflow

After the T-1 close and after the data update:

```bash
uv run python cli.py portfolio build ...
uv run python cli.py joinquant export-targets \
  --positions-path portfolio/run1.pq \
  --portfolio-name run1 \
  --mode target_shares \
  --execution-calendar-path data/daily_bars/<universe> \
  --start-date T \
  --end-date T
```

Before the T open, upload or expose the frozen target file to JoinQuant. During
the T open, the JoinQuant wrapper reads that file and submits orders, while the
internal simulator should run against the same frozen target. After T close or
after JoinQuant results are available, ingest the JoinQuant CSV files and run
`joinquant reconcile`.

Forward target files must be frozen before execution. Do not regenerate a
target file after observing open or close data for the same trading date. The
exporter writes a snapshot JSON with a SHA-256 hash for this reason and refuses
to overwrite existing target/snapshot files unless `--force` is passed.

For comparisons against the internal simulator, pass
`--execution-calendar-path` to `joinquant export-targets`. The positions file is
dated by construction/signal date, while the simulator executes at the next
available open. The calendar option shifts exported target files to that next
trading date, so JoinQuant reads the same target on the same execution session.

For JoinQuant 模拟盘, the browser automation has two operational phases:

```bash
# Before T open: upload frozen target(s), save strategy, and start/restart 模拟盘.
uv run python cli.py joinquant write-browser-config \
  --out-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
  --strategy-url "https://www.joinquant.com/<your 模拟盘 page>" \
  --flow sim-trade

uv run python cli.py joinquant run-browser-sim \
  --manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
  --config-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
  --storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json \
  --headed

# After T close: download/export JoinQuant fills, positions, and pnl, then
# ingest/reconcile. The same run-browser-sim command can do this if the config
# includes download actions, otherwise use the ingest/reconcile commands.
```

The default simulated-trading template includes selectors for saving the
strategy and clicking simulated-trading controls such as `模拟盘`, `模拟交易`,
`启动`, and `重启`. These selectors are intentionally configurable because the
JoinQuant web UI can differ by account and page version.

## Target-Shares Mode

`target_shares` is the default and preferred correctness mode. The exported
`target_shares` field comes from the internal `position_shares` column produced
by `portfolio build`, because the internal simulator executes that discretized
integer book. The generated wrapper calls:

```python
order_target(jq_symbol, target_shares)
```

This mode makes filled shares, position carry, and blocked trades easiest to
compare.

## Target-Value Mode

`target_value` mode exports `target_value` and `target_weight` from the
portfolio file. The generated wrapper calls:

```python
order_target_value(jq_symbol, target_value)
```

This can be useful for portfolio-level comparisons, but JoinQuant may apply its
own rounding, cash, and lot rules. Differences are often classified as
`JOINQUANT_INTERNAL_ROUNDING`, `LOT_ROUNDING`, or `CASH_CONSTRAINT` depending
on the observed output.

## Symbol Mapping

Internal symbols are converted as follows:

```text
sh600000 -> 600000.XSHG
sh688001 -> 688001.XSHG
sz000001 -> 000001.XSHE
sz300001 -> 300001.XSHE
```

Reverse mapping is also supported. Invalid exchanges or unsupported A-share
prefixes raise `ValueError` instead of silently guessing.

## Wrapper Strategy Usage

Generate a configured wrapper:

```bash
uv run python cli.py joinquant write-wrapper \
  --portfolio-name run1 \
  --mode target_shares \
  --out-path plugins_output/joinquant/wrapper_strategy_run1.py
```

Copy the generated file and daily CSV target files into JoinQuant. The default
loader uses JoinQuant `read_file`, which works for uploaded files. If your
JoinQuant runtime allows HTTP or another storage backend, replace only
`_read_target_file()` in the generated strategy.

The wrapper is long-only by default:

```python
ALLOW_SHORT = False
```

Negative targets are clipped to zero and logged. Use `--allow-short` only if
the target JoinQuant account supports the required shorting mechanics.

## Ingesting JoinQuant Outputs

The ingest command accepts permissive CSV column names and writes strict plugin
schemas:

```text
plugins_output/joinquant/ingested/{portfolio_name}/fills.pq
plugins_output/joinquant/ingested/{portfolio_name}/positions.pq
plugins_output/joinquant/ingested/{portfolio_name}/pnl.pq
```

Missing cost fields default to zero. Missing blocked status defaults to zero.
Symbols and dates are normalized.

## Reading Reconciliation Reports

The reconcile command writes:

```text
plugins_output/joinquant/reconcile/{portfolio_name}/daily_reconcile.pq
plugins_output/joinquant/reconcile/{portfolio_name}/summary.csv
plugins_output/joinquant/reconcile/{portfolio_name}/summary.md
```

`daily_reconcile.pq` is per-symbol and includes target shares, internal filled
shares, JoinQuant filled shares, realized positions, trade prices, costs, PnL,
and a `diff_reason`. `summary.csv` is the daily portfolio-level view for gross
exposure, net exposure, cash, total value, PnL, cumulative PnL, turnover, and
cost.

Difference reasons include:

```text
MATCH SYMBOL_MAPPING PRICE_MISMATCH LOT_ROUNDING SUSPENSION LIMIT_UP_BLOCK
LIMIT_DOWN_BLOCK VOLUME_OR_LIQUIDITY COST_MODEL CASH_CONSTRAINT
SHORT_NOT_SUPPORTED CORPORATE_ACTION JOINQUANT_INTERNAL_ROUNDING
MISSING_IN_OUR_SYSTEM MISSING_IN_JOINQUANT UNKNOWN
```

Default tolerances are exact share matching, `1e-4` relative trade-price
tolerance, and value tolerance `max(1 yuan, 1e-6 * booksize)`. PnL tolerance is
configurable with `--pnl-tolerance`.

## Minimal Example

Create a 5-stock equal-weight or fixed-share test portfolio:

```text
sh600000, sz000001, sh600519, sz002594, sz300750
```

Build positions for a small date range, export `target_shares`, upload the CSV
files and wrapper to JoinQuant, run the JoinQuant backtest, export fills,
positions, and PnL, then run ingest and reconcile. Start with one or two days
before expanding the sample.

For the first one-stock long-only smoke test, the local side can be prepared in
one command:

```bash
uv run python cli.py joinquant prepare-smoke \
  --out-dir /tmp/chinese-equity-quant-realdata
```

The command downloads a tiny public daily-bar sample, builds a fixed-share
`sh600000` long-only position file, simulates it internally, exports aligned
JoinQuant target files, writes a configured wrapper strategy, and creates
`joinquant_smoke_manifest.json` with all output paths.

## Browser Backtest Automation

JoinQuant's public `jqdatasdk` is a data SDK. It supports authenticated data
calls such as `auth(username, password)` and `get_price(...)`, but cloud
strategy upload, backtest execution, and result export are web-application
workflows. The plugin therefore automates those remote steps through Playwright
with a saved browser session.

Install the optional browser runner in the uv environment:

```bash
uv sync --extra joinquant-browser
uv run playwright install chromium
```

Save a reusable login state. This opens a browser; log in normally, including
any CAPTCHA or 2FA, then press Enter in the terminal to save state:

```bash
uv run python cli.py joinquant browser-login \
  --storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json
```

Create a selector/action config for a historical backtest:

```bash
uv run python cli.py joinquant write-browser-config \
  --out-path /tmp/chinese-equity-quant-realdata/joinquant_browser_config.json \
  --strategy-url "https://www.joinquant.com/<your strategy page>" \
  --flow backtest
```

If selectors need tuning, capture the logged-in page:

```bash
uv run python cli.py joinquant browser-snapshot \
  --url "https://www.joinquant.com/<your strategy page>" \
  --out-dir /tmp/chinese-equity-quant-realdata/browser_snapshot
```

Then run the remote backtest automation:

```bash
uv run python cli.py joinquant run-browser-backtest \
  --manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
  --config-path /tmp/chinese-equity-quant-realdata/joinquant_browser_config.json \
  --storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json \
  --headed
```

The config is declarative: actions can navigate, paste the generated wrapper,
upload all target CSV files, fill dates, click run, wait for completion,
download result CSVs, and take screenshots. When the configured downloads
produce fills, positions, and PnL CSVs, the runner automatically calls
`joinquant ingest` and `joinquant reconcile`.

For forward testing / 模拟盘, create the config with `--flow sim-trade` and run:

```bash
uv run python cli.py joinquant run-browser-sim \
  --manifest-path /tmp/chinese-equity-quant-realdata/joinquant_smoke_manifest.json \
  --config-path /tmp/chinese-equity-quant-realdata/joinquant_sim_config.json \
  --storage-state ~/.config/chinese-equity-quant/joinquant_storage_state.json \
  --headed
```

Do not store raw JoinQuant passwords in this repository. The browser state file
is created with `0600` permissions and should live under `~/.config`, outside
the repo.

## Recommended First Sanity Checks

1. One liquid stock with a fixed target share count.
2. A 10-stock equal-weight long-only portfolio.
3. A forced suspension, limit-up, and limit-down sample.
4. A short target in long-only mode to confirm `SHORT_NOT_SUPPORTED`.
5. A 5-day reversal portfolio after the mechanical checks pass.

## Known Limitations

- JoinQuant internal execution details may differ from the reference simulator.
- External file loading depends on the JoinQuant environment.
- Short selling may not be supported.
- Fee, tax, slippage, and minimum-fee models may differ.
- Corporate actions may need special handling and should not be hidden.
- The internal simulator does not currently emit execution price in
  `FILL_COLUMNS`; price reconciliation uses explicit price columns if supplied.