- Add _remove_incomplete_candle() for timestamp-based conditional
slicing on both 15m and 1h data (replaces hardcoded [:-1])
- Add MetaFilter indicator caching to eliminate 3x duplicate calc
- Fix notifier encapsulation (_send → notify_info public API)
- Remove DataFetcher.poll_update() dead code
- Fix evaluate_oos.py symbol typo (xrpusdtusdt → xrpusdt)
- Add 20 pytest unit tests for MetaFilter, TriggerStrategy,
ExecutionManager, and _remove_incomplete_candle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- exchange.py: cancel_all_orders() now cancels both standard and algo orders
- exchange.py: get_open_orders() merges standard + algo orders
- exchange.py: cancel_order() falls back to algo cancel on failure
- bot.py: store SL/TP prices for price-based close_reason re-determination
- bot.py: add _cancel_remaining_orders() for orphan SL/TP cleanup
- bot.py: re-classify MANUAL close_reason as SL/TP via price comparison
- bot.py: cancel orphan orders on startup when no position exists
- tests: fix env setup for testnet config and ML filter mocking
- docs: add backtest market context and algo order fix design specs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add LOOKAHEAD embargo between train/val splits in all 3 WF functions
to prevent label leakage from 6h lookahead window
- Add --ablation flag to train_model.py for signal_strength/side
dependency diagnosis (A/B/C experiment with drop analysis)
- Criteria: A→C drop ≤0.05=good, 0.05-0.10=conditional, ≥0.10=redesign
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Module-level ATR_SL_MULT was 1.5, now 2.0 to match config.py and CLI defaults.
This ensures generate_dataset_vectorized() produces correct labels even without
explicit parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- MLFilter.from_model() classmethod eliminates brittle __new__() private-attribute
manipulation in backtester walk-forward model injection
- backtest_validator._check_invariants() now accepts cfg and uses cfg.initial_balance
instead of a hardcoded 1000.0 for the negative-balance invariant check
- backtester.py walk-forward injection block simplified to use the new factory method
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add normalize=False parameter to MLXFilter.fit() so external callers
can skip internal normalization. Remove the external normalization +
manual _mean/_std reset hack from walk_forward_auc() in train_mlx_model.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add atr_sl_mult and atr_tp_mult parameters to _calc_labels_vectorized
and generate_dataset_vectorized, defaulting to existing constants (1.5,
2.0) for full backward compatibility. Callers (train scripts, backtester)
can now pass symbol-specific multipliers without modifying module-level
constants.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
C1: /api/reset에 API key 인증 추가 (DASHBOARD_RESET_KEY 환경변수)
C2: /proc 스캐닝 제거, PID file + SIGHUP 기반 파서 재파싱으로 교체
C3: daily_pnl 업데이트를 trades 테이블에서 재계산하여 idempotent하게 변경
I1: CORS origins를 CORS_ORIGINS 환경변수로 설정 가능하게 변경
I2: offset 파라미터에 ge=0 검증 추가
I3: 매 줄 commit → 파일 단위 배치 commit으로 성능 개선
I4: _pending_candles 크기 제한으로 메모리 누적 방지
I5: bot.log glob 중복 파싱 제거 (sorted(set(...)))
I6: /api/health 에러 메시지에서 내부 경로 미노출
I7: RSI 차트(데이터 없음)를 OI 변화율 차트로 교체
M1: pnlColor 변수 shadowing 수정 (posPnlColor)
M2: 거래 목록에 API total 필드 사용
M3: dashboard/ui/.dockerignore 추가
M4: API Dockerfile Python 3.11→3.12
M5: 테스트 fixture에서 temp DB cleanup 추가
M6: 누락 테스트 9건 추가 (health, daily, reset 인증, offset, pagination)
M7: 파서 SIGTERM graceful shutdown + entrypoint.sh signal forwarding
DB: 양쪽 busy_timeout=5000 + WAL pragma 설정
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(bot): prevent PnL double recording in _close_and_reenter using asyncio.Event
- fix(bot): prevent SYNC detection PnL duplication with _close_handled_by_sync flag
- fix(notifier): move sync HTTP call to background thread via run_in_executor
- fix(risk_manager): make is_trading_allowed async with lock for thread safety
- fix(exchange): cache exchange info at class level (1 API call for all symbols)
- fix(exchange): use `is not None` instead of truthy check for price/stop_price
- refactor(backtester): extract _calc_trade_stats to eliminate code duplication
- fix(ml_features): apply rolling z-score to OI/funding rate in serving (train-serve skew)
- fix(bot): use config.correlation_symbols instead of hardcoded BTCUSDT/ETHUSDT
- fix(bot): expand OI/funding history deque to 96 for z-score window
- cleanup(config): remove unused stop_loss_pct, take_profit_pct, trailing_stop_pct fields
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The combined summary (PF, MDD, win_rate) was indirectly reconstructed
from per-symbol averages using round(win_rate * n), which introduced
rounding errors. MDD was max() of individual symbol MDDs, ignoring
simultaneous drawdowns across the correlated crypto portfolio.
Now computes all combined metrics directly from the trade list:
- PF: sum(wins) / sum(losses) from actual trade PnLs
- MDD: portfolio equity curve from time-sorted trades
- Win rate: direct count from trade PnLs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5. Add daily PnL reset loop — UTC midnight auto-reset via
_daily_reset_loop in main.py, prevents stale daily_pnl accumulation
6. Fix set_base_balance race condition — call once in main.py before
spawning bots, instead of each bot calling independently
7. Remove realized_pnl != 0 from close detection — prevents entry
orders with small rp values being misclassified as closes
8. Rename xrp_btc_rs/xrp_eth_rs → primary_btc_rs/primary_eth_rs —
generic column names for multi-symbol support (dataset_builder,
ml_features, and tests updated consistently)
9. Replace asyncio.get_event_loop() → get_running_loop() — fixes
DeprecationWarning on Python 3.10+
10. Parallelize candle preload — asyncio.gather for all symbols
instead of sequential REST calls, ~3x faster startup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hardcoded round(qty, 1) caused -1111 Precision errors for TRXUSDT and
DOGEUSDT (stepSize=1, requires integers). Now lazily loads quantityPrecision
and pricePrecision from Binance futures_exchange_info per symbol. SL/TP
prices also use symbol-specific precision instead of hardcoded 4 decimals.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_signal() now returns (signal, detail) tuple with long/short scores,
ADX value, volume surge status, and HOLD reason for easier diagnosis.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added functionality to fetch live trade data from the dashboard API.
- Implemented weekly report generation that includes backtest results, live trade statistics, and performance trends.
- Enhanced error handling for API requests and improved logging for better traceability.
- Updated tests to cover new features and ensure reliability of the report generation process.
- Fix LightGBM predict_proba ValueError by filtering FEATURE_COLS and casting to float64
- Extract BTC/ETH correlation data from embedded parquet columns instead of missing separate files
- Disable ONNX priority in ML filter tests to use mocked LightGBM correctly
- Add NO_ML_FILTER=true to .env.example (ML adds no value with current signal thresholds)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add [SYMBOL] prefix to all bot/user_data_stream log messages
- Rewrite log_parser.py with multi-symbol regex, per-symbol state tracking, symbol columns in DB schema
- Rewrite dashboard_api.py with /api/symbols endpoint, symbol query params on all endpoints, SQL injection fix
- Update App.jsx with symbol filter tabs, multi-position display, dynamic header
- Add tests for log parser (8 tests) and dashboard API (7 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add two new OI-derived features to improve ML model's market microstructure
understanding:
- oi_change_ma5: 5-candle moving average of OI change rate (short-term trend)
- oi_price_spread: z-scored OI minus z-scored price return (divergence signal)
Both features use 96-candle rolling z-score window. FEATURE_COLS expanded from
24 to 26. Existing tests updated to reflect new feature counts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Migrate ADX from hard filter (ADX < 25 blocks entry) to ML feature so
the model can learn optimal ADX thresholds from data. Updates FEATURE_COLS,
build_features(), and corresponding tests from 23 to 24 features.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Log current price and unrealized PnL every 5 minutes while holding a position,
using the existing kline WebSocket's unclosed candle data for real-time price updates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Added HOLD candles as negative samples to increase training data from ~535 to ~3,200 samples. Introduced a negative_ratio parameter in generate_dataset_vectorized() for sampling HOLD candles alongside signal candles. Implemented stratified undersampling to ensure signal samples are preserved during training. Updated relevant tests to validate new functionality and maintain compatibility with existing tests.
- Modified dataset_builder.py to include HOLD negative sampling logic
- Updated train_model.py to apply stratified undersampling
- Added tests for new sampling methods
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The two HOLD negative tests (test_hold_negative_labels_are_all_zero,
test_signal_samples_preserved_after_sampling) were passing vacuously
because sample_df produces 0 signal candles (ADX ~18, below threshold
25). Added signal_producing_df fixture with higher volatility and volume
surges to reliably generate signals. Removed if-guards so assertions
are mandatory. Also restored the full docstring for
generate_dataset_vectorized() documenting btc_df/eth_df,
time_weight_decay, and negative_ratio parameters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add negative_ratio parameter to generate_dataset_vectorized() that
samples HOLD candles as label=0 negatives alongside signal candles.
This increases training data from ~535 to ~3,200 samples when enabled.
- Split valid_rows into base_valid (shared) and sig_valid (signal-only)
- Add 'source' column ("signal" vs "hold_negative") for traceability
- HOLD samples get label=0 and random 50/50 side assignment
- Default negative_ratio=0 preserves backward compatibility
- Fix incorrect column count assertion in existing test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ADX < 25 now returns HOLD in get_signal(), preventing entries during
trendless (sideways) markets. NaN ADX values fall through to existing
weighted signal logic. Also syncs the vectorized dataset builder with
the same ADX filter to keep training data consistent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ADX (Average Directional Index) with period 14 to calculate_all()
for sideways market filtering. Includes test verifying the adx column
exists and contains non-negative values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
바이낸스 OI 히스토리 API가 최근 30일치만 제공하는 제약을 우회하기 위해
upsert_parquet() 함수를 추가. 매일 실행 시 기존 parquet의 oi_change/funding_rate가
0.0인 구간만 신규 값으로 덮어써 점진적으로 과거 데이터를 채워나감.
--no-upsert 플래그로 기존 덮어쓰기 동작 유지 가능.
Made-with: Cursor
- _fetch_market_microstructure: oi_val > 0 체크 후에만 _calc_oi_change 호출하여
API 실패(None/Exception) 시 0.0으로 폴백하고 _prev_oi 상태 오염 방지
- README: ML 피처 수 오기재 수정 (25개 → 23개)
- tests: _calc_oi_change 첫 캔들 및 API 실패 시 상태 보존 유닛 테스트 추가
Made-with: Cursor
- Add asyncio import to bot.py
- Add _prev_oi state for OI change rate calculation
- Add _fetch_market_microstructure() for concurrent OI/funding rate fetch with exception fallback
- Add _calc_oi_change() for relative OI change calculation
- Always call build_features() before ML filter check in process_candle()
- Pass oi_change/funding_rate kwargs to build_features() in both process_candle() and _close_and_reenter()
- Update _close_and_reenter() signature to accept oi_change/funding_rate params
Made-with: Cursor
- Updated README.md to reflect new features including dynamic margin ratio, model hot-reload, and multi-symbol streaming.
- Modified bot logic to ensure raw signals are passed to the `_close_and_reenter` method, even when the ML filter is loaded.
- Introduced a new script `run_tests.sh` for streamlined test execution.
- Improved test coverage for signal processing and re-entry logic, ensuring correct behavior under various conditions.
