Module-level ATR_SL_MULT was 1.5, now 2.0 to match config.py and CLI defaults.
This ensures generate_dataset_vectorized() produces correct labels even without
explicit parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- MLFilter.from_model() classmethod eliminates brittle __new__() private-attribute
manipulation in backtester walk-forward model injection
- backtest_validator._check_invariants() now accepts cfg and uses cfg.initial_balance
instead of a hardcoded 1000.0 for the negative-balance invariant check
- backtester.py walk-forward injection block simplified to use the new factory method
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add normalize=False parameter to MLXFilter.fit() so external callers
can skip internal normalization. Remove the external normalization +
manual _mean/_std reset hack from walk_forward_auc() in train_mlx_model.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add atr_sl_mult and atr_tp_mult parameters to _calc_labels_vectorized
and generate_dataset_vectorized, defaulting to existing constants (1.5,
2.0) for full backward compatibility. Callers (train scripts, backtester)
can now pass symbol-specific multipliers without modifying module-level
constants.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
C5: Remove duplicate entry_fee deduction in backtester (balance and net_pnl)
C1: Add SL/TP retry (3x) with emergency market close on final failure
C3: Add _close_lock to prevent PnL double recording between callback and monitor
C8: Add SIGTERM/SIGINT handler with per-symbol order cancellation before exit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(data_stream): add reconnect loop to MultiSymbolStream matching UserDataStream pattern
Prevents bot-wide crash on WebSocket disconnect (#3 Critical)
- fix(data_stream): increase buffer_size 200→300 and preload 200→300
Ensures z-score window (288) has sufficient data (#5 Important)
- fix(bot): sync risk manager when Binance has no position but local state does
Prevents ghost entries in open_positions blocking future trades (#1 Critical)
- fix(ml_filter): add np.nan_to_num for ONNX input to handle NaN features
Prevents all signals being blocked during initial ~2h warmup (#2 Critical)
- fix(bot): replace _close_handled_by_sync with current_trade_side==None guard
Eliminates race window in SYNC PnL double recording (#4 Important)
- feat(bot): add _ensure_sl_tp_orders in _recover_position
Detects and re-places missing SL/TP orders on bot restart (#6 Important)
- feat(exchange): add get_open_orders method for SL/TP verification
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(bot): prevent PnL double recording in _close_and_reenter using asyncio.Event
- fix(bot): prevent SYNC detection PnL duplication with _close_handled_by_sync flag
- fix(notifier): move sync HTTP call to background thread via run_in_executor
- fix(risk_manager): make is_trading_allowed async with lock for thread safety
- fix(exchange): cache exchange info at class level (1 API call for all symbols)
- fix(exchange): use `is not None` instead of truthy check for price/stop_price
- refactor(backtester): extract _calc_trade_stats to eliminate code duplication
- fix(ml_features): apply rolling z-score to OI/funding rate in serving (train-serve skew)
- fix(bot): use config.correlation_symbols instead of hardcoded BTCUSDT/ETHUSDT
- fix(bot): expand OI/funding history deque to 96 for z-score window
- cleanup(config): remove unused stop_loss_pct, take_profit_pct, trailing_stop_pct fields
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace full file scan with _tail_lines() that reads from EOF
- Only load max(FAST_KILL=8, SLOW_KILL=15) lines on boot
- Trim in-memory trade history to prevent unbounded growth
- No I/O bottleneck regardless of history file size
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fast Kill: 8 consecutive net losses → block new entries for symbol
- Slow Kill: last 15 trades PF < 0.75 → block new entries for symbol
- Trade history persisted to data/trade_history/{symbol}.jsonl (survives restart)
- Boot-time retrospective check restores kill state from history
- Manual reset via RESET_KILL_SWITCH_{SYMBOL}=True in .env + restart
- Entry blocked, exits (SL/TP/manual) always work normally
- Discord alert on kill switch activation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add payoff_ratio and max_consecutive_losses to backtester summary
- Add compare_symbols.py: per-symbol parameter sweep for candidate evaluation
- Add position_sizing_analysis.py: robust Monte Carlo position sizing
- Fetch historical data for SOL, LINK, AVAX candidates (365 days)
- Update existing symbol data (XRP, TRX, DOGE)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MARKET orders can fill in multiple trades (PARTIALLY_FILLED → FILLED).
Previously only the last fill's rp/commission was captured, causing
under-reported PnL. Now accumulates rp and commission across all
partial fills per order_id and sums them on final FILLED event.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5. Add daily PnL reset loop — UTC midnight auto-reset via
_daily_reset_loop in main.py, prevents stale daily_pnl accumulation
6. Fix set_base_balance race condition — call once in main.py before
spawning bots, instead of each bot calling independently
7. Remove realized_pnl != 0 from close detection — prevents entry
orders with small rp values being misclassified as closes
8. Rename xrp_btc_rs/xrp_eth_rs → primary_btc_rs/primary_eth_rs —
generic column names for multi-symbol support (dataset_builder,
ml_features, and tests updated consistently)
9. Replace asyncio.get_event_loop() → get_running_loop() — fixes
DeprecationWarning on Python 3.10+
10. Parallelize candle preload — asyncio.gather for all symbols
instead of sequential REST calls, ~3x faster startup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Margin ratio calculated on per_symbol_balance instead of total balance
— previously amplified margin reduction by num_symbols factor
2. Replace Algo Order API (algoType=CONDITIONAL) with standard
futures_create_order for SL/TP — algo API is for VP/TWAP, not
conditional orders; SL/TP may have silently failed
3. Fallback PnL (SYNC close) now sums all recent income rows instead
of using only the last entry — prevents daily_pnl corruption in
multi-fill scenarios
4. Explicit state transition in _close_and_reenter — clear local
position state after close order to prevent race with User Data
Stream callback on position count
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the position monitor detects a missed close via API fallback, it
now queries Binance futures_income_history to get the real realized PnL
and commission instead of logging zeros. Exit price is estimated from
entry price + PnL/quantity. This ensures the dashboard records accurate
profit data even when WebSocket events are missed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The position monitor now checks Binance API every 5 minutes to verify
the bot's internal state matches the actual position. If the bot thinks
a position is open but Binance shows none, it syncs state and logs a
SYNC close event. This prevents the bot from being stuck in a phantom
position when User Data Stream misses a TP/SL fill event.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hardcoded round(qty, 1) caused -1111 Precision errors for TRXUSDT and
DOGEUSDT (stepSize=1, requires integers). Now lazily loads quantityPrecision
and pricePrecision from Binance futures_exchange_info per symbol. SL/TP
prices also use symbol-specific precision instead of hardcoded 4 decimals.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- #1: OI division by zero — already fixed (prev_oi == 0.0 guard exists)
- #2: cumulative trade count used max() instead of sum(), breaking ML trigger
- #4: fetch_history API calls now retry 3x with exponential backoff
- #5: parquet upsert now deduplicates timestamps before sort
- #6: record_pnl() is now async with Lock for multi-symbol safety
- #8: exit_price=0.0 skips close handling with warning log
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_signal() now returns (signal, detail) tuple with long/short scores,
ADX value, volume surge status, and HOLD reason for easier diagnosis.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WF backtester always passed trained models to Backtester.run(ml_models=...),
overriding ml_filters even when use_ml=False. This caused 0 trades in
--no-ml mode because underfitted models (trained on ~27 samples) blocked
all entries with proba < 0.55 threshold.
- Skip model training when use_ml=False (saves computation)
- Only inject ml_models when use_ml=True
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix LightGBM predict_proba ValueError by filtering FEATURE_COLS and casting to float64
- Extract BTC/ETH correlation data from embedded parquet columns instead of missing separate files
- Disable ONNX priority in ML filter tests to use mocked LightGBM correctly
- Add NO_ML_FILTER=true to .env.example (ML adds no value with current signal thresholds)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add [SYMBOL] prefix to all bot/user_data_stream log messages
- Rewrite log_parser.py with multi-symbol regex, per-symbol state tracking, symbol columns in DB schema
- Rewrite dashboard_api.py with /api/symbols endpoint, symbol query params on all endpoints, SQL injection fix
- Update App.jsx with symbol filter tabs, multi-position display, dynamic header
- Add tests for log parser (8 tests) and dashboard API (7 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Include RSI, MACD_H, ATR in bot entry log so the log parser can
extract and store them in the trades DB for dashboard display
- Update log parser regex and _handle_entry() to persist indicator values
- Add dashboard section to README (tech stack, screens, API endpoints)
- Add dashboard/ directory to project structure in README
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Introduced a new trading dashboard consisting of a FastAPI backend (`dashboard-api`) for data retrieval and a React frontend (`dashboard-ui`) for visualization.
- Implemented a log parser to monitor and store bot logs in an SQLite database.
- Configured Docker setup for both API and UI, including necessary Dockerfiles and a docker-compose configuration.
- Added setup documentation for running the dashboard and accessing its features.
- Enhanced the Jenkins pipeline to build and push the new dashboard images.
- Added .worktrees/ to .gitignore to prevent tracking of worktree files.
- Marked `optuna-precision-objective-plan` as completed in CLAUDE.md.
- Added new training log entry for a LightGBM model with updated parameters and performance metrics in training_log.json.
- Updated error handling in ml_filter.py to return False on prediction errors instead of True, improving the robustness of the ML filter.
Add two new OI-derived features to improve ML model's market microstructure
understanding:
- oi_change_ma5: 5-candle moving average of OI change rate (short-term trend)
- oi_price_spread: z-scored OI minus z-scored price return (divergence signal)
Both features use 96-candle rolling z-score window. FEATURE_COLS expanded from
24 to 26. Existing tests updated to reflect new feature counts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Migrate ADX from hard filter (ADX < 25 blocks entry) to ML feature so
the model can learn optimal ADX thresholds from data. Updates FEATURE_COLS,
build_features(), and corresponding tests from 23 to 24 features.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Log current price and unrealized PnL every 5 minutes while holding a position,
using the existing kline WebSocket's unclosed candle data for real-time price updates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added ML_THRESHOLD to .env.example and updated Config class to include ml_threshold with a default value of 0.55.
- Modified MLFilter initialization in bot.py to utilize the new ml_threshold configuration.
- Updated Jenkinsfile to change the registry URL for Docker image management.
These changes enhance the model's adaptability by allowing for a configurable machine learning threshold, improving overall performance.
The two HOLD negative tests (test_hold_negative_labels_are_all_zero,
test_signal_samples_preserved_after_sampling) were passing vacuously
because sample_df produces 0 signal candles (ADX ~18, below threshold
25). Added signal_producing_df fixture with higher volatility and volume
surges to reliably generate signals. Removed if-guards so assertions
are mandatory. Also restored the full docstring for
generate_dataset_vectorized() documenting btc_df/eth_df,
time_weight_decay, and negative_ratio parameters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add negative_ratio parameter to generate_dataset_vectorized() that
samples HOLD candles as label=0 negatives alongside signal candles.
This increases training data from ~535 to ~3,200 samples when enabled.
- Split valid_rows into base_valid (shared) and sig_valid (signal-only)
- Add 'source' column ("signal" vs "hold_negative") for traceability
- HOLD samples get label=0 and random 50/50 side assignment
- Default negative_ratio=0 preserves backward compatibility
- Fix incorrect column count assertion in existing test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ADX < 25 now returns HOLD in get_signal(), preventing entries during
trendless (sideways) markets. NaN ADX values fall through to existing
weighted signal logic. Also syncs the vectorized dataset builder with
the same ADX filter to keep training data consistent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ADX (Average Directional Index) with period 14 to calculate_all()
for sideways market filtering. Includes test verifying the adx column
exists and contains non-negative values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added a new entry to the training log with detailed metrics for a LightGBM model, including AUC, precision, recall, and tuned parameters.
- Enhanced the MLFilter class to include a guard clause that prevents execution if the filter is disabled, improving robustness.
