- mtf_bot: ExecutionManager saves entry/exit records to
data/trade_history/mtf_{symbol}.jsonl on every close
- Jenkinsfile: split MTF_CHANGED from BOT_CHANGED so mtf_bot.py-only
changes restart mtf-bot service without touching cointrader
- docker-compose: mount ./data:/app/data on mtf-bot for persistence
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- README: add MTF bot to feature list and project structure
- ARCHITECTURE: add section 5-1 with MTF bot architecture, data flow,
and design principles (4-module structure, 250 candle buffer, etc.)
- CLAUDE.md: add mtf-pullback-bot to plan history table
- mtf_bot.py: fix stale comment (maxlen=200 → 250)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
deque(maxlen=200) + [:-1] slice left only 199 completed candles,
causing EMA 200 to return NaN and the bot to stay in WAIT forever.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Loop sleep 30s → 1s to never miss the 4-second TimeframeSync window
- Data polling remains at 30s intervals via monotonic timer
- Force poll before signal check to ensure fresh data
- Add [Heartbeat] log every 15m with Meta/ATR/Close/Position
- HOLD reasons now logged at INFO level (was DEBUG)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sends alerts for: bot start, virtual entry (LONG/SHORT with SL/TP),
and SL/TP exits with PnL. Uses existing DiscordNotifier via
DISCORD_WEBHOOK_URL from .env. Also added env_file to mtf-bot
docker-compose service.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- exchange.py: cancel_all_orders() now cancels both standard and algo orders
- exchange.py: get_open_orders() merges standard + algo orders
- exchange.py: cancel_order() falls back to algo cancel on failure
- bot.py: store SL/TP prices for price-based close_reason re-determination
- bot.py: add _cancel_remaining_orders() for orphan SL/TP cleanup
- bot.py: re-classify MANUAL close_reason as SL/TP via price comparison
- bot.py: cancel orphan orders on startup when no position exists
- tests: fix env setup for testnet config and ML filter mocking
- docs: add backtest market context and algo order fix design specs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add BINANCE_TESTNET env var to switch between live/demo API keys
- Add KLINE_INTERVAL env var (default 15m) for configurable candle interval
- Pass testnet flag through to Exchange, DataStream, UDS, Notifier
- Add demo mode in bot: forced LONG entry with fixed 0.5% SL / 2% TP
- Fix UDS close_reason: use ot (original order type) field to correctly
classify STOP_MARKET/TAKE_PROFIT_MARKET triggers (was MANUAL)
- Add UDS raw event logging with ot field for debugging
- Add backtest market context (BTC/ETH regime, L/S ratio per fold)
- Separate testnet trade history to data/trade_history/testnet/
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add set_margin_type("ISOLATED") call before each position entry.
With cross margin, a single bad trade's loss draws from the entire
account balance. Isolated margin caps loss to the position's allocated
margin only.
Binance returns -4046 if already ISOLATED, which is silently ignored.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After keepalive ping timeout, the existing BinanceSocketManager's
listenKey becomes invalid. Reusing it causes silent failure — the
WebSocket appears connected but receives no ORDER_TRADE_UPDATE events.
This caused SL/TP fills to be detected only by the 5-minute position
monitor polling (SYNC) instead of real-time User Data Stream (TP/SL),
affecting 11 of 12 production trades.
Fix: create fresh AsyncClient + BinanceSocketManager on every reconnect
iteration, ensuring a valid listenKey is obtained each time.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds Fast Kill (8 consecutive losses) and Slow Kill (PF < 0.75 over 15 trades)
to the backtester, matching bot.py behavior. Without this, ML OFF overtrades
and self-destructs, making ML ON look artificially better.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Module-level ATR_SL_MULT was 1.5, now 2.0 to match config.py and CLI defaults.
This ensures generate_dataset_vectorized() produces correct labels even without
explicit parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- MLFilter.from_model() classmethod eliminates brittle __new__() private-attribute
manipulation in backtester walk-forward model injection
- backtest_validator._check_invariants() now accepts cfg and uses cfg.initial_balance
instead of a hardcoded 1000.0 for the negative-balance invariant check
- backtester.py walk-forward injection block simplified to use the new factory method
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add normalize=False parameter to MLXFilter.fit() so external callers
can skip internal normalization. Remove the external normalization +
manual _mean/_std reset hack from walk_forward_auc() in train_mlx_model.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add atr_sl_mult and atr_tp_mult parameters to _calc_labels_vectorized
and generate_dataset_vectorized, defaulting to existing constants (1.5,
2.0) for full backward compatibility. Callers (train scripts, backtester)
can now pass symbol-specific multipliers without modifying module-level
constants.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
C5: Remove duplicate entry_fee deduction in backtester (balance and net_pnl)
C1: Add SL/TP retry (3x) with emergency market close on final failure
C3: Add _close_lock to prevent PnL double recording between callback and monitor
C8: Add SIGTERM/SIGINT handler with per-symbol order cancellation before exit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(data_stream): add reconnect loop to MultiSymbolStream matching UserDataStream pattern
Prevents bot-wide crash on WebSocket disconnect (#3 Critical)
- fix(data_stream): increase buffer_size 200→300 and preload 200→300
Ensures z-score window (288) has sufficient data (#5 Important)
- fix(bot): sync risk manager when Binance has no position but local state does
Prevents ghost entries in open_positions blocking future trades (#1 Critical)
- fix(ml_filter): add np.nan_to_num for ONNX input to handle NaN features
Prevents all signals being blocked during initial ~2h warmup (#2 Critical)
- fix(bot): replace _close_handled_by_sync with current_trade_side==None guard
Eliminates race window in SYNC PnL double recording (#4 Important)
- feat(bot): add _ensure_sl_tp_orders in _recover_position
Detects and re-places missing SL/TP orders on bot restart (#6 Important)
- feat(exchange): add get_open_orders method for SL/TP verification
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- fix(bot): prevent PnL double recording in _close_and_reenter using asyncio.Event
- fix(bot): prevent SYNC detection PnL duplication with _close_handled_by_sync flag
- fix(notifier): move sync HTTP call to background thread via run_in_executor
- fix(risk_manager): make is_trading_allowed async with lock for thread safety
- fix(exchange): cache exchange info at class level (1 API call for all symbols)
- fix(exchange): use `is not None` instead of truthy check for price/stop_price
- refactor(backtester): extract _calc_trade_stats to eliminate code duplication
- fix(ml_features): apply rolling z-score to OI/funding rate in serving (train-serve skew)
- fix(bot): use config.correlation_symbols instead of hardcoded BTCUSDT/ETHUSDT
- fix(bot): expand OI/funding history deque to 96 for z-score window
- cleanup(config): remove unused stop_loss_pct, take_profit_pct, trailing_stop_pct fields
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace full file scan with _tail_lines() that reads from EOF
- Only load max(FAST_KILL=8, SLOW_KILL=15) lines on boot
- Trim in-memory trade history to prevent unbounded growth
- No I/O bottleneck regardless of history file size
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fast Kill: 8 consecutive net losses → block new entries for symbol
- Slow Kill: last 15 trades PF < 0.75 → block new entries for symbol
- Trade history persisted to data/trade_history/{symbol}.jsonl (survives restart)
- Boot-time retrospective check restores kill state from history
- Manual reset via RESET_KILL_SWITCH_{SYMBOL}=True in .env + restart
- Entry blocked, exits (SL/TP/manual) always work normally
- Discord alert on kill switch activation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add payoff_ratio and max_consecutive_losses to backtester summary
- Add compare_symbols.py: per-symbol parameter sweep for candidate evaluation
- Add position_sizing_analysis.py: robust Monte Carlo position sizing
- Fetch historical data for SOL, LINK, AVAX candidates (365 days)
- Update existing symbol data (XRP, TRX, DOGE)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MARKET orders can fill in multiple trades (PARTIALLY_FILLED → FILLED).
Previously only the last fill's rp/commission was captured, causing
under-reported PnL. Now accumulates rp and commission across all
partial fills per order_id and sums them on final FILLED event.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5. Add daily PnL reset loop — UTC midnight auto-reset via
_daily_reset_loop in main.py, prevents stale daily_pnl accumulation
6. Fix set_base_balance race condition — call once in main.py before
spawning bots, instead of each bot calling independently
7. Remove realized_pnl != 0 from close detection — prevents entry
orders with small rp values being misclassified as closes
8. Rename xrp_btc_rs/xrp_eth_rs → primary_btc_rs/primary_eth_rs —
generic column names for multi-symbol support (dataset_builder,
ml_features, and tests updated consistently)
9. Replace asyncio.get_event_loop() → get_running_loop() — fixes
DeprecationWarning on Python 3.10+
10. Parallelize candle preload — asyncio.gather for all symbols
instead of sequential REST calls, ~3x faster startup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Margin ratio calculated on per_symbol_balance instead of total balance
— previously amplified margin reduction by num_symbols factor
2. Replace Algo Order API (algoType=CONDITIONAL) with standard
futures_create_order for SL/TP — algo API is for VP/TWAP, not
conditional orders; SL/TP may have silently failed
3. Fallback PnL (SYNC close) now sums all recent income rows instead
of using only the last entry — prevents daily_pnl corruption in
multi-fill scenarios
4. Explicit state transition in _close_and_reenter — clear local
position state after close order to prevent race with User Data
Stream callback on position count
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the position monitor detects a missed close via API fallback, it
now queries Binance futures_income_history to get the real realized PnL
and commission instead of logging zeros. Exit price is estimated from
entry price + PnL/quantity. This ensures the dashboard records accurate
profit data even when WebSocket events are missed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The position monitor now checks Binance API every 5 minutes to verify
the bot's internal state matches the actual position. If the bot thinks
a position is open but Binance shows none, it syncs state and logs a
SYNC close event. This prevents the bot from being stuck in a phantom
position when User Data Stream misses a TP/SL fill event.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hardcoded round(qty, 1) caused -1111 Precision errors for TRXUSDT and
DOGEUSDT (stepSize=1, requires integers). Now lazily loads quantityPrecision
and pricePrecision from Binance futures_exchange_info per symbol. SL/TP
prices also use symbol-specific precision instead of hardcoded 4 decimals.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- #1: OI division by zero — already fixed (prev_oi == 0.0 guard exists)
- #2: cumulative trade count used max() instead of sum(), breaking ML trigger
- #4: fetch_history API calls now retry 3x with exponential backoff
- #5: parquet upsert now deduplicates timestamps before sort
- #6: record_pnl() is now async with Lock for multi-symbol safety
- #8: exit_price=0.0 skips close handling with warning log
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_signal() now returns (signal, detail) tuple with long/short scores,
ADX value, volume surge status, and HOLD reason for easier diagnosis.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WF backtester always passed trained models to Backtester.run(ml_models=...),
overriding ml_filters even when use_ml=False. This caused 0 trades in
--no-ml mode because underfitted models (trained on ~27 samples) blocked
all entries with proba < 0.55 threshold.
- Skip model training when use_ml=False (saves computation)
- Only inject ml_models when use_ml=True
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix LightGBM predict_proba ValueError by filtering FEATURE_COLS and casting to float64
- Extract BTC/ETH correlation data from embedded parquet columns instead of missing separate files
- Disable ONNX priority in ML filter tests to use mocked LightGBM correctly
- Add NO_ML_FILTER=true to .env.example (ML adds no value with current signal thresholds)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add [SYMBOL] prefix to all bot/user_data_stream log messages
- Rewrite log_parser.py with multi-symbol regex, per-symbol state tracking, symbol columns in DB schema
- Rewrite dashboard_api.py with /api/symbols endpoint, symbol query params on all endpoints, SQL injection fix
- Update App.jsx with symbol filter tabs, multi-position display, dynamic header
- Add tests for log parser (8 tests) and dashboard API (7 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Include RSI, MACD_H, ATR in bot entry log so the log parser can
extract and store them in the trades DB for dashboard display
- Update log parser regex and _handle_entry() to persist indicator values
- Add dashboard section to README (tech stack, screens, API endpoints)
- Add dashboard/ directory to project structure in README
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Introduced a new trading dashboard consisting of a FastAPI backend (`dashboard-api`) for data retrieval and a React frontend (`dashboard-ui`) for visualization.
- Implemented a log parser to monitor and store bot logs in an SQLite database.
- Configured Docker setup for both API and UI, including necessary Dockerfiles and a docker-compose configuration.
- Added setup documentation for running the dashboard and accessing its features.
- Enhanced the Jenkins pipeline to build and push the new dashboard images.
- Added .worktrees/ to .gitignore to prevent tracking of worktree files.
- Marked `optuna-precision-objective-plan` as completed in CLAUDE.md.
- Added new training log entry for a LightGBM model with updated parameters and performance metrics in training_log.json.
- Updated error handling in ml_filter.py to return False on prediction errors instead of True, improving the robustness of the ML filter.
