Nightly resets, restart times, and recovery

There is no universal fixed restart time

IBKR performs daily regional server maintenance. The window depends on the region, day, and current IBKR schedule; it can change. Read the live IBKR system status and do not hard-code a time copied from documentation. TWS/IB Gateway can also be configured for an application auto-restart/auto-logoff time, which is a separate local setting.

Plan to lose upstream connectivity at least once per day. During the server reset, login and order management may be unavailable. Native exchange-held orders can continue working; IBKR-simulated orders and execution reports may be delayed. Do not operate intentionally in the maintenance window.

Connectivity codes

Code	Meaning	Required application action
1100	TWS/IBG lost connectivity to IBKR (maintenance, internet, or competing session)	Enter degraded mode; stop new dependent actions; do not assume the local socket is gone or working orders are cancelled.
1101	Upstream restored; data lost	Re-submit market-data and account/data subscriptions, then reconcile orders/executions.
1102	Upstream restored; data maintained	Resume cautiously; do not duplicate subscriptions blindly, but reconcile critical order state.
1300	TWS socket port was reset and connection dropped	Re-read/reconfigure the port and reconnect.
2103/2105	Market-data/HMDS farm disconnected	Pause affected data assumptions; TWS usually reconnects farms automatically.
2104/2106/2158	Farm connection OK	Informational; not evidence that every subscription was replayed.
2110	TWS-to-server connectivity broken; automatic restore expected	Treat like upstream degradation and await subsequent connectivity codes.

Two failure planes

1. Python-to-TWS socket failure: disconnectedEvent, EOF, application restart, port change, duplicate client ID, or TWS process exit. Reconnect the socket and rebuild all local synchronized state.
2. TWS-to-IBKR upstream failure: the local socket may remain connected while code 1100 reports loss. TWS normally reconnects upstream. React to 1101 versus 1102 rather than repeatedly restarting the local process.

Restarting TWS on every 1100 can make maintenance behavior worse and can interrupt simulated-order visibility. ib_async.Watchdog does restart on errors {100, 1100} by design; understand that policy before using it in production.

Recovery state machine

READY
  -> DEGRADED on 1100/farm loss/staleness
  -> RECONNECTING on socket loss
  -> SYNCHRONIZING after connect or 1101
  -> RECONCILING orders, executions, positions
  -> RESUBSCRIBING market data when required
  -> READY only after invariants pass

Use exponential backoff with jitter for refused socket connections. A fixed aggressive loop can trigger duplicate client-ID races and overload a starting gateway. Ensure only one process owns a chosen client ID.

ib_async.Watchdog

Watchdog(controller, ib, host, port, clientId, connectTimeout, appStartupTime, appTimeout, retryDelay, readonly, account, raiseSyncErrors, probeContract, probeTimeout) controls an IBC-launched application. After appTimeout seconds without traffic it sends a 30-second/5-second-bar historical midpoint probe. No bars before probeTimeout is considered hard failure; it terminates and restarts the controlled app after retryDelay.

Important edge cases:

• a quiet healthy session triggers probes;
• a probe can fail from pacing, entitlement, farm maintenance, contract issues, or market-data conditions—not only a dead gateway;
• the default probe is EURUSD and may not fit every account/environment;
• Watchdog restarts on message-rate error 100 and connectivity error 1100;
• it expects event-driven reinitialization on every connectedEvent;
• it is not intended for notebooks or imperative scripts.

For container orchestration, it is often better to separate process liveness from IBKR upstream readiness: keep the service alive during known maintenance, expose degraded readiness, and let a single connection manager own reconciliation.

Auto restart versus auto logoff

Recent TWS/IB Gateway versions support automatic restart on most days, but IBKR still requires periodic full reauthentication (commonly weekly) and may require 2FA after logout. Exact UI options/version behavior change. IBC's on2fatimeout can choose restart or exit when authentication times out; it cannot bypass IBKR authentication requirements.

Store no plaintext credentials in source or command lines. Prefer secured IBC configuration/secret injection and restrict filesystem/process-list exposure.

Official sources: system message codes and IBKR Campus connectivity guidance.