How to backtest AI trading strategies before using real money

Backtesting simulates an AI trading strategy on historical market data to find logical errors and estimate behavior before any real capital is deployed. It does not predict future profits or guarantee that the strategy will outperform. It is a filtering step, not a promise. The goal is to discover whether the agent does what you intended, and whether that action would have been destructive under past conditions.

What does backtesting actually do?

Backtesting runs a strategy against past price data to see how it would have traded. For AI agents, this means replaying historical candles, tick data, or market events through the agent's decision logic. The goal is to catch bugs: a condition that never triggers, a sign error in position sizing, a loop that spams orders, or a misinterpretation of a timestamp. It also reveals whether the strategy's logic is coherent. Suppose an agent is instructed to buy when sentiment is positive and volatility is low. Backtesting shows whether those conditions ever overlapped in the past, and what the resulting trades looked like.

The output is usually a set of descriptive metrics: number of trades, average win, average loss, maximum drawdown, win rate, and total return. These numbers describe the past. They do not describe the future. Markets change. Liquidity shifts. Correlations break. A strategy that worked in a trending market may fail in a ranging one. Backtesting only tells you that the agent did what you told it to do under old conditions, and what the financial result of those actions would have been.

For AI agents, there is an extra layer of approximation. The agent might use an LLM to interpret prompts, parse news, or evaluate market data. Backtesting cannot fully replicate the LLM's non-deterministic outputs unless every inference is recorded and replayed exactly. Most backtests use simplified logic or cached responses. This is a gap. The simulation is an approximation of the agent, not a perfect clone. If the agent's behavior depends on nuanced language interpretation, the backtest may be especially optimistic.

Developers should also check whether the backtest engine matches the live API semantics. If the live API rejects orders below a certain size, or requires rounding to a specific tick size, the backtest should do the same. Otherwise the agent will encounter errors it never saw in simulation. Normalizing orders in plain US dollars helps, but the backtest still needs to know how those dollars map to contracts or shares at the time of each historical bar.

Why do AI agents need backtesting differently than human traders?

Human traders backtest to validate a thesis. AI agents backtest to validate implementation. An agent does not have intuition. It follows instructions literally. A human might notice that a signal feels wrong and pause. An agent will execute until stopped. This means backtesting for agents must be stricter about edge cases and failure modes.

Consider slippage. A human trader might mentally adjust for the fact that a large order moves the market, or that a thin order book cannot absorb the size. An agent might not, unless explicitly coded. Backtesting should include conservative slippage assumptions and delay assumptions. If the backtest assumes perfect fills at the mid-price, the real-world results will likely be worse. The agent does not know the difference between a backtest fantasy fill and a live market bid.

Another difference is speed. Agents can evaluate signals and submit orders in milliseconds. In backtesting, this speed is often modeled as instant. In reality, network latency, API rate limits, and matching engine delays matter. A backtest that looks profitable with instant execution may lose money when realistic latency is introduced. The path from signal to order is longer than zero in the real world.

Agents also trade across multiple market types. A strategy might involve stocks and perps, or options and prediction markets. Each venue has different margin rules, fee structures, minimum tick sizes, and settlement times. Backtesting must account for these variables. A unified API normalizes order sizing in plain US dollars, but the backtest still needs to model venue-specific costs. Otherwise the simulation is fiction. How AI agents misread order books step by step discusses execution risks that backtests rarely capture because they assume the agent sees the same liquidity that the backtest engine sees.

How do paper trading and backtesting differ?

Backtesting uses historical data. Paper trading uses live data with simulated money. Both are essential, but they test different things. Backtesting is fast. You can run a year of data in minutes. Paper trading is slow. You wait for real markets to move, and you see how the agent behaves when the future is unknown.

Paper trading reveals integration bugs. Does the agent correctly parse the live API response? Does it handle rate limits? Does it recover from a timeout or a malformed payload? Backtesting cannot test these because the data is already clean and local. Paper trading also shows how the agent behaves when nothing happens. A backtest moves from signal to signal. Paper trading includes long periods of boredom where the agent might drift, retry unnecessarily, or misinterpret stale data.

Felix offers paper trading for testing. When the agent connects through MCP or the REST API, it can route orders to a simulation environment. Live trading requires explicit owner authorization of a key. This two-step process, backtest then paper trade, creates a buffer between idea and capital. A practical checklist for non-custodial trading beginners includes steps for moving from simulation to live keys safely, including verifying that your scoped permissions and budget caps are active before any real money is at risk.

The gap between paper and live trading is often underestimated. Paper trading may fill orders at the last traded price, while live trading fills at the available bid or ask. Paper trading may not model partial fills, rejections, or price impact. It is a better test than backtesting, but it is still not real money. The emotional and financial reality of losses only appears when capital is actually deployed.

What makes backtesting unreliable for future results?

Backtesting suffers from several well-known failures. Overfitting is the most common. An agent with many parameters, or an LLM with a long prompt full of specific rules, can be tuned to fit past noise. It looks perfect in the backtest and fails immediately in live trading. To reduce this, some builders use walk-forward analysis: train on one segment, test on the next, and never reuse test data for training. If the agent is too complex relative to the number of trades, it is probably overfit.

Look-ahead bias is another risk. If the backtest accidentally uses information that was not available at the time of the trade, the results are invalid. This happens easily with data timestamps. A human might not notice that a signal relies on a value published five minutes after the trade. The agent will not notice either. The backtest engine must be carefully audited to ensure that each bar or tick only knows what was known at that moment.

Survivorship bias matters for stock strategies. If the backtest only includes companies that still exist today, it ignores delisted failures. The strategy looks better than it is because the losers are gone. The database must include dead tickers, expired contracts, and delisted assets. Data quality is another subtle problem. Bad ticks, erroneous prints, and missing bars can make a strategy look brilliant or terrible. If the backtest relies on free or unadjusted data, the results may be artifacts of the data feed rather than the strategy. Always use clean, adjusted data and understand how dividends, splits, and funding rates are handled.

Market impact is hard to model. A backtest assumes the agent's trades do not move prices. For small orders this is reasonable. For large orders or thin markets, it is not. An agent trading prediction markets or options might be a significant fraction of volume. The backtest will not reflect this. The agent might also face different liquidity at different times of day.

Finally, regime change invalidates historical patterns. A strategy built on low volatility data may collapse when volatility spikes. A strategy that shorts meme stocks may work until it does not. Backtesting cannot warn you about events that have never happened before. Black swans, regulatory changes, and technological shocks are invisible to historical data. Trading can lose money, including everything, and backtesting does not change that risk. It only gives you a false sense of history.

How should you connect backtesting to live safety controls?

A backtest is only useful if it informs the controls you deploy with real money. The maximum drawdown seen in backtesting should be smaller than the kill switch threshold you set live. If the backtest shows a ten percent drawdown, setting a five percent live kill switch is reasonable. Setting a fifty percent kill switch because the backtest was profitable is not. The backtest's worst case is a floor, not a ceiling.

Budget caps should be lower than the backtest's total capital. Suppose the backtest simulated one hundred thousand dollars. The live agent should start with a small fraction of that. Position limits should be stricter. The agent should not be allowed to take concentrated risks just because the backtest survived them. The backtest had no emotional pressure, no API downtime, and no slippage surprises.

Safety controls include scoped keys, budget caps, position limits, exit plans, and a panic switch. How to build a kill switch your trading agent cannot override explains how to flatten positions and revoke access automatically when thresholds are breached. These controls are not optional extras. They are the translation of backtest caution into live practice.

The architecture that keeps agents secure is designed around the idea that no test is perfect. The architecture that keeps AI trading agents secure in 2026 describes how non-custodial design and scoped permissions limit damage when a strategy behaves differently than expected. Because funds sit in a wallet the owner controls, and withdrawal addresses are owner-approved only, the agent can spend within limits but can never withdraw to itself or steal. This means even if the backtest was wrong and the live strategy bleeds money, the bleed is bounded by the limits you set.

When moving from backtest to live, authorize the key explicitly, set hard limits tighter than the backtest, and monitor for the first few sessions. The backtest is a hypothesis. Live trading is the experiment. Treat it accordingly.