How to backtest an AI trading strategy before going live

Backtesting an AI trading strategy means simulating its decisions against historical market data to find flaws before it risks real capital. A valid backtest requires clean data, fixed rules, conservative cost assumptions, and a strict separation between training data and test data. If the strategy survives honest out-of-sample testing and paper trading, it may be ready for a small live trial with tight limits. If it fails any of these checks, it should not be deployed.

What is backtesting and why does it matter for AI agents?

Backtesting is the process of simulating a trading strategy against historical market data to estimate how it might have performed in the past. For an AI agent, this step is not optional. It is the primary way to discover whether the agent’s decision logic, entry conditions, and risk rules behave sensibly before any real money is exposed to market risk. The purpose of backtesting is not to generate an impressive equity curve that you can use to justify deployment. The purpose is to act as a rejection mechanism. A backtest should make it easy to identify strategies that are broken, overly risky, or dependent on perfect foresight. If a strategy cannot survive a well-constructed backtest, it is almost certainly not ready for live trading. Even if it does survive, the backtest only proves that the strategy was not immediately disqualified. Markets change, and past performance does not predict future results. This is especially true for agentic systems that operate across multiple market types through a single API. An agent that trades stocks, crypto, perps, options, or prediction markets needs to know whether its intended position sizes, entry frequencies, and holding periods are compatible with the historical behavior of each asset class. Without backtesting, you are simply hoping the agent avoids catastrophic errors in production. Hope is not a risk control.

What data do you need before you start?

You need historical price data that precisely matches the instruments and markets your agent will target. If the agent is designed to trade stocks, you need stock price history that includes corporate actions. If it targets crypto or perpetual futures, you need the relevant history for those markets. If it targets options or prediction markets, you need data that reflects the unique structure of those instruments. The data must be free of survivorship bias. Survivorship bias occurs when you only include assets that are still trading today, ignoring delisted stocks, dead crypto tokens, or expired prediction market contracts. A strategy tested only on survivors will look artificially profitable because the worst performers have been erased from the record. You also need data granularity that matches your intended trading frequency. A strategy that holds positions for weeks can be tested on daily closing prices. A strategy that enters and exits within hours needs intraday bars or tick data. Before you run any simulation, inspect the data for missing timestamps, bad prints, corporate splits, dividends, and irregular gaps. Clean the data carefully, because errors in historical data will produce errors in your conclusions. Store the data in a versioned format so you can reproduce any backtest result later. When you compare backtest results to paper trading, you will need to know exactly which data version produced which outcome.

How should you define entry and exit rules?

Every trading strategy needs concrete, testable rules for when to open and close positions. Write these rules in plain language before you write any simulation code. If you cannot explain the strategy in two or three sentences, the agent does not have a strategy. It has a vague idea. Entry rules should specify the exact condition that triggers a position. This could be a price crossing a moving average, a volatility spike, a signal from a machine learning model, or any other quantifiable event. Exit rules should specify when to close the position, whether that is a profit target, a stop loss, a time limit, or a reversal of the entry signal. Avoid ambiguous instructions such as "sell when momentum fades" or "reduce exposure if the market looks shaky." An AI agent cannot interpret subjective language. It needs numbers, thresholds, and logical conditions. Once you define the rules, treat them as fixed during the backtest. Do not change them after seeing the results. If you notice the strategy would have performed better with a different parameter and then rerun the test, you are no longer backtesting. You are optimizing, and that optimization must be done on separate data. The discipline of locking rules before testing is what separates a valid backtest from a curve-fitting exercise. If the rules cannot be expressed as deterministic logic, they cannot be backtested at all.

What costs and slippage should you include?

A backtest that ignores transaction costs is not a simulation. It is a work of fiction. Every order you place incurs fees, and every fill occurs at a price that may differ from the price you assumed. You must model these costs conservatively. Different market types have different cost structures. A stock broker may charge commissions or use payment for order flow. A crypto venue may have maker-taker fees. A perps venue charges funding rates. An options venue has wide bid-ask spreads and assignment risk. A prediction market may have fees and resolution delays. You do not need exact historical fee schedules for a first backtest, but you should use estimates that are equal to or worse than what you expect in live trading. For slippage, assume that market orders fill at a price slightly worse than the last quoted price. The larger the position relative to typical volume, the more slippage you should assume. If your strategy relies on frequent small trades, costs will compound quickly and can turn a nominally profitable strategy into a losing one. Include the cost of crossing the spread. Include funding for leveraged positions. If you use stop orders, assume they slip during fast moves. When in doubt, add a penalty to every trade. If the strategy still shows a reasonable risk-adjusted return after these penalties, it may be worth testing further. If the profits disappear, the strategy was never robust.

How do you avoid curve fitting and overfitting?

Curve fitting is the process of tailoring a strategy so precisely to historical data that it captures noise rather than signal. Overfitting is a similar risk when the AI model itself learns patterns that do not generalize. Both problems produce backtests that look spectacular and perform terribly in live trading. The primary defense is a strict separation between in-sample and out-of-sample data. Use the in-sample data to develop and tune the strategy. Use the out-of-sample data only once, at the end, to evaluate the final rules. If you iterate on the strategy after peeking at out-of-sample results, you have contaminated the test. Another approach is walk-forward analysis. In this method, you optimize on a past window, test on the next window, then roll forward and repeat. This mimics the process of adapting to new data without hindsight bias. Keep the number of free parameters small. A strategy with many adjustable thresholds is easier to overfit than one with a few simple rules. Be especially skeptical of strategies that show smooth equity curves with almost no drawdowns. Real markets contain randomness, and a backtest with too-perfect results often indicates a look-ahead bias or a data leak. Ask whether the strategy requires information that would not have been available at the time of the trade. If the agent needs to know the closing price of a bar to place an order inside that same bar, the backtest is using future information. That is a guaranteed way to generate fake profits.

What metrics actually matter for a beginner?

Beginners often focus on total return because it is the simplest number to understand. This is a mistake. Total return ignores risk, luck, and the path taken to achieve the result. A more useful starting point is the maximum drawdown, which measures the largest peak-to-trough decline in the account balance. This tells you how bad the worst period would have felt. Next, look at risk-adjusted returns, such as the Sharpe ratio or the Sortino ratio. These attempt to measure return per unit of risk, though they are not perfect and should not be treated as guarantees. Examine the win rate, but only alongside the average winner and average loser. A strategy that wins only forty percent of the time can still be profitable if its average win is much larger than its average loss. Count the total number of trades. A backtest with fewer than thirty trades is not statistically meaningful. You cannot distinguish skill from luck with a handful of observations. Look at the time spent in the market. A strategy that is always exposed may simply be a leveraged version of buy-and-hold, and its returns may come from market beta rather than alpha. Finally, verify that the strategy respects the same safety limits you would impose on a live agent, including sensible position sizing step by step and spend caps and drawdown limits. If the backtest shows violations, the live agent will violate them too.

How do you move from backtest to paper trading?

A backtest that passes all your checks is still a simulation. The next mandatory step is paper trading, where the agent connects to live market data but executes against a simulated balance. This tests whether the strategy logic works when data arrives in real time, whether order sizing maps correctly through the API, and whether the agent handles delays or rejections gracefully. Before you start paper trading, freeze the strategy code and document the exact version, the data set used for the backtest, and the key metrics you observed. This creates a benchmark. If paper trading results diverge significantly from the backtest, you have a bug, a look-ahead bias, or a cost assumption that was too optimistic. Do not authorize live trading until the paper trading curve aligns with the backtest in both shape and magnitude. Even after alignment, treat live trading as an experiment. Start with a small budget cap, strict position limits, and an active kill switch. Trading can lose money, including the entire budget, so the initial live phase should expose the least capital possible. Review what beginners get wrong about autonomous trading before you enable a live key. The transition from backtest to paper to live is a pipeline designed to catch errors at the cheapest possible stage.

The exact request schema is in the docs; the shape looks like this:

curl -X POST https://api.felix.trade/simulation \
  -H "Authorization: Bearer YOUR_KEY" \
  -d '{"mode": "paper", "budget_cap_usd": 500}'