Trying to build an algorithmic trading strategy without a process is like walking through the jungle without a map.

You’re bound to get lost.

Unfortunately, most people think they have a process to form strategies. In reality, it goes something more like this:

1. Read about a brand-new indicator on Reddit
2. Find the code for the brand-new indicator
3. Copy-and-paste it into a backtest framework
4. Tweak the parameters until the results are good
5. Trade the system live, lose money, and give up

Or maybe they have a process but rely on data mining to find strategies.

Consider a more rigorous approach

Algorithmic trading systems start with a model of a market inefficiency. Inefficiencies can be based on trader psychology, economics, market microstructure, company events, or anything else that affects the price. These inefficiencies cause patterns that deviate from the normal randomness of the market. Sometimes, these patterns repeat and can be detected, predicted, and traded.

It's your job to detect, predict, and trade these patterns. Without a step-by-step framework to do it, you’re stuck in the jungle. In today’s newsletter, I’m going to share the basics.

It’s a framework I’ve used for years. It’s also something I teach (in a lot more detail) in Getting Started With Python for Quant Finance and is one of the most popular lessons in the course.

Ready?

Step 1: The idea

Start with one of the market inefficiencies described above. Then observe the market, read, and take part in forums. Look for market anomalies that repeat. Or, form a hypothesis that might cause repeated anomalies and check for it in the price history.

Your strategy should be based on some economic rationale. In other words, “using machine learning” is not a trading strategy.

Once you think you find an anomaly, attempt to model it so you can generate trade signals.

Step 2: Research

Find out if the anomaly you think you found actually exists in history. For this you need data. Can you get access to the data you need cheaply? It’s unlikely you can build a backtest for a barrier option strategy because you do not have the data.

Consider how far back you should look and how frequently the anomaly might occur. If you're modeling price reactions to covid news, you don't need data from 2010. Otherwise, you want to look as far back as possible.

Use Minimum Viable Python to figure out how frequently the inefficiency appears and under what market conditions. If you find no evidence of the inefficiency or no significant difference from random data, iterate on your detection method or start over.

Step 3: Trade signals

This step is the "algorithmic" part of algorithmic trading. You need to generate trade signals to tell your system when to buy and sell. The inefficiencies will be weak so the algorithm must be good at distinguishing it from noise (use a filter).

Do your best to make the algorithm as simple as possible. The more parameters it has, the higher the risk of overfitting to random noise and not capturing the true inefficiency.

Step 4: Minimum Viable Python

If you made it this far, assess the strategy for an edge. Details like transaction costs, slippage, and other "real-life" considerations don't matter. Performance doesn't even matter. You're only trying to determine if the algorithm has an edge or not. If it does not at least generate a series of profitable trades, improve the algorithm or try another one.

You're not backtesting the strategy in this step, you're assessing if it has an edge. The algorithm must be able to produce positive results with only basic exits.

Step 5: The backtest setup

Since you made it to this step, it means your strategy has evidence of an edge. It's time to build the backtest.

The backtest lets you optimize the parameters, add elements that will affect performance, and run out of sample tests. Now is the time to add your commission and slippage model. Backtests are not easy to get right so don’t build your own—use an existing framework that is event-driven, detailed, and fast.

Be careful when you optimize the parameters. Focus on those that are dependent on the market including things like trade entry and exit points. (Not lookback windows and filter constants.) Most beginners use brute force optimization to find the combination of parameters that existed at the peak profit.

Don’t.

Instead, optimizing the backtest should be focused on robustness (i.e. how small changes in parameters impact performance).

Step 6: The backtest analysis

At this point your strategy has an edge, your model generates signals, the backtest is set up, reflects reality, is optimized, and is completely worthless.

To get a closer reflection of true performance, there are two more things to do: run out-of-sample tests and check the validity of the backtest.

Out-of-sample tests are common in all data sciences. The state-of-the-art method of doing this for backtests is a rolling walk-forward optimization (WFO). A WFO analysis uses a rolling window and repeats the training and testing process many times.

The second thing you need to do is check the validity of the backtest. There's a good chance your backtest will generate a good result and fail in real life.

The best way to do it is to randomize the historic prices, retrain the model, test it again, and record a performance metric (e.g. Sharpe ratio, profit factor, etc). Then repeat the process 1,000 times and plot the distribution of the results. Since you've completely eliminated the market inefficiency by randomizing the prices, you should see poor performance. Finally, plot the performance metric from the real historic prices.

If the performance metric lies near the center of the distribution, start over.

Step 7: Risk and performance analysis

Check risk metrics like drawdown, volatility, and VaR. Since active trading takes time, your results should be significantly better than a simple buy-and-hold strategy.

Since the trading system survived the scrutiny of this strategy formation process, you can focus on reducing risk and improving performance. To do this, optimize the strategy's exit criteria. Instead of the simple exit rules you started with during strategy development, try different stops.

For example, if you have a winning trade, automatically enter a stop after your break-even point. The first rule of trading is don't let winners turn into losers. This enforces that rule.

Remember to re-optimize and test the strategy after making changes. If the performance doesn't improve, try again.

Step 8: Execution

When you're finally ready to trade the system, there are three steps to take.

Trading is hard enough. Dealing with technology issues adds a new layer of complexity that you do not want to deal with if you’re losing money. Plus, you want to make sure the strategy behaves like you expect it to behave.

So before hooking up your code to your broker, let the strategy run and manually trade the signals it generates.

Next, it's time to hook up the strategy to a paper trading account. Find a broker who lets you automate your execution in a paper trading account. That way you can test whether the technology is working without risking your money.

Let it run while you manually trade. The results should be the same!

Finally, move to live execution. Make sure you have a way to control trade volume during the trading day. Build a way to lock in profits or exit all open positions in the case of bad news. Make it easy to retrain the model and update parameters in real-time. Most importantly, give yourself a way to compare the backtest results with the live results.

That way you can make sure you modeled the market inefficiency and that it actually exist

The truth is, algorithmic trading is hard. But without a map, it’s impossible.