The number one question I get from readers is how to find systematic trading strategies. Most people ask “How can I compete with the big hedge funds?” They think if an idea exists, there’s no money left to be made.

That’s not exactly true.

Developing strategies is the first step to trading. Without an idea that captures market edge, at best you’ll make money by getting lucky.

More likely, you’ll lose it.

All beginners start the same: with data mining technical indicators. Systems that rely on data mining overfit to random market noise. They look great in backtests but fail in real life.

There’s a better way.

10 profitable trading ideas for an advantage (even if you’re new)

A trading system based on data mining looks for patterns in past price data and fits a model to them. Market forces that cause these patterns are not considered. The only assumption is that the patterns of the past will repeat in the future. This is where machine learning comes in handy.

Few successful trading systems are built through data mining.

Model-based systems start with a model of a market inefficiency. Inefficiencies can be based on trader psychology, economics, market microstructure, company events, or anything else that affects the price. These inefficiencies cause patterns that deviate from the normal randomness of the market.

Most successful algorithmic trading systems are built by modeling market inefficiencies.

Inefficiency #1: Trend strategies

Price momentum is the tendency for prices to move in the same direction. The classic method of trend following is the moving average cross-over. Since trend following is easy to detect, it’s not easy to make money. Moving average cross-overs frequently fail because they can’t distinguish between real momentum and random noise.

The problem is that it’s difficult to identify a true trend. A random price series that appears to be trending can still have zero momentum that you can consistently exploit. That’s why traders use filters that detect the market regimes. You can use low-pass filters or the Hurst exponent to detect if a market is entering or exiting a trending market regime.

Inefficiency #2: Mean reversion strategies

A mean reverting price series is one that tends to revert to some unknown “fair value” consistently. If the price gets “too high” or “too low,” it will eventually revert back to the fair value.

Strategies that exploit this inefficiency seek to model the fair value and determine when a price has got too low or too high. The system will buy when the price is too low and sell when it returns to fair value. Similarly, it will sell when the price is too high and buy when it returns to fair value. You can use the Hurst exponent to detect mean reverting markets.

The edge in these strategies is modeling the “true” fair value and correctly predicting when the price is too low or too high.

Inefficiency #3: Statistical arbitrage strategies

Strategies can exploit the similarity, and occasional dislocation, of moves between two or more assets. In the most simple case, one asset is bought and the other is sold. The ratio of the purchase and sale is based on the historic relationship between the two assets. One acts as a hedge for the other. This concentrates exposure not on the assets themselves, but on the relationship between them.

A popular statistical arbitrage strategy is pairs trading.

Inefficiency #4: Cyclical strategies

Non-seasonal cycles are caused by an internal feedback loop from historic prices. When traders target a price they often buy or sell the asset when the price gets “too far” from that target price hoping it will revert back. Or they close winning or losing positions when a trend begins to fade. These effects cause a large number of traders to do the same thing. This in turn causes the price to swing in a similar way many times.

Quants use signal processing techniques like fast Fourier transformations to decompose these movements and model them. Once they’re modeled, they can be predicted and traded.

Inefficiency #5: Cluster strategies

The same thing that causes prices to swing in cycles can cause them to cluster at certain levels. At the extreme, these clusters cause support and resistance. Any beginner can draw lines on a chart to identify support and resistance. But, just because you can draw a line on a chart does not mean it’s a consistent, exploitable inefficiency.

Still, line or no line, clusters do exist and can be detected by signal processing techniques.

Inefficiency #6: Chart pattern strategies

Trader psychology is an inefficiency that can be exploited. Chart patterns arise from the repetitive behavior of traders. The herd mentality of seeing a pattern form on a chart is a self-fulfilling prophecy. There is no economic rationale behind chart patterns so strategies reliant on them are fragile. However, they can be identified with methods like the Fréchet distance.

Market microstructure can also play a role in chart patterns. The most common is violent moves after long periods of little price action. These moves are caused by traders’ tendency to place stop losses at similar prices. Once the stops are triggered, the price can move quickly. This effect can be exploited by a system that detects side wise price action and waits for sudden moves.

Inefficiency #7: Seasonality strategies

The supply and demand of assets can follow time-based patterns every day, week, or month. For example, asset managers rotate into high-quality assets toward the end of the month to avoid reporting losers.

Since these types of inefficiencies are easy to exploit, they are usually short lived, messy, and hard to detect manually. That’s why using Minimum Viable Python is important to assess whether there’s an edge before spending a lot of time writing backtests.

Inefficiency #8: Gap strategies

Like any professional, traders contemplate the work day after they shut down the trading screens. Unlike other professionals, traders tend to reach similar conclusions about what to do next. This concensus can cause prices to spike or “gap” up or down at the market open when everyone reaches the same conclusion. It’s a well-known trader axiom that “gaps tend to fill.”

It’s not uncommon for prices to trend in the direction of the gap for a short time. Systems that identify price gaps (e.g. today’s open – yesterday’s close > X%) enter positions and close them when the gap is filled.

Inefficiency #9: Time series strategies

Two time series models you will encounter are ARIMA and GARCH. They assume that future returns or volatility can be predicted with a linear combination of past observations. Modeling asset returns is messy because financial time series are non-stationary. That means the mean and variance (and associated probability distribution) vary over time. Because time series analyses are dependent on a stationary probability distribution, assumptions are violated, models are insignificant, and dreams are broken.

That said, strategies using time series analysis can work. (The linked article uses R but is still worth a read.)

Inefficiency #10: Price shock strategies

When companies report earnings misses or economic data is reported outside analyst expectations, prices move. Even without knowing the news, strategies can detect early signs of price moves and join the momentum. Relative strength indicators can help here for lower frequency, intraday strategies.