Simple past, future perfect

Algorithmic trading is playing an ever-growing role on energy exchanges, helping market participants to deal with the intricacy of the intraday markets. And the more heavily you rely on trading automation, the more critical thorough testing of your trading algorithms becomes. For many companies taking steps into the world of algotrading, proper design of backtesting (or its lesser known cousin) represents a significant challenge.

Algo testing: What’s the big deal?

Trading algorithms are developed and tested to the same careful standards as any other code. But just as a new racecar first has to be proven through countless rounds on a test track before it’s allowed to compete, before you take your algorithm live on the market with real money and real products, you must validate that it will perform as expected in real-world market situations.

Algorithm testing is thus done using some form of market data. In this way, potential unexpected behaviors can be identified and corrected. Testing on market data also allows you to benchmark financial performance so you can compare with other trading strategies. There are two major methods to test algorithms: backtesting and agent-based modelling of the market.

Reliving the past: Backtesting

In the case of backtesting, trading algorithms are tested with recorded historical market data. Market liquidity and the algorithm's own possible influence on the market need to be taken into consideration, as historical prices alone are not sufficient to execute any backtesting procedure. Rather, the complete historical order book data is required. And data alone is not enough; the exact matching algorithm of the exchange must also be simulated in order to create a virtual market.

There are many benefits of this kind of testing, especially since the exact behavior of the algorithm can be tested in different market situations, allowing you to fine tune the algo. You can test it in a number of historical situations: for instance when wind production was higher or lower than expected, or when a major power plant unexpectedly went offline. After a few rounds of adjustment, the trading algorithm can be deployed for live operation. 

On the other hand, the energy market changes drastically, therefore this historical data may already be out of date. Changes in regulation such as market coupling, or the ongoing shift towards variable renewable generation, have had a significant influence on market behavior and will continue to do so. You wouldn’t want to predict today’s algo behavior based on pre-XBID/SIDC market data, for instance, and future changes will at some point render today’s data obsolete. However, even though backtesting results do not perfectly reflect future performance of the algorithm, the results are still valuable as they present some measurement of its success or failure. 

Creating the future: Market simulation

The second option to apply testing on trading algorithms is agent-based market simulation. In this case, instead of using historical data, software agents generate a theoretical market based on predefined rules. Actions of human and algorithmic traders can be simulated, with different goals, strategies and reaction times, for instance:

Market maker: Plays neutrally against the market, just like the real market makers on energy exchanges.

Liquidity taker: Reduces liquidity by posting market orders that are likely to be immediately executed at the best available price. Used to increase volatility.

Liquidity provider: Increases liquidity by placing buy and sell limit orders without immediately triggering a trade. Used to increase the depth and stability of the market. 

The actions of the algorithm can be tested in this simulated environment. Furthermore, this feature allows the users to enter a range for the specified input, from which the simulation can select the best input to be used. Considering that any scenarios are configurable, this testing method makes it possible to debug the algorithm in real time through simulated market situations that would not be possible with backtesting due to the lack of historical data. Potential future scenarios can be tested such as various outage situations, abundant production of electricity from renewable sources, or low-demand situations in which certain conventional energy sources cannot operate below a technical minimum.

Naturally, there is a disadvantage to this method as well: performance. This type of simulation requires lots of resources and plenty of time. Moreover, this kind of testing optimization can lead to an algorithm that performs poorly in live operation. When this happens, the algorithm is called “over-optimized”; it’s too focused on a specific, narrow situation, or too complex to apply in real-life market situations. 

Designing your optimal test strategy

Which testing method is best? It depends on the goals of the strategy you plan to test. You know the historical data as well as the events that triggered specific market behavior. If the situations you need to test against can be adequately represented by historical data, then backtesting is probably the best choice. If, on the other hand, you need to test behavior in rare situations that are not well-represented in historical data, then agent-based market simulation is the better approach. 

In either case, it’s no easy task to set up a suitable test environment with the proper data and parameters to meet your specific needs. Fortunately you don’t have to do it alone. Whether through backtesting or agent-based simulation, we can test your trading algorithms for you, so you can concentrate your efforts on identifying trading signals and developing trading strategies. Contact us to learn more about our algorithm testing service.