According to the Cambridge Dictionary, Machine Learning (ML) is the process of computers changing the way they carry out tasks by learning from new data, without a human being needing to give instructions in the form of a program.
Let’s see which machine learning strategies/models generally exist and can be applied to trading and then slowly move on to the question of the most common pitfalls you can stumble upon while letting those models trade for you. But first – why use machine learning in trading at all?
Machine learning in trading
If you’ve been actively selling and buying cryptocurrency over a certain period of time now, you know how important the skill of spotting patterns is. Spotting patterns helps you predict the future movement of the price, and, thus, enter and exit positions in a timely manner. All in all, it is the key to successful trading, and we humans can be good at this skill. But machines can be better.
In contrast to algorithmic trading, which is an improvement on manual trading and relies on a human that has to provide an algo with a set of instructions like “when this happens, do that”, machine learning algos train to generate reliable predictions on the trade and thus choose the optimal move on their own.
Why use deep learning in trading?
Most experts agree on the fact that deep learning (DL) algorithms are best to use in relation to trading strategies. And not even because DL is the subset of machine learning that is really hot right now and advancing rapidly – but due to the very nature of DL models.
Deep learning is a subset of machine learning and is essentially a neural network with three or more layers. These neural networks attempt to simulate the behavior of the human brain—albeit far from matching its ability—allowing it to “learn” from large amounts of data.
They are simply better than so-called shallow algorithms in a few areas. They can spot patterns in large volumes of information, use them to find associations in historical data, apply those associations to uncover the market’s next move and open/close positions in bulk.
Where does one have to deal with large amounts of data, learn to see patterns and then, based on them, uncover trading opportunities? Right, in trading, whether we’re talking about traditional stock exchanges or crypto.
As a software entrepreneur Dima Vonko said in his article for Investopedia, “for a serious, thinking trader, neural networks are a next-generation tool with great potential that can detect subtle non-linear interdependencies and patterns that other methods of technical analysis are unable to uncover.”
According to Vonko, however, you should keep in mind that a 10% increase in efficiency is probably the most you will ever get from a neural network. So, please, parachute down from the Blade Runner 2049 world and keep it real – otherwise, you’ll only spend your time and lose your money.
Deep learning algorithms used in trading
So, now we know that in machine learning, there’s a subset called deep learning, which is basically neural networks. Neural networks can be applied to trading because of the large amount of data they can process. But there are, in general, three different types of neural networks out there: Artificial Neural Networks (ANN), Convolution Neural Networks (CNN), and Recurrent Neural Networks (RNN). Which ones exactly should be applied to trading?
So far, only two deep learning approaches have been used in stock market prediction mostly: Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN). Why? These neural networks can be fed with time-series data, which is central to investment and trading.
Let’s talk about what time-series data is and how it can be applied in trading.
Convolutional Neural Networks (CNNs)
If you want a CNN to uncover trading opportunities for you, you will have to feed this neural network a graph of time series data for past prices of a given asset. Say, Bitcoin. It sounds difficult on paper. But in reality, the simplest way to represent time series data, which is recorded over consistent intervals of time, is a line graph. At a high level, the CNN network will take an image of this graph and predict the next movement of the price. In more detail, you will have to find a way to retrieve historical datasets with prices somewhere, learn how to check the quality of your dataset, format it, reduce, clean, etc. and only after, feed the data to the model.
Recurrent Neural Networks (RNNs)
Compared to the CNN networks, Recurrent Neural Networks (RNNs) are distinguished by their “memory”. RNNs not only remember past prices but also use them to make predictions for the future. If you feed the time series data to this network, it will predict the direction of the next mid-price change and forecast when the change would occur. According to the paper produced by the CS 230 students at Stanford, the main idea of RNN is to apply the sequential observations learned from the earlier stages to forecast future trends. Long-Short Term Memory (LSTM) model is a very popular and updated version of RNN often used for solving corresponding trading tasks.
Now, this might seem all butterflies and rainbows, but we don’t live in the perfect world where neural networks function with no problems whatsoever. So, here’s a lineup of some challenges that might put you out of humor as a neural network developer.
Common pitfalls in using neural networks in trading
Time Lag
Since we’re developing a neural network predicting prices, we’re working with the financial time series data. And this kind of data comes with the time lag problem you will have to deal with.
Let’s assume you’ve chosen the RNN neural network for solving your problem, namely, the RNN’s updated version – Long Short-Term Memory (LSTM).
The pic is the property of Vegard Flovik, physicist and data scientist
The red line in the picture above represents the real data set with the possible prices of the possible asset (y axis) on certain days (x axis). The blue line is the result predicted by the neural network. And on the first look, the results seem quite impressive. But let’s see why everything happening is just so wrong.
A good way to start is to tell you that one of the methods to measure a similarity between two series is called cross correlation. Cross correlation is used to see at which point two data sets were similar and at which not.
Suppose, you have two datasets that look too similar to be true. Would you want to check what’s going on? Just in case? If you did and used the cross correlation technique for our example, you’d probably see something like this:
The pic is the property of Vegard Flovik, physicist and data scientist
Take a look at this picture generated by the cross correlation method and see how the cross correlation showed a clear peak. This peak indicates that between two datasets there’s a time lag of one day (y axis). And this time lag happened on day one (x axis), in the very beginning of the prediction process.
One last question: why did the values on the first pic look so similar, then? When predicting the value at time “t+1”, the neural network learnt to use the value at time “t” as its prediction. To put it very simply, if the closing price today was $500, the network thought the opening price in the morning would also be $500. Not the very best way to learn how to minimize losses, isn’t it? In case you worry about the rise of the machines, that use case kind of puts it on indefinite hold.
Accuracy metrics are not necessarily accurate
When you train a neural network to basically predict the future prices of the asset, a crypto or a traditional one, you need some accuracy metrics to see when the network is really ready to be moved from concept to action. That said, accuracy metrics in machine learning help define the accuracy levels of your network’s predictions.
There are different types of those accuracy metrics, also sometimes called error metrics. For example, when working with LTSM Network, a type of an RNN network, you might want to consider using mean absolute percentage error (MAPE) and/or R-squared altogether.
Simply put, R-squared generates a number and shows the difference between the samples in the dataset and the predictions made by the model. And MAPE is all about absolute percentage errors of each entry in a dataset. MAPE is often preferred because managers understand percentages better than squared errors.
However, because some models might correlate themselves in time, producing a time lag, they take wrong time values to calculate the outcome for accuracy metrics. Thus, those metrics can’t necessarily be relied on.
No data preprocessing
In order to generate the correct output, it’s also utterly important to preprocess your data in the right way. Even if you have the best possible model ever, just taking it and applying it to some raw financial time series data won’t work. You need time series data where mean and variance do not vary across time – this kind of data is called stationary data.
You can generate stationary data by applying First Differences and Fractional Differentiation to your time series. There is no need to explain the math in detail here, but simply take a look at the way of thinking about it to get an idea:
In statistics and econometrics, to calculate First Differences, you need to subtract the second y value from the first y value.
Let’s calculate the returns at every time step: a return r is the difference between current price and previous price.
Now, this is where you might lose the time series history. Some models might be able to reconstruct this data, but for a lot of them this information is lost, so in order not to lose this information, we would have to add up all price-returns in between.
To do so, the method of Fractional Differentiation is normally used, but we’ll not delve into math here.
The point is that as a result of applying Fractional Differentiation, your data will turn stationary while still keeping some history.
Conclusion
An AI agent that interacts with the financial market can serve as a pillar of your technical analysis in addition to all the other means that you’re already using for your trading process. However, we still live in a world where artificial intelligence is much less perfect than depicted in the movies. So, experts suggest that a 10% increase in efficiency is probably the most you will ever get from using it.
However, if you still want to build an agent of your own for your trading process, here’s a way of thinking about it.
Understand that machine learning is a type of artificial intelligence that can be applied not only to trading, but also to a lot of activities, such as speech recognition, autonomous vehicles, healthcare advancement etc. What fits best for trading, is a subset of machine learning, deep learning, which is essentially a neural network with three or more layers.
When choosing which neural network type will do, remember that so far, only two deep learning approaches have been used in price prediction mostly: Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN). However, both these networks come with a set of pitfalls you have to deal with all the way through while using them. And they are not well-oiled as of the end of 2022.
What you can do so far, if you like the idea of bringing the power of the machines into your trading strategy, is using well-oiled mechanisms, such as automated trading strategies. They will act in accordance with the set of instructions you provide and will help you stay ahead of your competitors.
Are you into crypto? TradeSanta gives you an opportunity to trade based on the algorithm using a powerful trading bot integrated with such exchanges like Binance, OKX, Huobi, Bybit, HitBTC, FTX, Kraken and Coinbase Pro and other prominent crypto exchanges.
Run Grid and DCA bots on spot and futures trading venues, trade 24/7 and don’t worry about spending all your spare time on building a complicated trading agent.
Still have questions? Shoot them on Facebook, Telegram, or Twitter!
FAQ
What is machine learning in trading?
In contrast to algorithmic trading, which is an improvement on manual trading and relies on a human that has to provide an algo with a set of instructions like “when this happens, do that”, machine learning algos train to generate reliable predictions on the trade and thus choose the optimal move on their own.
Why use deep learning in trading?
Deep learning is a subset of machine learning and is essentially a neural network with three or more layers. These neural networks attempt to simulate the behavior of the human brain—albeit far from matching its ability—allowing it to “learn” from large amounts of data.