Algo Trading for Dummies  - 3 Useful Tips When Storing Trade Signals (Part 2)

Handling & Storing Trading Signals Are Hard

The calculation of simple trading indicators is made easy with the use of any one of the Technical Analysis libraries available, however, the efficient handling and storage of trading signals can be one of the most complex aspects of a live trading system.

Photo by  Jeremy Thomas  on  Unsplash

Calculating Basic Indicators? No Problem

While it’s often necessary to create custom indicators and trading signals, there is still significant benefit to using a standard library such as Ta-Lib for the basics. This saves a lot of time rather than having to reimplement a set of common indicators in your language of choice. It also has the added bonus of increased processing speed as opposed to calculation done in native Python, for example.

When it comes to moving averages and other simple time-series indicators, the process is fairly self explanatory — at every time step you calculate the next numerical value which is then used as the most up-to-date signal to trade against.

(Code Snippet to read data CSV files and process into trading indicators)

The signals themselves will be stateless in that respect — you aren’t concerned with previous signals that have been made, only the combination of indicators present at that moment. However, you may still wish to store some of the information from the indicators, if only for external analysis at a later point.

Different Story For Advanced Pattern Recognition

Meanwhile, more advanced pattern recognition cannot be handled in such a simple manner. If, for example, your strategy relies on finding divergence between indicators, its possible to get a significant performance boost by storing some past data-points from which to construct the signal at each new step, rather than having to reprocess the full set of data in the look-back period every time.

This is the trade-off between storage/ RAM efficiency and processing efficiency, with the latter also requiring greater software complexity to achieve.

How You Should Store Signals Depends On How Fast You Need It To Be

For optimal processing efficiency, you would not only store all the previously calculated signals from past time-stamps, but also the relevant information to calculate the next step in as fewer steps as possible.

While this would be completely unnecessary for any system with a polling rate above a seconds, it is exactly the kind of consideration you would have for a higher frequency strategy.

Meanwhile, a portfolio re-balancing system, or even most day-trading strategies, have all the time in the world (relatively). You could easily recalculate all the relevant signals at each time-step, which would cut down on the need for the handling of historical indicator sets.

Depending on the trading period of the system, it may also be worth using a hybrid approach to indicator and signal storage. Rather than permanently saving the data, you could calculate the full set of indicators at start-up and periodically dump and refresh the data to keep only whats going to be used in RAM.

The precise design trade-offs should considered on an individual basis, as holding more data in RAM may not be an option when running the software from lower power cloud computing instances nor, at the other end of the spectrum, would you be able to spare the seconds to recalculate everything for a market making bot.

3 Useful Tips When Storing Trade Signals

As mentioned in the part 1 of this series, there are range of different storage solutions that can be used for trading data. However, there are several best practices which apply across all:

  1. Keep indicators in a numeric or boolean format where possible for storage. For example, splitting a more complex signal set into boolean components. This particular problem caused me several issues in projects I’ve had to work on in the past.
  2. Only store what is complex or time-consuming to recalculate. If a set of signals can be calculated in time in a stateless manner, its probably easier to do so than add the design complexity of storing extra information.
  3. Plan out the flow of data through your system before you start programming anything. What market data is going to be pulled for each time-step? What will then be calculated from this and what is necessary to store? A well thought-out design will reduce complexity and hassle down the line.

Past this, common sense applies. Its probably best to store the indicators and signals in the same time-series format as, and along side, the underlying symbols they’re derived from. More complex signals, or indicators derived from multiple symbols, may even warrant their own calculation and storage process.

You could even go as far as to create a separate indicator feed script which calculates and stores everything separately from the trading bot software itself. The database could then be read by each bot as just another data feed. This not only has the benefit of keeping the system more modular, but also allowing you to create a highly optimized calculation function without the complexity of direct integration into a live system.

Whatever flavour of system you end up using, make sure to plan out the data storage and access first and foremost, before starting the rest of the design and implementation process.

By Matthew Tweed


So You Want to Trade Crypto  - Fundamental Analysis (Part 1)

This will be a mini-series looking into: 

  • the nature of Cryptocurrency trading and investment, 
  • the key factors which shift the markets, and
  • the important differences between Crypto and more traditional traded assets.

Fundamental factors of Cryptocurrencies vs traditional assets.

In terms of investing and long term trading, Cryptocurrency can be analyzed somewhere between a stock and a commodity.

It should be seen very much as a speculative investment — in the case of most Cryptocurrencies, you’re buying into a token which represents no stake in a company nor any voting rights, similar to a commodity. However, you’re not investing into a physical asset either. You’re simply betting that over time the use-case and thus demand for your token will grow, driving up the price.

Since price over time will be relative to the success of product, it can be analyzed similarly to an investment in a start-up. The main focus when looking at a token should be the team and their ability to deliver a working product that will actually be used. Not only this, you should also take
into consideration the use of the token itself within their product.

Will growth of the product actually lead to higher demand for the token, or is it simply tacked on as an after thought?

Additionally, Cryptocurrency projects seem to have a very different valuation split to previous technologies. A shift in valuation from the applications built on top of a protocol to the Blockchain protocol itself. No-one is investing in creating a new protocol layer for the internet, as the captured value comes from the applications and companies build upon it, such as Google, Facebook and Amazon.

On the other hand, Cryptocurrencies such as Bitcoin and Ethereum have market caps of $100B while very few applications have even been produced yet, let alone applications in wide use outside of the sphere of Crypto-enthusiasts.

This may be a symptom of a wider speculative bubble around the Cryptocurrency market as a whole, or simply a fact of the technology — requiring a sufficient valuation to be maintained to allow efficient flow of payments. That being said, we haven’t seen a truly successful product built
on Blockchain yet, maybe the Blockchain killer app will dwarf current valuations in the long term - only time will tell.

What to look for in projects

The core proposition of Cryptocurrencies over the traditional financial system is the fact that you don’t have to trust a third party. 

Trust-less decentralization

A transaction can be sent to anyone anywhere in the world without needing a bank or escrow account in the middle. This property of trust-less decentralization should be the backbone of any good Cryptocurrency.

However, simple transactions are a very basic and limited use-case, which has been offered by Bitcoin since its first release in 2009. A project has to offer a unique and useful advancement on basic Blockchain technology. Many new Cryptos are becoming “smart contract platforms”.

Smart contracts are pieces of code which can be run on top of the Blockchain, in the same decentralized and trust-less manner as payments. This allows the easy creation of full applications on top of the basic transaction layer, providing much wider uses for businesses and developers.

Longterm Scalability

Longer term, one of the fundamental issues that needs to be solved in Blockchain technology is that of scaling. Since the basic Blockchain model relies on all connected nodes to process every transaction, it quickly runs into limitations of how fast a payment can be distributed and verified, not to mention that of storage.

While there are many projects out there which claim “1,000,000 transactions per second”, almost all of these approaches are either impractical in real life or break the fundamental premise of a Cryptocurrency being trust-less and decentralized. Its relatively easy to achieve “1M tps” when running high powered servers in the same location that all have implicit trust.
However, in reality, flow of transactions is limited by latency between nodes around the globe and the fact that we shouldn’t rely on a central node for verification — otherwise why don’t we just use a regular bank?

The scaling issue is being progressed on with such technologies as “sharding” or a change from a traditional Blockchain to a “DAG”. In the long term, these are the kind of advancements which will achieve reliable scaling, rather than misleading promises from certain projects.

Product Value w/ Blockchain

Finally, and most importantly, you should consider who the Cryptocurrency is developed for and whether it is truly a useful product to them. Many have tried applying Blockchain in a misguided manner to any and all systems to join in with the hype. However, unless you need the very specific properties of a decentralized and trust-less system, Blockchain is no more than a glorified

Final thoughts

While there is innovation left and right in the Cryptocurrency space, caution is still strongly advised. Out of the over 1500 coins and tokens in existence, only a handful of projects are likely to survive. While the vast majority will fail in the long run, due diligence pays off — helping us sift through the rubbish to find those which have a chance at leading the revolution of decentralized finance.

by Matthew Tweed