Algo Trading News Headlines 7/5/2018

11 Ways Technology Has Transformed the Investing World

(www.tgdaily.com)

Technology has reshaped our world and transformed virtually every industry — including investing. Today’s investors have access to an endless amount of information and trading tools that help them make smart investments.

Cryptocurrency Markets Are Down, But Hedge Funds Can Still Profit

(blocktribune.com)

Additionally, quantitative trading has grown in popularity as the markets have declined. While many retail investors do not have the expertise to build out quant models and automate trading bots to operate within the specified parameters, there is a much simpler approach that operates on a similar basis.

Man Behind RBC’s Battle Against High-Frequency Trading to Leave

(www.bloomberg.com)

Mills, who led the global equities business out of Toronto since 2005, decided to depart after “productive discussions” the firm had on succession planning, according to an internal memo.

Algo Trading for Dummies — Building a Custom Back-tester (Part 3)

(blog.alpaca.markets)

Once the full framework has been designed, implemented and debugged should you start looking for ways to speed up and upgrade the inner loop of the back-tester (the order handling module). It is a lot easier to take a working program and make it faster than it is to take an overly optimized program and make it work.

Quants Might Not Be So Special Someday

(www.bloomberg.com)

I am not sure that “data science” will ever quite get to that point, where coding machine-learning models will be a baseline expectation of every first-year analyst. But you could easily imagine it getting close, to the point that every reasonably sophisticated investment firm will have a data-science department, and every reasonably competent analyst will at least be able to formulate useful questions for those departments and make good use of the answers.

It’s Been a Rough 2018 for Many Quant Hedge Funds

(www.bloomberg.com)

Quant funds, which rely on complex algorithms to navigate global markets, have suffered this year because of the rise in volatility. They were caught flat-footed in February when markets turned turbulent on concern over rising interest rates, followed by trade wars with China and the election in Italy.

/

9 Great Tools for Algo Trading

In the last 5–10 years algorithmic trading, or algo trading, has gained popularity with the individual investor. The rise in popularity has been accompanied by a proliferation of tools and services, to both test and trade with algorithms. I’ve put together a list of 9 tools you should consider using for your algo trading process.

 Photo by  Adrian Curiel  on  Unsplash

Photo by Adrian Curiel on Unsplash

Web Services:

The following are managed-services that you can use through web browsers, and don’t require much setup from the user. As someone who’s recently started in this field, I found it easy for new algo traders to try out.

(1) Quantopian:

1_xmqbJh3dGQIXylJTSCD7dA.png

A Boston-based crowd-sourced hedge fund, Quantopian provides an online IDE to backtest algorithms. Their platform is built with python, and all algorithms are implemented in Python. When testing algorithms, users have the option of a quick backtest, or a larger full backtest, and are provided the visual of portfolio performance.

Live-trading was discontinued in September 2017, but still provide a large range of historical data. They also have a serious community of developers, and now hold an ongoing DAILY contest with 10 winners awarded each day for a total of $5,000 per month in prize money (*updated from previously written as "periodically hold contests"). Quantopian provides capital to the winning algorithm. 

(2) QuantConnect:

1_eQopy9i2kr63RFeCFQq-sA.png

QuantConnect, is another platform that provides an IDE to both backtest and live-trade algorithmically. Their platform was built using C#, and users have the options to test algorithms in multiple languages, including both C# and Python.

QuantConnect also embraces a great community from all over the world, and provides access to equities, futures, forex and crypto trading. They offer live-trading integration with various names such as InteractiveBrokers, OANDA, and GDAX.

(3) QuantRocket:

QuantRocket is a platform that offers both backtesting and live trading with InteractiveBrokers, with live trading capabilities on forex as well as US equities. It’s specifically designed for trading with InteractiveBrokers, and sets itself apart with its flexibility.

QuantRocket supports multiple engines — its own Moonshot, as well as third party engines chosen by the user. While QuantRocket doesn’t have a traditional IDE, it is integrated well with Jupyter to produce something similar. One thing to keep in mind is that QuantRocket is not free. Pricing plans start at 19.99/month USD, with annual options.

Local Backtesting/LiveTrading Engines:

In today’s software world, you have lots more freedom if you make some effort outside of those managed-services. If you are comfortable this way, I recommend backtesting locally with these tools:

(4) Zipline/Zipline-Live:

quantopian/zipline
zipline - Zipline, a Pythonic Algorithmic Trading Librarygithub.com

Quantopian’s IDE is built on the back of Zipline, an open source backtesting engine for trading algorithms. Zipline runs locally, and can be configured to run in virtual environments and Docker containers as well. Zipline comes with all of Quantopian’s functions, but not all of its data. To balance that, users can write custom data to backtest on. Zipline also provides raw data from backtests, allowing for versatile uses of visualization.

Zipline discontinued live trading in 2017, but there is an open source project Zipline-live that works with Interactive Brokers. It has many of the same features Zipline does, and provides live trading.

(5) BackTrader:

backtrader - Backtesting / Trading
from datetime import datetime import backtrader as bt class SmaCross(bt.SignalStrategy): params = (('pfast', 10)…www.backtrader.com

Backtrader is currently one of the most popular backtesting engines available. It was built using python, and has a clean, simple, and efficient interface that runs locally (no Web Interface). One thing to keep in mind, backtrader doesn’t come with any data, but you can hook up your own market data in csv and other formats pretty easily.

Starting with release 1.5.0, BackTrader has live-trading capabilities. It’s been a popular choice with algo traders, especially after Zipline discontinued live trading.

(6) IBPy:

blampe/IbPy
IbPy - Python API for the Interactive Brokers on-line trading system.github.com

IBPy is an unaffiliated third party python wrapper for InteractiveBroker’s Trade Workstation API. Before IB started providing their official API library for python, this was the only way to connect to TWS for algorithms written in python.

IB has released an official python SDK, and this library is heading towards begin obsolete(while still being relevant for python2 users). But there still remain a significant number of live trading engines/tools that still use this library, and it’s good learning material for whoever wants to learn about implementing API’s.

While it’s good to learn about this library since it’s ubiquitous, if you are starting fresh, we recommend IB’s official python SDK.

Alpaca Trade API Python SDK is even much simpler to use!

Analytical Tools:

Back testing will output a significant amount of raw data. Some IDE’s will provide basic visualization and analysis, usually algorithm performance. If you’re looking for deeper evaluation, I recommend these tools:

(7) Pyfolio:

quantopian/pyfolio
pyfolio - Portfolio and risk analytics in Pythongithub.com

Pyfolio is another open source tool developed by Quantopian that focuses on evaluating a portfolio. What sets Pyfolio apart, is its ability to introduce degrees of uncertainty to a static set of data points, and evaluate Bayesian metrics from the user’s portfolio. The Pyfolio API offers a number of visualizations, which can be found on their GitHub repository.

(8) Alphalens:

quantopian/alphalens
alphalens - Performance analysis of predictive (alpha) stock factorsgithub.com

Alphalens is also an analysis tool from Quantopian. Unlike Pyfolio, Alphalens works well with the raw data output from Zipline, and rather than evaluate the portfolio, is performance analysis of predictive stock factors. Alphalens has its own range of visualizations found on their GitHub repository.

1_HnTL1cG_aBDoyBDyOzfkew.png

Median Daily Returns by Factor Quantile — one of the visualizations that alphalens offers

(9)TradingView:

1_Vy5CbHA0AdW7pwSmiwoZng.png

TradingView is a visualization tool with a vibrant open-source community. It’s entirely web-based, and allows users to visualize data, whether the data is the result of paper trading or algorithmic back-testing. Like Quantopian, TradingView allows users to share their results and visualizations with others in the community, and receive feedback.

(Bonus) Execution Platforms aka Broker-Dealers:

(10)InteractiveBrokers:

1_jAUoYLH-Fa3CyOtsfzsW3w.png

InteractiveBrokers is an online broker-dealer for active traders in general. They have been in the market since 1978. Algo trading isn’t IB’s focus, but multiple engines offer live trading through integration with their Trader Workstation. We’ve mentioned IB several times in this article — they’re just that good!

(11)Alpaca:

1_bhSUXCHVNP_JQmsMHklPlg.png

Finally, Alpaca! Alpaca was founded in 2015, and is an up and coming commission-free, broker-dealer designed specifically for algo trading. Alpaca also has a trade api, along with multiple open-source tools, which include a database optimized for time-series financial data known as the MarketStore.

The brokerage is scheduled to be publicly available this September (you can play around with the MarketStore right now), but if you can’t wait, head over to our website and jump on the waitlist for a chance at early access!

Alpaca | Algo Trading Commission Free with REST API
Lower your cost, Maximize your profits. Algo trading commision free.alpaca.markets

Miscellaneous Tools to Take a Look At:

  • qtpylib — another simplistic python backtesting engine
  • Multicharts — proprietary trading platform for forex and equities
  • WealthLab — desktop tool which allows C# backtesting, with live trading exclusive to Fidelity
  • Enygma Catalyst — for crypto trading
  • MetaTrader — backtesting/livetrading desktop app, de-fact in forex

I hope this quick primer on tools available right now was useful. If you liked it, please leave a clap (or two, I don’t mind). If you think there are tools that I missed, leave a comment below! I always appreciate any, and all feedback.

by Rao Vinnakota

/

Algo Trading for Dummies -  Building a Custom Back-tester (Part 3)

While there are many simple backtesting libraries available, they can be quite complex to use effectively — requiring a lot of extra processing of data sets. It is sometimes worth coding a custom back-tester to suit your needs.

Image from iOS (4).png

Building a back-tester is a fantastic conceptual exercise. Not only does this give you a deeper insight into orders and their interaction with the market, but it can also provide the framework for the order handling module of your trading bot.

Order Handling

One of the key pieces to an active trading strategy is the handling of more advanced orders types, such as trailing stops, automatically hedged positions or conditional orders.

For this you’ll want a separate module to manage the order logic before submitting to an exchange. You may even need a dedicated thread to actively manage orders once submitted, in case the platform itself doesn’t offer the necessary types.

Its best for the module to keep an internal representation of each position and its associated orders, which is then verified and amended as the orders are filled. This means you can run calculations against your positions without the need to constantly be querying the broker. It also allows you to easily convert the code for use in your back-tester, by simply altering the order fill checks to reference the historical data at each time step.

 (Code Snippet of an order handling function as part of a position handler — full script at end of article)

(Code Snippet of an order handling function as part of a position handler — full script at end of article)

It may also be worth implementing order aggregation and splitting algorithms. For example, you may want a function to split a larger limit order across multiple price levels to hedge your bets on the optimal fill. Or, indeed, you might need a system to net together the orders of multiple simultaneous strategies.

Assumptions and Issues of Back-testing

Unless you’re using tick data and bid/ask snapshots to back-test against, there will always be a level of uncertainty in a simulated trade as to whether it would fill fully, at what price, and at what time. The period of each data point can also cause issues if its above the desired polling rate of the trading bot.

These uncertainties are lessened as the average holding period for each trade increased vs the resolution of your data, but is never fully eliminated. It is advised to always assume the worst case scenario in your simulation, as its better for a strategy to be over prepared than under.

 (Back-testing order processing logic implemented into position handler — full script at end of article)

(Back-testing order processing logic implemented into position handler — full script at end of article)

For example, if a stop-loss order would have been triggered during the span of a bar, then you’d want to add some slippage to its trigger price and/or use the bar’s closing price. In reality, your are unlikely to get filled so unfavorably, but it’s impossible to tell without higher granularity data.

On top of this, it is impossible to simulate the effect of your order on the market movement itself. While this would be unlikely to have a noticeable effect on most strategies, if you’re using extremely short holding times on each trade or larger amounts of capital, it could certainly be a factor.

Designing an Efficient Back-tester

When calculating the next time step for an indicator, unless you’ve stored all relevant variables you will be recalculating a lot of information from the look-back period. This is unavoidable in a live system and, indeed, less of an issue, as you won’t be able to process data faster than it arrives. But you really don’t want to wait around longer than you have to for a simulation to complete.

The easiest and most efficient workaround is to calculate the full set of indicators over the whole dataset at start-up. These can then be indexed against their respective symbols and time stamps and saved for later. Even better, you could run a batch of back-tests in the same session without needing to recalculate the basic indicators between runs.

At each time you will then simply query the set of indexed indicators, construct the trading signals and push the orders to the order handling module, where the simulated positions are calculated along with their profit/ loss. You’ll also want to store the position and order fill information, either as a subscript to the back-tester or integrated directly into the position handling module.

Even Improving Your Back-tester

Back-testing is only as useful as the insight its statistics provide. Common review metrics include win/loss ratio, average profit/loss, average trade time, etc. However you may want to generate more insightful reports, such as position risk:reward ratios or an aggregate of price movement before and after each traded signal, which allows you to fine tune the algorithm.

Once the full framework has been designed, implemented and debugged should you start looking for ways to speed up and upgrade the inner loop of the back-tester (the order handling module). It is a lot easier to take a working program and make it faster than it is to take an overly optimized program and make it work.

By Matthew Tweed

Full position handling class framework:

/

Algo Trading News Headlines 6/28/2018

Oil Hedge Funds Struggle in Age of Algos

(www.wsj.com)

Raw materials have staged a powerful rally over the past year, but that hasn’t helped some of the biggest commodities hedge funds, which are closing up shop to a degree not seen since at least the turn of the century.

 Photo by  Zbynek Burival  on  Unsplash

I Built a Go Plugin for Alpaca’s MarketStore as a College Intern

(blog.alpaca.markets)

For this blog post, I’ll be going over how I created the plugin from start to finish in three sections: Installing MarketStore, understanding MarketStore’s plugin structure, creating the Go plugin., and installing the Go plugin.

Man Vs. Machine: Credit Suisse Finds Renewed Value In Human Traders

(www.forbes.com)

If we take into account the shift from active fund management to passive investing, firms have limited trading opportunities that serve hedge funds. Also, sometimes the most difficult trades are often the largest.

AI Trader unveils its OCO Trading MODE to cryptocurrency investors

(blog.coinspectator.com)

According to its website, AI Trader “uses Artificial Intelligence to identify patterns from big data processed by the program to recognize trends and autonomously process buy, hold and sell orders based on current market events.”

/

I'm a College Student, and I'm Still Building my Robo-Advisor

All code and outside instructions for setup, etc. can be found in the GitHub repository here. This is a post is a sequel to building a robo advisor in Quantopian. You can find the first part on Hackernoon here, or on Alpaca’s blog here.

 I’m…still…working…

I’m…still…working…

Hey all, I’m Rao, an intern at Alpaca. About two weeks ago, I published an article on HackerNoon about the Robo-Advisor I’ve been working on. In the last two weeks, I’ve moved my work offline, pushing development with Zipline. In that time, I’ve picked up a few tips and tricks, and I wanted to share that with you.

Zipline — What, When, Why, Where, How?

We’ve now created a functional robo advisor on Quantopian. But, the IDE puts limits on the scale and sophistication of our work. To extend this robo advisor further, it’s best to do our future work locally with Zipline.

This shift offers advantages. We can use multiple files, configuration files, and generally develop out robo-advisor like any other python app. Being free from a single file format means that our work can be more organized, and individual functions less bloated (more on that later).

But, it’s not all roses. Developing in Zipline comes with its own challenges. But, if I can do it, you can too!

Installing zipline on your local machine isn’t as simple as `pip install zipline`. It has several dependencies, which vary slightly for each OS. You can find all necessary installation instructions here.

I made several mistakes during installation, so I recommend you run zipline in either a virtual environment, or my personal preference, in a Docker container. I’ve attached both the finished container to play with, as well as the corresponding Dockerfile. Take some time, figure out your development environment. Ready? Let’s get started.

Zipline Commands — Bundles and Execution

Open a new python file locally. Copy and paste your code from the Quantopian IDE. Now, we’re ready to go- just yanking you around. We’re not close to ready. Quantopian’s IDE is a great development environment. It takes care of background processes, allowing you to focus solely on your algorithm.

First, we need historical data to run our algorithm on. Zipline reads through data using something called bundles. To use the bundles, they need to be ingested first, which is done using the command:

$zipline ingest -b <bundle>

Zipline provides two bundles, quandl and quantopian-quandl. To ingest quandl, you’ll need to make an account, and then obtain an API key. You won’t need to for quantopian-quandl. Zipline also provides the option to create custom bundles, by writing a custom ingest function. That will be reserved for a later post.

Both Quandl and quantopian-quandl don’t provide ETF data, so I’ve provided a custom alpaca bundle. The instructions for installing this bundle locally can be found in the GitHub repository readme.

You run zipline code using the following format:

$zipline run -f <filename> -b <bundle> --start <date> --end <date>

You can see a lot of the GUI input from Quantopian is configured from the command line with Zipline.

Docker Containers:

I’ve created a docker container running python 3.6 with zipline and necessary dependencies installed. If you’re hesitant about installing zipline locally, you can pull the container from the docker hub, and experiment with the environment.

Run the image with the command:

$docker run -it alpaca/roboadvisor /bin/bash

A First Zipline Example — Buy and Hold

Like my initial start with Quantopian, I started with Zipline using a simple buy and hold strategy.

Here are a few differences to keep in mind. First, all the custom functions like order, symbols, etc. are no longer automatically included, and have to be manually imported (from zipline.api import *). Next, this is rather silly, but now that we’re not in Quantopian’s IDE, we’re no longer going to use log.info to track transactions, but print statements.

Let’s run the code with the format from earlier, and run the tests from January to June:

$zipline run -f buy-and-hold.py -b alpaca --start 2018-01-01 --end 2018-06-01

If there’re no syntax errors, Zipline will spit out a whole bunch of data. How do we know if we’re right? Scroll until you find the following table from STDOUT:

 Look at the cumulative alpha value from the last simulated trading day (2018–06–01)

Look at the cumulative alpha value from the last simulated trading day (2018–06–01)

Find the cumulative alpha value (performance against benchmark), and compare it to the cumulative alpha value when you run the same algorithm on Quantopian. They should be very similar values.

We’ve got a good idea of how Zipline works, so we can go ahead and implement the rest of our single-universe algorithm (distance computation and rebalancing). I spent an entire post talking about this implementation, so I don’t think there’s a need to re-hash it. If you haven’t read that, I recommend you do — it’s a great read! (a totally unbiased opinion)

Multiple Universes

In the previous post, I showed how to expand the algorithm to cover all possible Vanguard universes by adding multiple dictionaries in initialize function. But, if we want to implement all six vanguard universes, we would finish with a bloated ingest function, which doesn’t look that pretty.

But now we’re away from Quantopian, and here with Zipline! We can spread our code across multiple files, so let’s go ahead and do that.

To be used by the algorithm, a universe needs two sets of information. The first is the set of symbols, and the second is the weight distribution based on the risk. From the way that the symbols function works, each universe’s list of symbols needs to set in the initialization function.

But, the weights are dictionaries that aren’t bound to any zipline.api functions, so we can actually configure those in a separate file. The ConfigParser package can read in data stored in an external .ini file. More importantly, it stores the data in a similar structure to a dictionary (key/value), and is easily called and organized.

First, we need to install the package:

$pip3 install ConfigParser

With the package installed, it’s time to create the INI file. The INI file contains the information for allocation based on risk level.

INI files are separated into sections, with a new section delineated by a section title in brackets. The section titles act much like a key in dictionaries, and is a useful way to organize inputs. This file organized each individual Vanguard universe as its own section. If you’d like to add your own universe, fork the gist, create a new section with the universe name in brackets, and list the information below it.

The robo-advisor’s algorithm parses input as a dictionary. While the INI file works much like a dictionary, we’ll still need to write a function to actually translate it into a dictionary.

def section_to_dict(section):
    config = ConfigParser()
    config.read('universe-config.ini')
    out_dict = {}
    for key in config[section]:
        out_dict[int(key)] = ast.literal_eval(config[section][key])
    return(out_dict)

The configparser is initialized, and then reads the given INI file. The section that’s read is given as user input. If we refer back to the INI file, we’ll see that the values take the form as key = value. So like a dictionary, we can iterate through the deys of an INI section. In each case, each key/value pair is added to the dictionary. (Note: If you receive a key error, it’s because the path to your config.ini file is wrong

Two more things to be aware of. In the INI file, everything is a string, but our algorithm expects an integer/tuple key-value pairs. For the key, just casting them as integers is enough, but for the tuple, we’ll have to unstring the value. For that, I found ast.literal_evalto be the function that worked best.

Now, let’s integrate this new method of retrieving weight-based allocation with our robo-advisor algorithm:

I’ve listed all the symbols for every universe in the initialization. The weights for that universe are determined by calling section_to_dict on the appropriate section of the INI file.

Now, feel free to add as many universes as you like!

What’s Next?

Going from here, I’m interested in visualizing all the output data from the Zipline backtest. With my next post, I’m hoping to explore the different visualizations available from that raw data.

by Rao Vinnakota

/

I Built a Go Plugin for Alpaca’s MarketStore as a College Intern

Hey all! I’m Ethan and recently started working for Alpaca as a Software Engineering Intern! For my first task, I created a Go plugin for Alpaca’s open source MarketStore server that fetches and writes Binance minute-level.

Image from iOS (2).jpg

You might be wondering — What is MarketStore? MarketStore is a database server written in Go that helps users handle large amounts of financial data. Inside of MarketStore, there are Go plugins that allow users to gather important financial and crypto data from third party sources.

For this blog post, I’ll be going over how I created the plugin from start to finish in three sections: Installing MarketStore, understanding MarketStore’s plugin structure, creating the Go plugin., and installing the Go plugin.

Experience Installing and Running MarketStore Locally

First, I set up MarketStore locally. I installed the latest version of Go and started going through the installation process outlined in MarketStore’s README. All the installation commands worked swimmingly, but when I tried to run marketstore using

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ marketstore -config mkts.yml

I got this weird error:

/usr/local/go/src/fmt/print.go:597:CreateFile/go/src/github.com/alpacahq/marketstore/executor/wal.go:87open /project/data/mktsdb/WALFile.1529203211246361858.walfile: no such file or directory: Error Creating WAL File

I was super confused and couldn’t find any other examples of this error online. After checking and changing permissions in the directory, I realized my mkts.yml file configuration root_directory was incorrect. To resolve this, I changed mkts.yml from

root_directory: /project/data/mktsdb

To

root_directory: /home/ethanc/go/bin/src/github.com/alpacahq/marketstore/project/data/mktsdb

and reran

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ marketstore -config mkts.yml

This time, everything worked fine and I got this output:

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ marketstore -config mkts.yml
…
I0621 11:37:52.067803 27660 log.go:14] Launching heartbeat service…
I0621 11:37:52.067856 27660 log.go:14] Enabling Query Access…
I0621 11:37:52.067936 27660 log.go:14] Launching tcp listener for all services
…

To enable the gdaxfeeder plugin which grabs data from a specified cryptocurrency, I uncommented these lines in the mkts.yml file:

and reran

ethanc@ethanc-Inspiron-5559:~$ marketstore -config mkts.yml

which yielded:

…
I0621 11:44:27.248433 28089 log.go:14] Enabling Query Access…
I0621 11:44:27.248448 28089 log.go:14] Launching tcp listener for all services…
I0621 11:44:27.254118 28089 gdaxfeeder.go:123] lastTimestamp for BTC = 2017–09–01 04:59:00 +0000 UTC
I0621 11:44:27.254189 28089 gdaxfeeder.go:123] lastTimestamp for ETH = 0001–01–01 00:00:00 +0000 UTC
I0621 11:44:27.254242 28089 gdaxfeeder.go:123] lastTimestamp for LTC = 0001–01–01 00:00:00 +0000 UTC
I0621 11:44:27.254266 28089 gdaxfeeder.go:123] lastTimestamp for BCH = 0001–01–01 00:00:00 +0000 UTC
I0621 11:44:27.254283 28089 gdaxfeeder.go:144] Requesting BTC 2017–09–01 04:59:00 +0000 UTC — 2017–09–01 09:59:00 +0000 UTC
…

Now that I got MarketStore running, I used Jupyter notebooks and tested out the commands listed in this Alpaca tutorial and got the same results. You can read more about how to run MarketStore in MarketStore’s README, Alpaca’s tutorial, and this thread.

Understanding how MarketStore Plugins work

After installing, I wanted to understand how their MarketStore repository works and how their current Go plugins work. Before working in Alpaca, I didn’t have any experience with the Go programming language. So, I completed the Go’s “A Tour of Go” tutorial to get a general feel of the language. Having some experience with C++ and Python, I saw a lot of similarities and found that it wasn’t as difficult as I thought it would be.

Creating a MarketStore Plugin

To get started, I read the MarketStore Plugin README. To summarize at a very high level, there are two critical Go features which power plugins: Triggers and BgWorkers. You use triggers when you want your plugin to respond when certain types data are written to your MarketStore’s database. You would use BgWorkers if you want your plugin to run in the background.

I only needed to use the BgWorker feature because my plugin’s goal is to collect data outlined by the user in the mkts.yml configuration file.

To get started, I read the code from the gdaxfeeder plugin which is quite similar to what I wanted to do except that I’m trying to get and write data from the Binance exchange instead of the GDAX exchange.

I noticed that the gdaxfeeder used a GDAX Go Wrapper, which got its historical price data public endpoint. Luckily, I found a Go Wrapper for Binance created by adshao that has the endpoints which retrieves the current supported symbols as well as retrieves Open, High, Low, Close, Volume data for any timespan, duration, or symbol(s) set as the parameters.

To get started, I first created a folder called binancefeeder then created a file called binancefeeder.go inside of that. I then first tested the Go Wrapper for Binanceto see how to create a client and talk to the Binance API’s Kline endpoint to get data:

I then ran this command in my root directory:

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ go run binancefeeder.go

and received the following response with Binance data:

&{1529553060000 6769.28000000 6773.91000000 6769.17000000 6771.34000000 32.95342700 1529553119999 223100.99470354 68 20.58056800 139345.00899491}
&{1529553120000 6771.33000000 6774.00000000 6769.66000000 6774.00000000 36.43794400 1529553179999 246732.39415947 93 20.42194600 138288.41850603}
…

So, it turns out that the Go Wrapper worked!

Next, I started brainstorming how I wanted to configure the Binance Go plugin. I ultimately chose symbols, queryStart, queryEnd, and baseTimeframe as my parameters since I wanted the user to query any specific symbol(s), start time, end time, and timespan (ex: 1min). Then, right after my imports, I started creating the necessary configurations and structure for BinanceFetcher for a MarketStore plugin:

The FetcherConfig’s members are what types of settings the user can configure in their configuration file (ex: mkts.yml) to start the plugin. The BinanceFetcher’’s members are similar to the FetcherConfig with the addition of the config member. This will be used in the Run function later.

After creating those structures, I started to write the background worker function. To set it up, I created the necessary variables inside the backgroundworker function and copied the recast function from the gdaxfeeder. The recast function uses Go’s Marshal function to encode the config JSON data received, then sets a variable ret to an empty interface called FetcherConfig. Then it stores the parsed JSON config data in the ret variable and returns it:

Then inside the NewBgWorker function, I started to create a function to determine and return the correct time format as well as set up the symbols, end time, start time, and time duration. If there are no symbols set, by default, the background worker retrieves all the valid cryptocurrencies and sets the symbol member to all those currencies. It also checks the given times and duration and sets them to defaults if empty. At the end, it returns the pointer to BinanceFetcher as the bgworker.BgWorker:

Then, I started creating the Run function which is implemented by BgWorker (see bgworker.go for more details). To get a better sense of how to handle errors and write modular code in Go, I read the code for plugins gdaxfeeder and polygon plugins. The Run function receives the BinanceFetcher (which is dereferenced since bgworker.BgWorker was the pointer to BinanceFetcher). Our goal for the Run function is to call the Binance API’s endpoint with the given parameters for OHLCV and retrieve the data and writes it to your MarketStore’s database.

I first created a new Binance client with no API key or secret since I’m using their API’s public endpoints.

Then, to make sure that the BinanceFetcher doesn’t make any incorrectly formatted API calls, I created a function to check the timestamp format using regex and change it to the correct one. I had to convert the user’s given timestamp to maintain consistency in the Alpaca’s utils.Timeframe which has a lot of helpful functions but has different structure members than the one’s Binance uses (ex: “1min” vs. “1m”). If the user uses an unrecognizable timestamp format, it sets the baseTimeframe value to 1 minute:

The start and end time objects are already checked in the NewBgWorker function and returns a null time.Time object if invalid. So, I only have to check if the start time is empty and set it to the default string of the current Time. The end time isn’t checked since it will be ignored if incorrect which will be explained in the later section:

Now that the BinanceFetcher checks for the validity of its parameters and sets it to defaults if its not valid, I moved onto programming a way to call the Binance API. 

To make sure we don’t overcall the Binance API and get IP banned, I used a for loop to get the data in intervals. I created a timeStart variable which is first set to the given time start and then created a timeEnd variable which is 300 times the duration plus the timeStart's time. At the beginning of each loop after the first one, the timeStart variable is set to timeEnd and the timeEnd variable is set to 300 times the duration plus the timeStart’s time:

When it reaches the end time given by the user, it simply alerts the user through glog and continues onward. Since this is a background worker, it needs to continue to work in the background. Then it writes the data retrieved to the MarketStore database. If invalid, the plugin will stop because I don’t want to write garbage values to the database:

Installing Go Plugin

To install, I simply changed back to the root directory and ran:

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ make plugins

Then, to configure MarketStore to use my file, I changed my config file, mkts.yml, to the following:

Then, I ran MarketStore:

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ marketstore -config mkts.yml

And got the following:

…
I0621 14:48:46.944709 6391 plugins.go:42] InitializeBgWorkers
I0621 14:48:46.944801 6391 plugins.go:45] bgWorkerSetting = &{binancefeeder.so BinanceFetcher map[base_timeframe:1Min query_start:2018–01–01 00:00 query_end:2018–01–02 00:00 symbols:[ETH]]}
I0621 14:48:46.952424 6391 log.go:14] Trying to load module from path: /home/ethanc/go/bin/bin/binancefeeder.so…
I0621 14:48:47.650619 6391 log.go:14] Success loading module /home/ethanc/go/bin/bin/binancefeeder.so.
I0621 14:48:47.651571 6391 plugins.go:51] Start running BgWorker BinanceFetcher…
I0621 14:48:47.651633 6391 log.go:14] Launching heartbeat service…
I0621 14:48:47.651679 6391 log.go:14] Enabling Query Access…
I0621 14:48:47.651749 6391 log.go:14] Launching tcp listener for all services…
I0621 14:48:47.654961 6391 binancefeeder.go:198] Requesting ETH 2018–01–01 00:00:00 +0000 UTC — 2018–01–01 05:00:00 +0000 UTC
…

Testing:

When I was editing my plugin and debugging, I often ran the binancefeeder.go file:

ethanc@ethanc-Inspiron-5559:~/go/bin/src/github.com/alpacahq/marketstore$ go run binancefeeder.go

If I ran into an issue I couldn’t resolve, I used the equivalent print function for Go (fmt). If there is an issue while running the plugin as part of MarketStore via the marketstore -config mkts.yml command, I used the glog.Infof() or glog.Errorf() function to output the corresponding error or incorrect data value.

Lastly, I copied the gdaxfeeder test go program and simply modified it for my binancefeeder test go program.

You’ve made it to the end of the blog post! Here’s the link to the Binance plugin if you want to see the complete code. If you want to see all of MarketStore’s plugins, check out this folder.

To summarize, if you want to create a Go extension for any open source repository, I would first read the existing documentation whether it is a README.md or a dedicated documentation website. Then, I would experiment around the repositories code by changing certain parts of the code and see which functions correspond with what action. Lastly, I would look over previous extensions and refactor an existing one that seems close to your plugin idea.

Thanks for reading! I hope you take a look at the MarketStore repository and test it out. If you have any questions, few free to comment below and I’ll try to answer!

Special thanks to Hitoshi, Sho, Chris, and the rest of the Alpaca’s Engineering team for their code reviews and help as well as Yoshi and Rao for providing feedback for this post.

By: Ethan Chiu

/

Algo Trading for Dummies  - 3 Useful Tips When Storing Trade Signals (Part 2)

Handling & Storing Trading Signals Are Hard

The calculation of simple trading indicators is made easy with the use of any one of the Technical Analysis libraries available, however, the efficient handling and storage of trading signals can be one of the most complex aspects of a live trading system.

 Photo by  Jeremy Thomas  on  Unsplash

Calculating Basic Indicators? No Problem

While it’s often necessary to create custom indicators and trading signals, there is still significant benefit to using a standard library such as Ta-Lib for the basics. This saves a lot of time rather than having to reimplement a set of common indicators in your language of choice. It also has the added bonus of increased processing speed as opposed to calculation done in native Python, for example.

When it comes to moving averages and other simple time-series indicators, the process is fairly self explanatory — at every time step you calculate the next numerical value which is then used as the most up-to-date signal to trade against.

(Code Snippet to read data CSV files and process into trading indicators) https://gist.github.com/yoshyoshi/73f130026c25a7dcdb9d6909b1990277

The signals themselves will be stateless in that respect — you aren’t concerned with previous signals that have been made, only the combination of indicators present at that moment. However, you may still wish to store some of the information from the indicators, if only for external analysis at a later point.

Different Story For Advanced Pattern Recognition

Meanwhile, more advanced pattern recognition cannot be handled in such a simple manner. If, for example, your strategy relies on finding divergence between indicators, its possible to get a significant performance boost by storing some past data-points from which to construct the signal at each new step, rather than having to reprocess the full set of data in the look-back period every time.

This is the trade-off between storage/ RAM efficiency and processing efficiency, with the latter also requiring greater software complexity to achieve.

How You Should Store Signals Depends On How Fast You Need It To Be

For optimal processing efficiency, you would not only store all the previously calculated signals from past time-stamps, but also the relevant information to calculate the next step in as fewer steps as possible.

While this would be completely unnecessary for any system with a polling rate above a seconds, it is exactly the kind of consideration you would have for a higher frequency strategy.

Meanwhile, a portfolio re-balancing system, or even most day-trading strategies, have all the time in the world (relatively). You could easily recalculate all the relevant signals at each time-step, which would cut down on the need for the handling of historical indicator sets.

Depending on the trading period of the system, it may also be worth using a hybrid approach to indicator and signal storage. Rather than permanently saving the data, you could calculate the full set of indicators at start-up and periodically dump and refresh the data to keep only whats going to be used in RAM.

The precise design trade-offs should considered on an individual basis, as holding more data in RAM may not be an option when running the software from lower power cloud computing instances nor, at the other end of the spectrum, would you be able to spare the seconds to recalculate everything for a market making bot.

3 Useful Tips When Storing Trade Signals

As mentioned in the part 1 of this series, there are range of different storage solutions that can be used for trading data. However, there are several best practices which apply across all:

  1. Keep indicators in a numeric or boolean format where possible for storage. For example, splitting a more complex signal set into boolean components. This particular problem caused me several issues in projects I’ve had to work on in the past.
  2. Only store what is complex or time-consuming to recalculate. If a set of signals can be calculated in time in a stateless manner, its probably easier to do so than add the design complexity of storing extra information.
  3. Plan out the flow of data through your system before you start programming anything. What market data is going to be pulled for each time-step? What will then be calculated from this and what is necessary to store? A well thought-out design will reduce complexity and hassle down the line.

Past this, common sense applies. Its probably best to store the indicators and signals in the same time-series format as, and along side, the underlying symbols they’re derived from. More complex signals, or indicators derived from multiple symbols, may even warrant their own calculation and storage process.

You could even go as far as to create a separate indicator feed script which calculates and stores everything separately from the trading bot software itself. The database could then be read by each bot as just another data feed. This not only has the benefit of keeping the system more modular, but also allowing you to create a highly optimized calculation function without the complexity of direct integration into a live system.

Whatever flavour of system you end up using, make sure to plan out the data storage and access first and foremost, before starting the rest of the design and implementation process.

By Matthew Tweed

/

Algo Trading for Dummies  -  Collecting & Storing The Market Data (Part 1)

The lifeblood of any algorithmic trading system is, of course, its data — so that’s what we’ll cover in the first two posts of the mini-series.

 Photo by  Farzad Nazifi  on  Unsplash

Always Always Collect Any Live Data

For the retail trader, most platforms and brokers are broadly the same, you’ll be provided with a simple wrapper for a relatively simple REST or Websocket API. It’s usually worth modifying the provided wrapper to suit your purposes, and potentially create your own custom wrapper — however, that can be done later once you have a better understanding of the structure and requirements of your trading system.

Depending on the nature of the trading strategy, there are various types of data you may need to access and work with — OHLCV data (candlesticks), bid/ asks, and fundamental or exotic data. OHLCV is usually the easiest to get historical data for, which will be important later for back-testing of strategies. While there are some sources for tick data and historic bid/ask or orderbook snapshots, they generally come at high costs.

With this last point in mind, it’s always good to collect any live data which will be difficult or expensive to access at a later date. This can be done by setting up simple polling scripts to periodically pull and save any data that might be relevant for back-testing in the future, such as bid/ask spread. This data can provide helpful insight into the market structure, which you wouldn’t be able to track otherwise.

Alpaca Python Wrapper Lets You Start Off Quickly

The Alpaca Python Wrapper provides a simple API wrapper to begin working with to create the initial proof of concept scripts. It serves well for both downloading bulk historical data and pulling live data for quick calculations, so will need little modification to get going.

It’s also to be noted that the Alpaca Wrapper returns market data in the form of pandas Dataframes, which has slightly different syntax compared to a standard Python array or dictionary — although this is covered thoroughly in the documentation so shouldn’t be an issue.

Keeping A Local Cache Of Data

While data may be relatively quick and easy to access on the fly, via the market API, for live trading, even small delays become a serious slow down when running batches of backtesting across large time periods or multiple trading symbols. As such, it’s best to keep a local cache of data to work with. This also allows you to create consistent data samples to design and verify your algorithms against.

There are many different storage solutions available, and in most cases it will come down to what you’re most familiar with. But, we’ll explore some of the options anyway.

No Traditional RDB For Financial Data Please

Financial data is time-series, meaning that each attribute is indexed by its associated time-stamp. Depending on the volume of data-points, traditional relational databases can quickly become impractical, as in many cases it is best to treat each data column as a list rather than the database as a collection of separate records.

On top of this, a database manager can add a lot of unnecessary overhead and complexity for a simple project that will have limited scaling requirements. Sure, if you’re planning to make a backend data storage solution which will be constantly queried by dozens of trading bots for large sets of data, you’ll probably want a full specialised time-series database.

However, in most cases you’ll be able to get away with simply storing the data in CSV files — at least initially.

Cutting Down Dev Time By Using CSVs

(Code Snippet to download and store OHLCV data into a CSV) https://gist.github.com/yoshyoshi/5a35a23ac263747eabc70906fd037ff3

The use of CSVs, or another simple format, significantly cuts down on usage of a key resource — development time. Unless you know that you will absolutely need a higher speed storage solution in the future, it’s better to keep the project as simple as possible. You’re unlikely be using enough data to make local storage speed much of an issue.

Even an SQL database can easily handle the storage and querying of hundreds of thousands of lines of data. To put that in perspective, 500k lines is equivalent to the 1 minute bars for a symbol between June 2013 and June 2018 (depending on trading hours). A well optimized system which only pulls and processes the necessary data will have no problem in overheads, meaning that any storage solution should be fine. Whether than be an SQL database, NoSQL or a collection of CSV files in a folder.

Additionally, it isn’t infeasible to store the full working dataset in RAM while in use. The 500k lines of OHLCV data used just over 700MB of RAM when serialized into lists (Tested in Python with data from the Alpaca client mentioned earlier).

When it comes to the building blocks of a piece of software, its best to keep everything as simple and efficient as possible, while keeping the components suitably modular so they may be adjusted in future if the design specification of the project changes.

By Matthew Tweed

/

So You Want to Trade Crypto - Hedging with Cryptocurrency and correlation structure (Part 6)

As a new asset class with historically low correlation to traditional financial products, many see Cryptocurrencies as a useful hedging tool against global downturns. However, the structure of Crypto volatility and correlation relative to market capitalization may prove somewhat detrimental to this use-case.

 Photo by  Tyler Milligan &nbsp;on  Unsplash

A story of Volatility

 (Raw data from  coinmarketcap.com . These charts show the mean of the 60 day annualized volatility from 1st Jan 2017 to time of writing.)

(Raw data from coinmarketcap.com. These charts show the mean of the 60 day annualized volatility from 1st Jan 2017 to time of writing.)

As within equity markets, we see a small decrease in volatility as the market cap of coins increase (albeit with a relatively low correlation). This can be likened to blue-chip stocks vs mid-caps, with the former providing greater stability due to their established dominance in their respective sectors.

Although market cap is a slightly misleading metric when applied to Cryptocurrencies, it at least implies a higher value to a coin - thus requiring more money to shift its direction dramatically. That being said, volatility has been higher across the board over the last couple of years as Crypto shifted from the accumulation phase post 2013 into the major bull run. Finally pushing to record high as we moved into the final phase of the bull run and subsequent bear market as we entered 2018.

This structure of volatility allows Crypto portfolios and indexes to be constructed similarly to those of equities: high-cap only selection for reduced risk and volatility; mid-caps for higher risk and reward; or a more diversified index to try to capture a middle ground.

The Trend of Correlation

 (Raw data from  coinmarketcap.com . These charts show the mean of the 60 day Pearson’s Correlation Coefficient against Bitcoin USD from 1st Jan 2017 to time of writing.)

(Raw data from coinmarketcap.com. These charts show the mean of the 60 day Pearson’s Correlation Coefficient against Bitcoin USD from 1st Jan 2017 to time of writing.)

Here we see nearly zero correlation between the market capitalization of a coin and its average correlation to Bitcoin (the historical leader of the Cryptocurrency space).

While this disproves the theory of high cap Cryptos holding a closer correlation to Bitcoin, it highlights the extremely high levels of correlation present throughout the market. This, as mentioned in previous posts, is likely due to the highly speculative and sentiment driven nature of the market, along with its relative immaturity compared to more traditional traded assets.

Interestingly, there isn’t much difference between the mean of correlation and the mean of absolute (positive only) correlation, meaning that we rarely see any negative correlation between ALT/USD pairs and BTC/USD.

Cryptocurrency as an Asset Class for hedging

Crypto holds the useful property of historically low correlation to other asset classes, such as equity and commodities, suggesting it to be a good hedge against external global factors. However, there are two main issues to this plan: Cryptocurrency has never weathered a global financial crisis; and the internal correlation within the Crypto space.

Since Bitcoin, and the rest of the Cryptocurrency market, has been experiencing its own market cycles due to its rapid growth over the past few years, any fluctuations due to correlation with equity markets has been almost unnoticeable - leading many to speculate that Cryptocurrency would continue this trend and make a good hedging tool against global downturns.

This observation happens to come on the back of a decade of huge growth in both US and global equity markets. Investors have been increasingly complacent in their gains over the past few years, and are happy to take greater and greater risks, betting money on more speculative assets such as Cryptocurrencies. However, such high yield assets are always the first to tumble at the onset of a recession, as investors scramble claw back their risk as their other positions drop.

Always "Different This Time"

Many will claim that its somehow “different this time” - it always is until the inevitable pullback. This was true of the dot-com bubble and I wouldn’t be surprised if the same fate will hold true for Cryptocurrency during a global dip. Not to say that Cryptocurrencies won’t be successful long term - the internet didn’t exactly disappear after 2000. But it should be approached with the same caution as any other high risk investment.

As alluded to in the first half of the article, the levels of volatility and correlation in Cryptocurrency make it difficult to create a well diversified portfolio - no matter what you pick you’re still at the mercy of Bitcoin and can incur the same volatility spikes and drawdowns.

While it may be possible to hedge a portfolio by shorting Bitcoin itself and creating synthetic ALT/BTC pairs, this won’t be able to eliminate the sensitivity of low-mid cap coins to shifts in market sentiment, so would have to be more actively managed.

All-in-all, Cryptocurrencies provide an interesting new opportunity for traders and investors alike - with high risk but much higher reward possibilities. They will not be a miracle financial product, nor a get rich quick scheme - but they can provide something truly new and different for those who have the time to understand and appreciate them.

By Matthew Tweed

/

I am a College Student and I Built My Own Robo Advisor

I’m Rao, and I’m an intern at Alpaca working on building an open-source robo advisor. I don’t have much experience in the space, and I had to find a lot of answers. While there’s a wealth of material available on the web, very little is organized. This post is the combination of various threads and forums that I read through looking for answers.

 “Yes, you an do it too!”

“Yes, you an do it too!”

What is a Robo Advisor anyway?!

Robo advisors are automated advising services that require little to no user interaction. They specialize in maintaining portfolios based on the investors chosen risk level. Btw, they were launched at the start of the financial crisis in 2008.

The actual logic for a robo advisor is straightforward.

  • Allocation” — Given a risk level, portions of capital are allocated to different positions.
  • Distance” — Over a regular time interval, the adviser scans to see if there’s a significant change in the portfolio balance, and
  • Rebalancing” — If necessary, rebalances.

Allocation:

The process of allocation is where different weights are assigned to asset classes based on the developer’s algorithm. These asset classes cover the various aspects of the market, and keep a diversified portfolio. Wealthfront sees these asset classes as:

  • US Stocks
  • Foreign Stocks
  • US Bonds
  • Foreign Bonds
  • Inflation Protection

Each of these assets is represented with ETFs, who capture a broad look at the asset class.

Distance:

Rebalancing a portfolio comes with a certain amount of overhead and cost. To avoid this, the robo advisor computes a “distance” vector. To trigger a rebalance, this distance will have to be greater than a given threshold. Most robo advisors usually have this threshold set to 5 percent.

The distance in this case, is the individual change in weight of each position. This is done by first calculating the current allocation. This is the value of a position against the total value of the portfolio. If any one position has a weight that’s 5 percent greater (current_weight/target_weight > 1.05), then the robo advisor will trigger a rebalance

Rebalancing:

Rebalancing is the act of buying and selling to return the portfolio to its target state. It’s important that all selling is done first. Once all cash available is gathered, necessary purchases are made.

To calculate how much to sell, use the target allocation of cash (weight * portfolio value), and see the number of shares that result from that cash. The difference between the number of shares currently held and the target level of shares is the amount to buy/sell.

Implementing a Robo Advisor with the Vanguard Core Series:

Now that we’ve established the various parts of what makes a robo advisor, we’ll start executing the steps to build one. We’ll build our robo advisor using Quantopian’s IDE.

When implementing our advisor, I find it easiest to do in steps. Quantopian’s IDE can only run backtests on a full algorithm. What that means, is there’s no “halfway” development. So, each step is a self-contained algorithm by itself.

Note: The implementation of Vanguard’s Core Series is using the information found here. The portfolio includes a 2% investment in a money market fund, but that isn’t available on Quantopian, so for the purposes of this tutorial, we’ll ignore it.

Risk-Based Allocation:

The first step, is to determine our universe, and assign weights using risk level. Our universe will be the Vanguard Core series:

  • VTI — Domestic Equity
  • VXUS — International Equity
  • BND — Domestic Fixed Income (Bonds)
  • BNDX — International Fixed Income (Bonds)

The risk level is actually the ratio of fixed income to equity, with equity being more volatile. So, a risk level 0 would have all capital allocated to fixed income (BND, BNDX), while a risk level of 5 would be 50/50. All 11 possible risk levels are catalogued in a dictionary, with the keys being the risk level, and values being tuples containing weight allocation:

risk_level = 5
risk_based_allocation = {0: (0,0,0.686,0.294),
                         1: (0.059,0.039,0.617,0.265),
                         2: (0.118,0.078,0.549,0.235),
                         3: (0.176,0.118,0.480,0.206),
                         4: (0.235,0.157,0.412,0.176),
                         5: (0.294,0.196,0.343,0.147), 
                         6: (0.353,0.235,0.274,0.118),
                         7: (0.412,0.274,0.206,0.088),
                         8: (0.470,0.314,0.137,0.059),
                         9: (0.529,0.353,0.069,0.029),
                         10: (0.588,0.392,0,0)}

The next step will be to implement our allocation in Quantopian. For this step, the only methods we’ll need are initialize and handle_data. The rest are superfluous.

The initialize method is the main. When a backtest is started, it automatically calls initialize. So, initialize will be the function that contains the dictionary for risk levels, as well as choosing the risk levels. The handle_data method is called automatically after initialize. It’s here that we’ll actually purchase the positions outlined in initialize.

Both initialize and handle_data have the variables context and data. Context allows the user to store global variables that get passed from method to method. Data helps the algorithm fetch different sorts of data. (Note: This is a change from Quantopian 1 where data was the object where global variables were stored).

Let’s get started. In initialize, copy and paste the following code:

context.stocks = symbols(‘VTI’, ‘VXUS’, ‘BND’, ‘BNDX’)
context.bought = False

risk_level = 5
risk_based_allocation = {0: (0,0,0.686,0.294),
                         1: (0.059,0.039,0.617,0.265),
                         2: (0.118,0.078,0.549,0.235),
                         3: (0.176,0.118,0.480,0.206),
                         4: (0.235,0.157,0.412,0.176),
                         5: (0.294,0.196,0.343,0.147),
                         6: (0.353,0.235,0.274,0.118),
                         7: (0.412,0.274,0.206,0.088),
                         8: (0.470,0.314,0.137,0.059),
                         9: (0.529,0.353,0.069,0.029),
                         10: (0.588,0.392,0,0)}
    #Saves the weights to easily access during rebalance
context.target_allocation = dict(zip(context.stocks,
                            risk_based_allocation[risk_level]))

    #To make initial purchase
context.bought = False

The variable context.stocks is a list of stock objects. The symbols function turns the strings into objects. The objects have attributes like current price, closing price, etc. Using that list, as well as the dictionary for allocation, we’ll create a second dictionary context.target_allocation. The keys are each ticker (VTI, VXUS, etc.) and the values are the ticker’s weights. This dictionary will be useful for both allocation and rebalancing.

Copy and paste the following code to create handle_data:

if not context.bought:
        for stock in context.stocks:
            amount = (context.target_allocation[stock] *
                      context.portfolio.cash) / data.current(stock,’price’)
            if (amount != 0):
                order(stock, int(amount))
                #log purchase
            log.info(“buying “ + str(int(amount)) + “ shares of “ +
                     str(stock))

         #now won’t purchase again and again
         context.bought = True

The variable context.bought refers to the value originally set to False in initialize. Since handle_data is called every time a market event occurs. So, using context.bought ensures the shares are only bought once.

To buy shares for each stock, the list of stock objects is iterated through. For each stock, the total number of shares are calculated by allocating the proper amount of capital (weight * capital), and then dividing that capital by the current price. Since we can only buy whole shares, leftover capital is added back to the cash on hand.

Lastly, for a smoother backtesting experience, all transactions are logged. The log is in the lower right hand corner. Build the current algorithm. Set the start date to June 2013 or later as the ETF BNDX was first tradeable on June 7, 2013.

Calculating Distance:

Before rebalancing a portfolio, it’s best to first calculate if the current portfolio is worth rebalancing. In more sophisticated robo advisors, this step would have a target portfolio calculated with an algorithm. For this example, our target portfolio is going to be the initial allocation defined by the risk level in initialize.

As the value of each position fluctuates, their portfolio weight may change too. The distance vector will be the change in weight of each individual position. Calculating the distance takes place in three steps:

  1. Calculating the current weights of each position
  2. Calculating the target weights (already done)
  3. Comparing the two weights

We want to check the balance daily, so we’ll go ahead and add the following function , before_trading_starts, to our code:

def before_trading_starts(context, data):
    #total value
    value = context.portfolio.portfolio_value +
                                         context.portfolio.cash
    #calculating current weights for each position
    for stock in context.stocks:
        if (context.target_allocation[stock] == 0):
            continue
        current_holdings = data.current(stock,’close’) *
                      context.portfolio.positions[stock].amount
        weight = current_holdings/value
        growth = float(weight) /
                        float(context.target_allocation[stock])
        if (growth >= 1.025 or growth <= 0.975):
            log.info(“Need to rebalance portfolio”)
            break

We first calculate total value. Since that value doesn’t change, it can be down outside the loop. Then, each position’s individual weight is calculated, and compared to the target weight (we’re using division since we’re interested in the relative growth of the position, not the absolute). If that growth exceeds the threshold (currently 2.5%) or is under the threshold (de-valued), then a rebalance is triggered. We don’t have an actual rebalance function written, so for now, we’ll simply log that a rebalance is necessary. It’s important to note the break added. Once the algorithm realizes that it’s time to rebalance, there’s no need to continue checking the rest of the stock. It creates a better best-case scenario.

However, we’re not actually done. We have to call before_trading_start, but calling it in a loop is inefficient. That’s why we’ll use the schedule_function command in initialize. Add this line of code to the end of the function block:

    schedule_function(
    func=before_trading_starts,
    date_rule=date_rules.every_day(),
    time_rule=time_rules.market_open(hours=1))

This schedules a distance calculation as the market opens every day. By computing the distance in the morning, we have the time and space necessary to execute rebalancing actions.

Rebalancing

The act of rebalancing a portfolio occurs in two steps. First, all assets that need to be sold are sold, and then all assets that need to bought are bought. Selling first makes sure that we don’t run out of cash.

The following is the code for a rebalance function:

def rebalance(context, data):
    for stock in context.stocks:
        current_weight = (data.current(stock, ‘close’) *
                          context.portfolio.positions[stock].amount) /
                          context.portfolio.portfolio_value
        target_weight = context.target_allocation[stock]
        distance = current_weight — target_weight
        if (distance > 0):
            amount = -1 * (distance * context.portfolio.portfolio_value) /
                     data.current(stock,’close’)
            if (int(amount) == 0):
                continue
            log.info(“Selling “ + str(int(amount)) + “ shares of “ +
                      str(stock))
            order(stock, int(amount))
    for stock in context.stocks:
        current_weight = (data.current(stock, ‘close’) *
                          context.portfolio.positions[stock].amount) /
                          context.portfolio.portfolio_value
        target_weight = context.target_allocation[stock]
        distance = current_weight — target_weight
        if (distance < 0):
            amount = -1 * (distance * context.portfolio.portfolio_value) /
                     data.current(stock,’close’)
            if (int(amount) == 0):
                continue
            log.info(“Buying “ + str(int(amount)) + “ shares of “ +
                      str(stock))
            order(stock, int(amount))

The selling and buying mechanics are the same. The only difference is when selling stock, you use a negative value when calling the order function. In both cases, we take the absolute difference in weight (target — current), and use that value to find the number of shares to buy or sell.

Multiple Universes

Vanguard provides several more universes beyond the core-series. Currently, we’re able to manipulate the risk level, and observe the outcome. Let’s add the functionality of also being able to select a universe.

The first and straightforward method, is to implement multiple dictionaries. Here’s how something like that would look. The CRSP series was added to our algorithm. The user now chooses both the universe and the risk level.

    core_series = symbols(‘VTI’, ‘VXUS’, ‘BND’, ‘BNDX’)
    crsp_series = symbols(‘VUG’, ‘VTV’, ‘VB’, ‘VEA’, ‘VWO’, ‘BSV’, ‘BIV’, ‘BLV’, ‘VMBS’, ‘BNDX’)
    #universe risk based allocation
    core_series_weights = {0: (0,0,0.686,0.294),
                           1: (0.059,0.039,0.617,0.265),
                           2: (0.118,0.078,0.549,0.235),
                           3: (0.176,0.118,0.480,0.206),
                           4: (0.235,0.157,0.412,0.176),
                           5: (0.294,0.196,0.343,0.147),
                           6: (0.353,0.235,0.274,0.118),
                           7: (0.412,0.274,0.206,0.088),
                           8: (0.470,0.314,0.137,0.059),
                           9: (0.529,0.353,0.069,0.029),
                           10: (0.588,0.392,0,0)}
    crsp_series_weights = {0: (0,0,0,0,0,0.273,0.14,0.123,0.15,0.294),
1: (0.024,0.027,0.008,0.03,0.009,0.245,0.126,0.111,0.135,0.265),
2: (0.048,0.054,0.016,0.061,0.017,0.218,0.112,0.099,0.12,0.235),
3: (0.072,0.082,0.022,0.091,0.027,0.191,0.098,0.086,0.105,0.206),
4: (0.096,0.109,0.03,0.122,0.035,0.164,0.084,0.074,0.09,0.176),
5: (0.120,0.136,0.038,0.152,0.044,0.126,0.07,0.062,0.075,0.147),
6: (0.143,0.163,0.047,0.182,0.053,0.109,0.056,0.049,0.06,0.118),
7: (0.167,0.190,0.055,0.213,0.061,0.082,0.042,0.037,0.045,0.088),
8: (0.191,0.217,0.062,0.243,0.071,0.055,0.028,0.024,0.030,0.059),
9: (0.215,0.245,0.069,0.274,0.079,0.027,0.014,0.013,0.015,0.029),
10: (0.239,0.272,0.077,0.304,0.088,0,0,0,0,0)}

    #set universe and risk level
    context.stocks = crsp_series
    risk_based_allocation = crsp_series_weights
    risk_level = 1

The user can use the three variables, context.stocks, risk_based_allocation, and risk_level, to set both the universe and the risk level.

What’s Next

Developing in Quantopian is a great experience — they provide lots of useful tools. But, it’s also limiting, being forced to only work in one file, etc. To build further on this robo advisor, I plan to migrate my work so far to the backend, and use Quantopian’s open source python library Zipline to run backtests. The next installment will cover all this and more!

You can find the algorithm here. Try it yourself!

by Rao Vinnakota

/