Saturday, 22 December 2018

Google Developer Group MK at Bletchley Park

The Milton Keynes GDG hosted their December meetup at The National Museum of Computing inside Bletchley Park. We had a detailed demonstration of how the WITCH Harwell Computer operated, a talk about building serverless machine learning applications in Google Cloud, followed by a private tour of the museum and mince pies. 100% worth the drive.

Tuesday, 27 November 2018

Notes on Reinforcement Learning and Machine Learning for Cryptocurrency Trading

Last week I gave a talk at the Google Developer Group in Reading about some work I've been doing investigating the possibilities around using machine learning to drive an automated trading platform.

The talk walked through the basics of reinforcement learning, technical analysis and the basic ideas behind trading for a profit and some ideas around using technical analysis in feature engineering for the model and how to architect a system as a collection of microservices processing messages from PubSub to predict on live data and execute trades.

One of the key problems touched on was how an unfiltered stream of data from the trading platform means that trying to potentially classify trends as 'good' or 'bad' means that small mistakes incur the costs of buying and selling and the bot can end up thrashing it's account balance away.
But the main problem that makes reinforcement learning ineffective when employed in a straightforward manner to automated trading is that RL learns that an action causes a transition between some state to another, which is why memories are stored as State, Action, Next State, Reward. The agent however, is never actually affecting the state of the system. The state is the same whether we buy, sell, do nothing, or close a trade that's successful or otherwise.
As such, because of the limited number of actions possible, the training can be unrolled into a simple standard learning problem where you potentially predict the 'reward' of an action on a specific state.

After picking apart the data and using principal component analysis, I saw that there was so much overlap between the states where it was profitable and unprofitable that they were indistinguishable in the dataset that I examined (ETHBTC for the previous 12 months at 1H candles, using a variant of crossover as the entry points and looking to sell around 10 hours later).

It makes more sense in retrospect, that this problem would be better approached by finding a passable simple strategy and then using machine learning to try and reduce the number of unsuccessful trades. However the process was useful in learning how to engineer features, construct models, run them in production, and the techniques to assess their accuracy and suitability.

Saturday, 13 October 2018

Google Cloud Next, London 2018

At Hu:toma we're very much all in on Google Cloud, so it was great for the engineering team to be able to head over to Cloud Next this week and find out what's new. I somehow through random chance managed to get an unobstructed view from the second row (first row was reserved for Googlers) for the keynote, which made up for the obscenely early train to attend.

The venue was as meticulously decorated as you'd expect, with an upper floor demonstrating google products and accommodating talks, while the downstairs provided more of a breakout area, more vendors and the keynote room. It was interesting to see a TPU up close, albeit a much older one than is current.

We did compete to see who could pick up the most random swag, but a colleague ran away with it. There was a great demonstration from Thompson Reuters on pulling tick data and doing some simple filtering on it, and walkthroughs of some improvements to Tensorflow that improve the interface and walk it towards the user friendliness of Keras. On the second day I didn't feel like getting the pre-drawn train again so had a slower wander in to enjoy the improved weather.

Friday, 24 March 2017

Changing a Tail Light the Hard Way

At some point recently, the indicator on my dashboard started complaining about a tail light problem, specifically that the bulb was gone. Pretty standard, right? Except it was intermittent. I'm not especially mechanically minded, and I try not to mess with my car too much in case I do more harm than good.
However, half an hour of searching later it looked like it was going to be a little more serious than a simple loose connection, and I went through the process of taking everything out and unplugging the actual tail light so I could see what the problem was.

It turns out that the seal on the tail lights isn't great, and it's not uncommon for water and general moisture to get in, and can get bad enough that it gets into the socket and ending up something like the above. The metal pin used for the corner socket had completely disintegrated from the inside of the tail light once I unplugged it.

Since at least one of the pins in the light was gone and others looked to be going soon, I ordered a new tail light, which was fairly reasonably priced, and sourced a new plug like the above.

Or I thought I had. They looked identical but since I couldn't determine a part number I assumed that ordering something that looked the same off eBay would solve the problem. On the plus side I have another spare for whenever whatever that plug type is used for encounters a similar problem. Another attempt however was successful after finding the part number hidden on a forum somewhere.

I slowly transferred the wires from the broken plug to the new one to avoid mistakes with wires going into the wrong place. It needed a small tool to pinch the wires out of the previous plug so that they could be moved over.


Friday, 5 August 2016

A Docker Experiment

Containers are a topic that have been rising in occurrence for me, for the last year or so. From using them as part of the architecture at work or for various pet projects friends have been working on. I figured it was time to experiment myself and get to grips with what people have been talking about.
It seemed like a good idea to find some introductory material that would give me an overview of its uses without delving too far into the details so I could see what it actually was, so I found a PluralSight course about Docker, specifically “Docker and Containers:The Big Picture”. This gave a nice overview of what Docker is, and more importantly what it was trying to achieve and how to use it.
With a bit more of an understanding, I wanted to use it for something, preferably something familiar. I decided to try setting up ElasticSearch and Kibana containers, where Kibana would visualize the ElasticSearch data.
I used bits of this article along the way as a guide, if you'd prefer a more detailed reference:
If you're on Windows 10 you might have a slightly different experience as you're able to use Docker for Windows. The machine I had available is on Windows 8 so I used the Docker ToolBox, so this post will assume you're doing similar.
At installation, if you don't have Hyper-V you'll want to tick the box for installing VirtualBox as you'll need one or the other, you can also useVMWare, etc. but I'm using Hyper-V. If you've not used virtualisation on your system before you may need to turn on something like VT-x to continue. You will also require a virtual switch. If you're a Hyper-V user on a laptop you might want to read on a bit first, as I encountered a problem here.
With that installed I opted to use PowerShell to continue. Mostly because it looks better in screenshots. It needs to be elevated to Administrator to use Hyper-V.
With setup out of the way you want to choose a name for your docker machine and run something like:
    docker-machine create --driver hyperv elkmachine

If you're not using Hyper-V you'll need to change the driver to whichever you've decided to use, and replace elkmachine with your chosen machine name.
This will create a docker virtual machine that you can see in the Hyper-V Manager.
You may run into a problem where it seems to hang at 'Waiting for host to start'. I found this when I attempted it on my laptop initially, and came across this post:
Simply put, a standard virtual switch won't work, and you need to create an internal virtual switch and in the properties of your network connection, share the connection with the WiFi virtual switch. In addition to the article, if you have a regular virtual switch and a WiFi virtual switch, docker create might default to the first one. This could be the order that they're created in but I didn't test it, and just removed the external virtual switch as it probably wouldn't work anyway.
With that sorted, you should see something like this:

If you look in Hyper-V now, you'll see a virtual machine with the docker machine name with some boot2docker text in the window.

The docker machine's now been created, we need to use set env to create containers on it.

As the terminal says we need to run:
    "C:\Program Files\Docker Toolbox\docker-machine.exe" env elkbox | Invoke-Expression
Somewhere around or before this point you'd want to decide what container you want to run, there's an excellent resource of existing containers for most mainstream software at There are other alternative sources, but lucky for us, both ElasticSearch and Kibana have official containers here. Before we can use the containers we need to 'pull' them. This is on a per-machine basis, so each of your docker machines will need to pull them separately, but within a machine once you've pulled a container it's usable.
First we want to pull ElasticSearch:
    docker pull elasticsearch
Following that we want to run the container, with some flags specified, the command we will use is:
    docker run -d -p 9200:9200 -p 9300:9300 elasticsearch
The -d will run the container in detached mode, particularly useful with ElasticSearch which will otherwise block the terminal.
-p maps a container port to a host port. In these cases we want the mapping to be the same but this is useful for duplicate containers.
Once this is done we can check its status with "docker ps", you can see the result of the previous commands below.

We now have an ElasticSearch container running, but we should check on it further. To hit it we need to know the IP address of the docker machine. We can do this by using:

    docker-machine ip elkbox

When we ran the container we set ports 9200 and 9300, so with the IP address and the port, we can hit ElasticSearch from the browser and should get some JSON back to let us know it's there, like so:

So, we know ElasticSearch is ready to go, now we want to set up a Kibana container and point it at ElasticSearch. Like ElasticSearch, we need to pull the container, which is simply:

    docker pull kibana

After some fiddling I found the best way to run the Kibana container was with this command:

    docker run –name kibana -e ELASTICSEARCH_URL= -p 5601:5601 -d kibana

You can see once again we're mapping some ports, and we're using one of the containers parameters to set the URL of ElasticSearch for it to use. Once again, we can test it easily as we know it's at the same IP but on port 5601 as we specified in the run command. And in the browser you should see something like:

Now it's just a case of pushing your data to ElasticSearch. I'm finding this approach preferable to my previous way of running these 2 services in command prompts, since I don't have to have a bunch of windows open and I can stop the docker machine and not have to worry about remembering specifically which services I've started as they all belong to the same docker machine. 
It was also quicker to get going immediately even with the docker installation when considering installing Java and potentially other prerequisites. I'll be doing some more experimentation around this soon.

Monday, 13 June 2016

Self-Producing an EP Pt.6: Mastering

While everything was being mixed into the same chain of plug-ins individually, we did at least know that mastering should be a clearly defined separate step. Again, we considered getting it done professionally, but sort of figured if we've made a mess we may as well roll with it, and if we haven't then maybe this won't go so horribly either.
It didn't take long to get this part done comparatively, maybe a few evenings. Having tried out a large amount of presets from various plugins I had a good idea how I wanted it to sound. All the mixed tracks were rendered, aiming for their peak to be around -6db, and dropped into one project. The preset from the tracks was applied, and then a run-through to listen for consistency in volume, sound, etc.
Oh dear. Two of the tracks appeared to have almost no bass in them, while the other three were quite consistent with each other. That had to be fixed and took an evening to identify and correct some high-pass filtering that was too aggressive on the guitars, and low bass guitar volumes. They'd sounded fine on their own, but stood up next to the others were clearly lacking. There was also some fizzing making one of the tracks sound quite bad too, which also got cleared up. Then it was a case of getting the mastering plugins right to allow for the right volume when compared to a reference track, and then varying each individual songs output volume to match the perceived volume.
I'd like to say that was it, but after we had a few reviews of all the tracks we had a handful of issues that were going to need adjusting. Chorus vocals slightly too loud in one place, bass guitar too quiet in another, and a couple of others that required some mix changes, and then tweaking the mastering again to get the consistency back.

Things that were important:

  • It's good that there was a (after that initial bit of repair) consistency across all the tracks, I think the effects chains I made mixing the first song helped a lot here, and having a mastering pre-set applied.
  • Doing them all in one project, as you'd expect, helped put them all together.
  • Using a reference track to check where the perceived volume was good.
  • We didn't destroy all of the dynamics, which we said we wanted to be the case from the start.
  • Maybe get someone else to do it next time, but we'll see.
  • Try and identify the handful of mixing problems before this point so there's no painful back and forth of rendering new mixes.
  • It doesn't sound amazing in mono. It was referenced in mono a fair bit but it's heavily reliant on stereo for space. I'm not sure this is much of a problem but worth considering.

At this point the audio part is mostly complete, but with the intention of making a 'radio edit' as it were of 1 or 2 of the tracks (trim some extended sections and make sure it works in mono, probably) to submit to various places there's still a bit to be done. There is of course now all the other stuff that you need to do. Photos, a logo, all the social media stuff, web-sites, uploading for streaming, and attempting to book gigs. I might post about those when I've actually done them.

Google Developer Group MK at Bletchley Park

The Milton Keynes GDG hosted their December meetup at The National Museum of Computing inside Bletchley Park. We had a detailed demons...