Ever wondered what technology lies behind Alive Drumming’s smart web services? Here’s the gist –
Alive Drumming‘s smart web services were engineered in Elixir. The web service supplies Alive Drumming’s fully arranged rhythm tracks constructed from multiple takes of long-form audio of very talented drummers.
- Parses the web request,
- Determines the arrangement structure of the requested track, then
- Creates the audio-engineering scripts to splice slices of the long-form audio into the result, and finally,
- Executes these scripts, delivering the resulting audio as the output of the service.
Language Compiler as Web Service
Much of this is textual lexical analysis and classic compiler design as the description of the track is a simple LR1 language, and even the ejected audio-engineering scripts are optimised with a peep-hole optimisation phase. Step 3 above, “Creating the audio-engineering scripts“, additionally involved parsing textual meta-data relating to the long-form audio’s location of differing drumming intensities, fills, pre- and post- fills, drumming breaks, pushes, count-ins and endings. Each of these has multiple ‘takes’ and algorithms apply weightings in pseudo-random selections. Initially, languages strong in textual manipulation came to mind with the early algorithms prototyped in GAWK, but it was clear a language suitable for massively scalable web-services was needed
Technology Stack – Phoenix / Elixir / Erlang / Linux / GCP
That lead me to Elixir and Phoenix. Elixir is a purely functional language, strong in text processing facilities that compiles to the Erlang/OTP virtual machine. Phoenix is a web server framework written in Elixir. Elixir/Erlang includes all the advantages of the Open Telecom Platform (OTP) – designed for the ultra maintainability and reliability expected of telecom platforms, such as supervised tasks and in-service module updates. We hosted this on Linux servers on the Google Cloud Platform (GCP) using a highly scalable cluster of Google Compute VMs and Google Cloud Storage.
Dynamic Programming – A performance boost
The programming solution leverages Dynamic Programming in many places. Dynamic Programming is where a problem is (recursively) decomposed into many sub-problems and where sub-problems may occur that are identical to previous ones. If a sub-problem is a duplicate, there is no need to solve it again, just use the previous result. A cache of results allows the algorithm to always check the cache first before solving the sub-problem. This technique was used on many levels within the solution. Elixir’s data-structures proved very accommodating with simple Elixir maps used as caches. Performance measurements showed an average 60% improvement in CPU utilisation and completion times with the caches deployed.
Some areas that deploy caches are
- At the highest level, the resulting audio file itself is cached,
- In the creation of audio slices, parameterised by their type and length requirements,
- The pseudo-random selection of instances of audio from a group of weighted alternatives – here passed selection is used as input to the algorithm as well
Design Caveat – Google Drive
An early design had the resulting audio track cached on Google Drive with the web server response being a redirect to the cached file. This solution was appealing because it leveraged the very mature, scalable and cost-effective Google Drive. However, it proved problematical with Google Drive quickly applying a governor under quite small amounts of load. It turned out Google Drive was not a good fit for this type of service. If all the Drive requests came from the web service, throttling was applied that limited the service. Additionally, it was difficult to supply Google Drive URLs to clients in a way that would reliably not result in authentication requests. Google Storage was eventually used in place of Google Drive.
Summary – a great technology stack
Elixir proved both a perfect fit for this project and a real pleasure to work in. The tooling around the language is mature and fit for purpose, the OTP platform is the best solution for reliability and maintainability and the restriction of a purely functional language proved to be more of a benefit than a limitation. Less really can be more with programming language design. The Google Cloud Platform makes infrastructure commissioning, monitoring and maintenance so much easier than dealing with physical hardware. GCP toolset has matured and the facilities are extensive. We would fully recommend the Phoenix / Elixir / GCP technology stack for developing and deploying mission-critical, complex web services.
Also published on Medium.