Porting my Python 2.7 Flask app to NextJS

A couple of years ago I built a fun little project that took the approach I used to transcribe MIDI files into the structure used by Little Sound DJ (LSDJ), a tracker for the Nintendo Gameboy.

I used Python as I found a library for parsing MIDI files called python-midi and at the time I was working mostly with Python so it was easier for me to wrap that MIDI library in first a CLI and then finally a Flask app to return a serialised map of the Chains and Phrases that make up a song in LSDJ that was rendered in a basic React web-app.

Unfortunately that python-midi library was written against Python 2.7 which was finally discontinued, so the service I used to host the app — Heroku, stopped supporting new deployments of Python 2.7 services.

During the time since I wrote that Flask app I ended up being very busy and not making music so I had no immediate need to solve to the problem of upgrading the library or porting the app, but in October 2022 I started a three month sabbatical and set myself an aim to produce a 5-track cover EP of some of my favourite Technical Death Metal songs, made solely on the Gameboy.

Porting my MIDI transcribing logic from Python to TypeScript

After a quick search for a Python MIDI library I didn’t see an immediate candidate that I could swap out python-midi for and get my Flask app running again so I decided to port the transcription logic to TypeScript as the NodeJS ecosystem had a number of libraries.

I first tried jzz and jzz-midi-smf but I found the development experience quite painful, with the need to create a number of instances of various classes and with the interface being so messy I had to rely on the documentation which seemed to cover everything but the one use case I had of parsing a MIDI file instead of writing one.

After a bit more experimentation I eventually settled on midi-file , a library which is far easier to use as it takes a File Buffer and gives you the MIDI header and an array of tracks containing all the MIDI events (for which it has a really nice set of types to help with understanding each MIDI event’s properties).

The Good

Moving the code over from Python 2.7 to TypeScript allowed me to add a bit more context to the codebase by adding type information to the different objects that are passed around as the LSDJ structure is pulled together from the list of MIDI events.

As the code hadn’t been touched in over three years I had forgotten a lot of the reasoning for the logic in the application and because past me didn’t write any tests around the logic I relied heavily on the types I could infer as I performed code archeology to build up a sense of everything.

I broke the new MIDI transcribing logic into a library (published on npm as midi-to-lsdj) that I could use within an API which would then need to handle file uploads for the input and returning the output. This worked well as it separated the concerns which made understanding the transcription steps and testing each step easier.

The Bad

A core part of my transcription logic is quantising the MIDI notes into semi-quavers (16th notes, the smallest note duration in LSDJ) which in Python is really easy to do as the range() generator has a step argument which can be used to create a new list of values at that step between two numbers. JavaScript doesn’t have this functionality out of the box.

I used the function below to handle this functionality for my needs and it did the job, but in order to make use of it in my code I ended up turning the generator into an Array which essentially negated the benefit of using a generator.

Copycopy code to clipboard
1export function* range(start: number, stop: number, step: number = 1): Generator<number> {
2    if (stop == null) {
3        // one param defined
4        stop = start;
5        start = 0;
6    }
7
8    for (let i = start; step > 0 ? i < stop : i > stop; i += step) {
9        yield i;
10    }
11}

Another issue I faced with setting up the quantisation was how midi-file used a delta ticks instead of the absolute ticks that python-midi used. This meant that in order to know the correct tick the event triggered at I needed to keep a running total as I iterated over the MIDI events so I could plot the note at the correct 16th note tick.

The ugly

After the quantisation process, the second most important part of how I transcribe MIDI to LSDJ is using time signatures to know how many of those semi-quavers I have in a bar (Phrases in LSDJ have 16 notes, so a time signature like 9/8 will need 18 notes split across 2 Phrases).

Time signature events in MIDI if they follow the specification have a denominator as a reverse power of 2 (e.g. to get a quaver (8th note) the denominator would be 3 as 2 to the power of 3 is 8) and midi-file will process the file as such but more often than not the MIDI files I had would have the denominator at the resolved value (i.e. 8 instead of 3, which would be 256) which caused me a number of issues.

I think the cause is Logic Pro which is what I use to create the single track MIDI files I transcribe but because I’m building the app for my own consumption I decided to cater for this need and delete the reverse power solving logic I created. I’m not sure how python-midi processes denominators as I only ever used it with those Logic Pro generated MIDI files.

Copycopy code to clipboard
1export function getTimeSignatureinSemiQuavers(timeSignature: MidiTimeSignatureEvent): TimeSignatureValues {
2  const { numerator, denominator } = timeSignature
3  const deltaFromSemiQuaver = 4 - denominator
4  if (deltaFromSemiQuaver > 0) {
5    return Array(deltaFromSemiQuaver).fill(0).reduce((agg) => {
6      return {
7        numerator: agg.numerator * 2,
8        denominator: agg.denominator * 2
9      }
10    }, { numerator, denominator: 2**denominator })
11  } else if (deltaFromSemiQuaver < 0) {
12    return Array(Math.abs(deltaFromSemiQuaver)).fill(0).reduce((agg) => {
13      return {
14        numerator: agg.numerator / 2,
15        denominator: agg.denominator / 2
16      }
17    }, { numerator, denominator: 2**denominator })
18  }
19}

Porting the Flask & React app to NextJS

Once the core transcribing logic was moved into the midi-to-lsdj library I then moved my focus to the web-app side of the project. I had originally built the front-end in React so I decided to use NextJS for building the new web-app as I could use React throughout the app and use it’s API route functionality for the API that the front-end would call hen a MIDI file is uploaded.

The client-side code needed a little bit of an uplift because 2019 is a long time ago in JavaScript. I had built the front-end using redux & redux-saga for state management which has now been overtaken by hooks in React so in order to keep the code ‘modern’ and to remove a lot of the fluff that came with redux I ported the state to use a Context that could be consumed where the Higher Order Components were in the original code.

I also stripped out a lot of the display logic from the front-end as the original app with it’s ill-defined contextual boundaries ended up shifting a lot of value conversion (LSDJ uses hexadecimal values) and creation of iterables to the front-end when really the data returned from the API should have been structured and marked up properly.

With the front-end now essentially just handling the file upload and iterating over the returned values, the conversion from redux to hooks was relatively easy.

I did struggle a little with the API route though. This was my first time building a web-app with NextJS so I wasn’t familiar with the way the request parsing works. Technically I could have just spun up an express server and delegated the route to that but I wanted to try and use NextJS out of the box.

The issue lied with reading the uploaded MIDI file from the request. I could see the file in the request body but found no means of extracting it from the form data and every tutorial I found for working with file uploads just used a library called formidable to do this. I ended up using formidable in the end but I think the interface for the library has changed a couple of times as there were three separate implementations for extracting a file across as many blog posts.

Overall NextJS made it really easy to port the project over and get it deployed (Vercel, the makers of NextJS have their own deployment platform) and left me to focus on the hard part which was re-working the legacy code.

Summary

Over the last week or so I’ve been able to revive an old project, porting it from Python 2.7 to TypeScript and added measures to hopefully avoid similar issues with understanding the context of things when I pick it back up again in the future.

The midi-to-lsdj library is open source and I’ve created a roadmap for advanced transcription functionality which I’ll be working through every so often as I work on more complex songs. If you find it useful and want to contribute then get in touch.