Modernizing MML for 2022 (1) Overture

part 1 / part 2 / part 3 / part 4 / part 5

I am starting a series of blog posts on my hacks I have worked on the latter half of this year.

I have been hacking on my MML (music macro language) compiler toolchain, which is now at atsushieno/mugene-ng. MML is a classic language and no one uses in "modern" production (at least beyond prototypes), but I believe that whether to use MML or not should not be about production capabilities, or from different perspective, "taste of music" (e.g. chiptune-alike), but should be just a matter of workstyles per composer i.e. personal tastes. And I do prefer text editors over DAWs, or maybe I should claim not entirely on DAWs.

I often talk about my MML compilers, but I have had a complicated feeling that I didn't achieve what I claim i.e. full music authoring experience with it to certain level (that I can feel satisfied), and that means, in practice, fully leveraging audio plugins' capabilities.

That once led me to build an experimental tool augene, which combines my MML compiler and Tracktion Engine (which is literally the audio engine for Tracktion Waveform DAW) with JUCE AudioPluginHost. I even gave a lightning talk about it at Audio Developers Conference 2019 (recording is not publicly available but slides I used are uploaded).

excerpt from the LT slides But it was nothing but an experiment - it had various problems that I had to resolve (details later) and I haven't authored anything with it after all. I was also primarily working on another project that I wanted to concentrate on.

To make it clear, I did use my MML compiler to produce a music CD before, but it was up to MIDI (SMF) level. As I explained on the post as well as on my slides, the final production on DAW was my manual work.

However, I have always been serious about my desire to make it possible to author music files using MML (or anything that can be edited under my productivity toolset). When I brought this tool back onto my table, it was important for me to use it in some real production. And you know, here it is:

atsushieno · [mugene MML + sfizz] Gustav Holst - The Planets: I. Mars, the Bringer of War. (2nd. edit)

This entire 7 minutes full orchestral composition of 31 tracks was compiled from a set of project files i.e. MMLs and audiographs, in only 2000+ lines of MML with plugin settings, without resorting to DAWs, like:

mars_sfizz1.mugene sshot The production MP3 and the intermediate Tracktion Edit file can be even downloaded from GitHub Actions builds. I assume no one in the world had achieved such an automated production for music that makes full use of audio plugins before.

augene-ng-production CI builds

I still do use Tracktion Waveform, for load and play to examine and/or verify the details (my toolchains lack visuals a lot), but my edits are all done on the MML on the text editors.

The authoring workflow basics

When we are dealing with a DAW, we compose music through a project (can be whatever term depending on the actual DAW) which consists of a handful of tracks, which can be either audio tracks or MIDI tracks or whatever (can be labeled whatever depending on the DAW). Let me simply call it a MIDI track when it accepts MIDI input. A MIDI track is typically tied to an audio graph which contains a sequence of audio plugins, where people configure an instrument and effects. A MIDI track also typically has a set of clips, and each clip contains a sequence of MIDI-like events. It can be visually understood, in a sense, if you have a look at *.tracktionedit file:

track on tracktionedit

To generate a Tracktion Edit file, we'd need MIDI-based sequences and audio plugin settings for each track. The MIDI sequences can be generated by the MML compiler from MML files. For audio plugins, each plugin has a set of parameters and a state binary chunk. Parameters are in general float values and they can be programmatically set by some means, but state binaries can be generated only by the plugin apps, so we have to resort to external programs. I end up to use JUCE AudioPluginHost application as the solution. It can instantiate plugins, chain them from audio and MIDI inputs to audio outputs, and then save them as *.filtergraph files. Then I edit my project file *.augene to enlist those *.filtergraph files. I will explain more details in the later posts, but for now, I show an example project:

<AugeneProject>
  <Includes>
    <Include Source="../Banks/SfzBanks.augene" />
    <Include Source="../Banks/SfzBanks-VPO3.augene" />
  </Includes>
  <AudioGraphs>
    <AudioGraph Id="sfizz_nbo_tuba_full" Source="../Banks/sfizz-nbo-tuba-full.filtergraph" />
    <AudioGraph Id="simple_reverb" Source="../Banks/simple-reverb.filtergraph" />
  </AudioGraphs>
  <MasterPlugins />
  <MmlFiles>
    <MmlFile>mars_sfizz1.mugene</MmlFile>
  </MmlFiles>
  <MmlStrings />
</AugeneProject>

Then I run augene-console project builder (compiler) which compiles the *augene project into *.tracktionedit which is the "edit" file format for Tracktion Waveform or Tracktion Engine. The tool can also generate a simple *.tracktion file that Tracktion Waveform can open as its project. Tracktion Engine does not need a project; it can directly load into tracktion_engine::Edit object.

augene-ng comes with a standalone AugenePlayer JUCE application which uses Tracktion Engine to play and/or export it to WAV so that it can be use among Free Software community people. It implements a hacky "Hot Reload" feature so that it does not have to iterate full plugin instantiation on all tracks, which can save significant amount of time.

What is unique in my MML to DAW compiler toolchain

Compiling MML into MIDI is not popular but not very unique at all. There are couple of such projects before (I assume they are mostly in Japan though). For example Text Music Sakura is popular here, it was once even on a CS textbook for some high school. I was not satisfied by any of the earlier solutions though - for two simple reasons: (1) they were mostly for Windows or MS-DOS, and (2) they were mostly closed source. That's why I ended up to create my own compiler.

My augene-ng compiler toolset has some notable features.

It features MIDI 2.0 UMPs: we can express per-note pitchbend and per-note controllers in the MML, and then import some of them as MPE content within Tracktion <NOTE> element.
It works with audio plugins: the Midi2ToTracktionConverter in our augene Kotlin module converts those plugin name specification via INSTRUMENTNAME MML operation to <PLUGIN> element via SMF Meta event
The converter also supports audio plugin parameter changes via certain system exclusive sequences, exported to *.tracktionedit as <AUTOMATIONTRACK> element and its contents.
The MML syntax enables us to define linear change of controls in "spectra" operators, so things like automation curve is kind of possible.

I will explain more about them in the upcoming posts, but so far I believe that covering those features make it possible to generate what we produce within MIDI tracks on those modern DAWs in 2021.