It is around the end of the year 2024, so it's time to summarize up what I have done this year, just like I had been doing in the past. I'm primarily writing this for myself to record history.

2024 was a year that I got various work requests and I became much busier than last couple of years. On the other hand, I completely stopped managing local audio dev. community and interact with much less people in the worldwide audio domain. Nowadays what I have been working on is more interesting than them, to me.

Books

There was much less public development activity this year. What I had been spending time instead was, writing tech books in Japanese. Two are already published at my store at techbookfest.org. If you are curious about techbookfest, I have written a blog post about that many years ago. Now they have online store for ebooks. It's not like commercial publishing; take it more like Comic Market for programming books.

MIDI-CI Guidebook was literally about MIDI-CI. There was no book in the world that explains what MIDI-CI is about (maybe not even what MIDI 2.0 is; I wrote one about UMP back in 2020, and last year for June 2023 revamps). It is mostly based on my ktmidi development (I have written a detailed blog post about that in January: https://atsushieno.github.io/2024/01/26/ktmidi-ci-tool-released.html and https://atsushieno.github.io/2024/01/26/midi-ci-tools.html ). The relevant MMA/AMEI specifications (MIDI-CI core, Common Rules for PC, and Common Rules for PE) are of a few PDF documents that sum up to about 200 pages, and more if we count individual specifications such as GM profiles. This book compresses them into about 100 pages (in Japanese). The book will become more useful after a few years later, maybe not now.

Some folks told me that I should publish them in English, but haven't spent time on that.

"The Ideal Audio Plugin Format" is more like a collection of thoughts on audio plugin technology through my attempt to develop a unified plugin hosting foundation (WIP). The idea was born while I was writing another book that includes audio plugin host development where I will mostly just explain the plugin APIs. This book is totally different and I thought I should write what aspect of specific plugin formats are good and/or bad, where their technical choices have advantages and/or defects. For example...

  • VST3 and CLAP chose to require host to invoke their functions on the GUI (main) thread while AU does not. I describe it like VST3 and CLAP are single-threaded COM while AU is multithreaded COM. Technically AU is superior, but it'd be easy for plugin developers to just write single-threaded COM just like VisualBasic developers did (and that's kind of rational).
  • Fast plugin scanning; LV2 and AU, and VST3 after 3.7.6 are good as they can only parse metadata, while CLAP and VST3 prior to 3.7.5 still require loading of the DLL for just scanning (it was intentionally done at CLAP in favor of single source of truth).
  • On plugin instancing lifecycle, I mentioned LV2 instancing is annoying because its sampleRate must be given at instancing; unlike other plugin formats, changing sample rate after connecting requires re-creating plugin instances.

AAP and ktmidi for MIDI 2.0 updates

Unlike last year when I have achieved a bunch of new features, there was surprisingly not a lot of AAP development this year. There are handful of reasons for that, but the biggest reason was that I spent a lot of time on some part time work on Android audio stuff, apart from my primary jobless. It is still ongoing, and another book writing is still ongoing, and I can only spend the rest of my time for coding.

There was still a big new feature addition in the latest AAP v0.9.0 release: AAP plugins can now act as software MIDI 2.0 devices!

It is actually dependent on the new Android 15 feature: android.midi.MidiUmpDeviceService provides the overall foundation for UMP-based software MIDI devices, and I just adopted our "existing" MIDI 2.0 device service support (I have done that years ago; AAP MidiDeviceService sent and received UMPs with some protocol negotiation hack).

MIDI 2.0 device access

But. I had to question to myself: what can we do with this? What we clearly need is some playground for MIDI 2.0 devices. My Resident MIDI Keyboard already supported MIDI 2.0 UMPs (after the blog post I have fixed some issues to make sure that things really work well), but that's not enough. At least I want to "play some MIDI 2.0 based song files".

But there are too many missing bits for that. First of all, we don't even have MIDI 2.0 song files. I had been working on my MML (music macro language) compiler that supports MIDI 2.0 song file (it's like SMF but for UMP, in my own format, as MMA hadn't published any specification for SMF2 Container File format). And we don't have a decent MIDI 2.0 song file player.

That led me to rather work on ktmidi software stack than AAP stack. As an early adopter like me, 2024 was fairly a good year to start working on MIDI 2.0 device access support. Ubuntu 24.04 LTS was released with Linux kernel post-6.5 version that came with ALSA MIDI 2.0 support, and as I mentioned, Android 15 started to support UMP software devices. ktmidi had the most advanced software stack for building UMP and MIDI-CI support, but as its MIDI device access had been stuck around MIDI 1.0 APIs, its usage was limited.

And managing platform API access is quite tough area in Kotlin Multiplatform libraries - we have to provide platform bindings for JVM, Android (note that it is different from JVM), native (for each platform!), JS, and now WebAssembly. The most wanted areas are UMP support on JVM and Android. Supporting Android is easy, but JVM is quite annoying. If it were about native, cinterop support for CoreMIDI would not be hard. But on desktop, Compose Multiplatform apps run on JVM and there is no cinterop tool. ALSA support was another source of headache, as it was kind of special that we have to bundle libasound.so that may not blend with locally installed ALSA (especially at config sections), and bundling native libraries do not always work (like, Panama and jextract tool do not support that kind of packaging). I ended up buidling libremidi JavaCPP wrapper for MIDI 2.0 I/O support. It does not seem to always work yet, but on some environment I could finally send and receive UMPs.

KMDSP

Once I got MIDI 2.0 I/O working, I wanted to implement "MIDI 2.0 player" kind of stuff. One last missing piece I didn't bring from C# managed-midi project was a visual MIDI player called xmdsp. Since Kotlin 2.0 stable and the first stable version of Kotlin Multiplatform was released this year, it felt like a good time to implement it in Kotlin. It was done shortly:

https://github.com/atsushieno/kmdsp

... and even better, now it runs on the web browser (using Kotlin/Wasm):

https://atsushieno.github.io/kmdsp/

It works fairly well with MIDI 1.0 protocol. But what we want now is MIDI 2.0 music player. While it is able to send UMPs to MIDI output devices, we don't really have MIDI 2.0 devices. It is probably not a big deal as UMP outputs from those MIDI 2.0 apps will be translated to MIDI 1.0 messages if the connected device is MIDI 1.0. But then it does not make sense to use MIDI 2.0 there. What I want instead is to be able to play "MIDI 2.0 music" over a "MIDI 2.0 device" which has more than 16 channels. Now I'm exploring that kind of fields, but I need more time to achieve something useful.

AAP GUI frontline

It seems really disastrous that there was no accomplishments on AAP 2024 roadmap items. But as I described above, MIDI 2.0 UMP device work had disrupted and took place there. I needed too improve the overall ecosystem in other parts.

And on AAP itself, I was stuck at the first item I tackled. As it became possible to get in-plugin-process UI working last year already, I continued to get more plugin UI stuff working, namely aap-juce plugin UIs. However, apart from the fact that JUCE is not quite ready for non-standalone apps, there are still some blockers that fails to show remote UI via SurfaceControlViewHost. I got some plugin UI showing up, but its UI interaction does not work. I don't spend much time on things that are not fun to investigate. JUCE desktop plugin UI does not contribute much to AAP either. Their UI should respect mobile in the first place.

When juce_gui_basics die and they switch to newer UI technology like Web UI, things may become easier for porting and I'm looking forward to it. I myself also have been exploring WebView UI integration on desktop audio apps in my latest experiments. When I worked on that kind of stuff a few years ago, I was so minority, but it seems different nowadays.

On iOS side (AUv3) things are more like, desktop and mobile should not share the UI bits, and the same kind of trend would work for Android. I'm not hoping against code sharing, but redesigning UI code for diversity will cause rethinking the UIs, dumping obsolete technology for new ones.