fuzzix dot org :: Perl, IO::Async, and the RtMidi event loop

Perl, IO::Async, and the RtMidi event loop

8 Feb 2023

I should prefix this post with a note: I am far from an event systems expert. This post describes some fumbling about and getting to grips with event-driven systems, with a specific focus on MIDI to act as an example

While I continue working (somewhat slowly) on the Perl binding to RtMidi, I have been thinking about the general problem of integrating heterogeneous event systems.

Event notifiers

A typical async event system operates using a set of file descriptors (fd), which may signal their readiness when data becomes available for processing. File descriptors may represent actual files, sockets, pipes, or even timers and generic system events. File descriptors can be gathered into a simple data structure, which is then passed to a system call like select, kqueue, or some variant of poll. When one of the descriptors is ready (has data to be read - is read-y), this system call returns details on one or more events to be handled. Implementation details will differ across event systems and platform, but for our purposes this is an adequate description of event notifier internals.

Where Unix breaks down

While "Everything is a file" is a powerful tenet of the Unix paradigm, it also happens not to be true. Many things are files, other things are signals. One may not select or poll for signals - they must be explicitly handled outside normal program control flow. DJB's self-pipe trick pattern offers a simple way for signals to notify the event system - a pipe is opened and watched by select and friends, then signal handlers write to this pipe. IO::Async::Signal provides signal handling within its event loop - as I say, implementations vary.

Other things that are not files

RtMidi provides its own callback-based event system, which we can see in use in this callback example from the RtMidi documentation. Unlike some other event systems, there is no explicit loop runner and there isn't much we might attach a fd to. We might set up a callback to write to an eventfd or pipe watched by the program, but the linked example is instructive for another reason. The program's "loop" is effectively doing nothing. A callback example in Python's RtMidi binding demonstrates a similar pattern.

Not only will RtMidi not integrate with your event loop, it probably won't even coexist peacefully alongside it. A callback could be triggered at any point in your program's flow, and control may not be handed back cleanly - your call stack and event system state may now be classed as "deeply unhappy". That is, to be ready to process a RtMidi callback, your program should probably not be doing much of anything.

A measured solution

Knowing all of the above, how may we exploit the characteristics of RtMidi and IO::Async to have MIDI input be simply another event notifier we can await or attach an async callback to? A common approach when there is a requirement to meld incompatible event loops is to give each loop its own thread, each with its own specialised event handling, with some communication between them to notify about specific events relevant to both loops (e.g. tell the GUI that 20% of a file has been processed).

An approach came to mind to allow the handling of events be programmed in a single event system (IO::Async in this case) - use IO::Async::Routine to start a process (or thread) which attaches a simple callback to RtMidi to pass all events back to the originating program via an IO::Async::Channel.

my $loop = IO::Async::Loop->new;
my $midi_ch = IO::Async::Channel->new;

my $midi_rtn = IO::Async::Routine->new(
    channels_out => [ $midi_ch ],
    code => sub {
        my $midi_in = MIDI::RtMidi::FFI::Device->new( type => 'in' );
        $midi_in->open_port_by_name( qr/LKMK3/i ); # LaunchKey Mk 3

        $midi_in->set_callback(
            sub( $ts, $msg, $data ) {
                my $t0 = [ gettimeofday ];
                $midi_ch->send( [ $t0, $ts, $msg ] );
            }
        );

        sleep;
    }
);
$loop->add( $midi_rtn );

The code block passed to the routine instantiates the MIDI device, opens a port on a connected MIDI keyboard, then sets a callback to send MIDI events from the keyboard back to the main program via the $midi_ch channel. We also gettimeofday to facilitate some instrumentation in the handler for the purposes of this example. Finally, the routine sleeps to allow RtMidi to effectively take control of it.

The handler may now simply await MIDI messages from the keyboard:

async sub process_midi_events {
    while ( my $event = await $midi_ch->recv ) {
        say "recv took " . tv_interval( $event->[0] ) . "s";
        say unpack 'H*', $event->[2];
    }
}

The first line inside the while loop outputs roughly how long it took to be notified of and receive available MIDI data on the channel. The second simply outputs the hex representation of the received MIDI message.

The final step is to add some work to the loop so we can observe that it is not blocked by any of the above, then set it running:

my $tick = 0;
$loop->add( IO::Async::Timer::Periodic->new(
    interval => 1,
    on_tick => sub { say "Tick " . $tick++; },
)->start );

$loop->await( process_midi_events );

Sample output:

$ ./callback.pl
Tick 0
recv took 0.000653s
903c29
recv took 0.00056s
903c
Tick 1
Tick 2
recv took 0.000302s
903e2c
recv took 0.000324s
903e
Tick 3
recv took 0.000271s
904012
recv took 0.000288s
9040
Tick 4
...

We can see the ticks from our timer continue to display, interleaved with processed MIDI messages. The time between send and receive is sub-millisecond - this was run on a not-especially-high-spec virtual machine, so timings on real hardware are likely lower. A sub-millisecond delay is far below the limit of human perception, though it should be noted that every bit of latency added to a realtime system increases the chance of overall latency being perceived - that is, it all adds up.

The MIDI message itself is displayed as hex octets. Without diving into the MIDI spec, let's take a look at the first two messages:

Message 1 '903c29' - The byte 0x90 means a "note on" message on MIDI channel 1. The byte 0x3c is the note that the "note on" event refers to - this translates to middle-C. The byte 0x29 is the velocity - how hard I pressed the key. This is a number between 0x00 and 0x7F (or 0 and 127). This message means I pressed the middle-C key on the keyboard fairly softly.

Message 2 '903c' - This looks remarkably similar to the previous message - we are just missing the velocity value. A "note on" message with no (or zero) velocity should be interpreted as "note off". That is, I took my finger off the middle-C key.

The code for this example is on Github.

Coda

I can easily imagine using an approach like this for a program which performs dynamic MIDI processing while it receives realtime instructions from a performer, updates status visually, logs the performance to file, and so on. This might be more difficult working strictly within the RtMidi event loop, or attempting to pass aggregated statuses between more complete, though separate, heterogeneous event handlers.

There are a number of cases where this approach breaks down. For example if the routine-ified secondary event loop was designed to process a lot of data, passing that back and forth over a channel could be bad for guaranteeing timely responses. A similar situation could arise if your event handling has heavy CPU-bound processing. For integrating event systems with simple messages which don't work in a way that your existing event system expects, I can see myself using this pattern in future.

(Currently playing: Mission of Burma - Signals, Calls, and Marches)