fuzzix dot org

Revisiting Async and the RtMidi Event Loop

25 Feb 2026

Introduction

A coupla few years ago I wrote about Integrating RtMidi's event loop with IO::Async. The basic approach involved spawning a routine which set up a RtMidi callback to pass MIDI messages back to the main process via a channel. This routine then went to sleep to allow RtMidi's own event loop to take control.

While this approach works well enough, and has been used in a number of projects (such as The MIDI device filter project on this site, and Gene Boggs' MIDI::RtController ecosystem), it also adds no small amount of complexity and overhead to an implementation. Running Perl-based callbacks within the rtmidi callback-handler thread can also cause crashes, if not done carefully.

While playing around with libremidi, I noticed it offers an option which makes a nonblocking file descriptor available for integration into your event loop, and found myself thinking that it would be nice to have RtMidi offer a similar facility. What if...

What if we don't set RtMidi's callback to a FFI::Platypus::Closure wrapping our Perl code, but instead set the callback within some bundled C code? This callback would create a fd pair - two file descriptors attached to either end of a pipe, a write fd and a read fd. The read fd could be set nonblocking and, as it's a simple int, be trivially passed back to our Perl program for later integration into any event loop. This would eliminate the problems of calling Perl code in new threads, and should simplify event loop integration - no more routines and channels required, we can now process MIDI bytes as a stream.

This seems sound enough to me, but we are now left with the problem of bundling C - Makefiles, compilers, actually writing C ... We are surely in for a world of pain, or are we?

Bundling Native Code with FFI::Platypus

FFI::Platypus' bundling interface allows for seamless inclusion of C code in your distribution (or Go code, or Rust code, or Zig code, or...), without needing to worry about the build system too much. The moving parts in our case are:

This last part is pretty simple, the following line is added to dist.ini: 

[FFI::Build]

Let's take a look at the C.

Creating the pipe

Let's start with the includes. You'll always want ffi_platypus_bundle.h for bundled FFI code. There are some libc parts, then rtmidi_c.h, RtMidi's C interface header. This will be useful for dereferencing RtMidi device members, as we'll see later. 

#include <ffi_platypus_bundle.h>
#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <rtmidi_c.h>

Next comes some cheating ... while this is ostensibly C, we will need to build and link with a C++ compiler to access RtMidi's namespaces correctly. This is as simple as giving our file a .cpp extension (ffi/pipefd.cpp), and placing our C code in an extern block covering the remainder of the file. 

#ifdef __cplusplus
extern "C" {
#endif

Next is a struct called _cb_descriptor, which at the moment just stores the writefd which will be used in RtMidi's callback. 

typedef struct {
    int fd;
} _cb_descriptor;

The callback itself is a simple loop which attempts to write all available bytes in *message to the writefd passed in the userdata *data. Passing *data into the callback is a common pattern in C to work around the lack of closure support.

The important thing to note here is that our callback is no longer Perl wrapped in a FFI::Platypus::Closure - our Perl code no longer needs to cede control to RtMidi. 

void _callback( double deltatime, const char *message, size_t size, _cb_descriptor *data ) {
    if( size < 1 ) {
        return;
    }

    int total = 0;
    int remains = size;
    while ( total < size ) {
        int sent = write( data->fd, message + total, remains );
        if ( sent < 0 ) {
            perror("Callback write error.");
            return;
        }
        remains -= sent;
        total += sent;
    }

}

The last part is to create the fd pair, stash the writefd fd into the device's userdata, set the readfd side of the pipe nonblocking, register the callback and userdata with RtMidi, then pass the readfd back to our Perl program.

RTMIDIAPI is a handy macro from RtMidi which will handle symbol exporting in a cross-platform way. 

RTMIDIAPI
int callback_fd( RtMidiInPtr device ) {

    _cb_descriptor *data = (_cb_descriptor*)malloc( sizeof( _cb_descriptor ) );
    int pipefd[2] = { 0, 0 };

    if ( pipe(pipefd) < 0 ) {
        perror("Cannot create pipe!");
        return -1;
    }
    fcntl( pipefd[0], F_SETFL, O_NONBLOCK );
    data->fd = pipefd[1];

    rtmidi_in_set_callback( device, (RtMidiCCallback)&_callback, data );

    return pipefd[0];
}

Building and Binding

The .fbx file required to build this is called ffi/rtmidi-ffi.fbx (so the dynamic library it builds will be called librtmidi-ffi.so/rtmidi-ffi.dll etc. depending on platform). It does some linker flag setup for Windows and MacOS, sets an 'Alien' package the compiler should use to find headers and libraries from a non-system install, then sets the source to all .cpp files in ffi/ - there is currently just one, ffi/pipefd.ccp. 

our $DIR;

my $WINDOWS = $^O eq 'MSWin32' || $^O eq 'cygwin';
my $MACOS = $^O eq 'darwin';

my $libs = '';

$libs = '-lrtmidi -lwinmm -lws2_32' if $WINDOWS;
$libs = '-framework CoreFoundation -framework CoreMidi' if $MACOS;

{
    alien  => ['Alien::RtMidi'],
    source => ["$DIR/*.cpp"],
    libs   => $libs,
};

Something to note about the .fbx file is, it's probably not needed in many cases. If your bundled code is portable enough, and doesn't have external dependencies, you can likely get away without it.

There are a couple of moving parts needed back in Perl land to connect all this up. First we tell Platypus to build and load the bundled code - this will automagically build the bundle as it changes, even in a development environment - no need to invoke make or similar: 

my $ffi = FFI::Platypus->new( api => 2, ... );
$ffi->bundle;

Then we need to attach the C function exported from our bundle: 

$ffi->attach( callback_fd => [ [ 'RtMidiInPtr*' ] => 'int' ] )

Finally, an exported Perl function is added to the binding library to wrap the fd integer returned from C in a readonly IO::Handle. This handle can then be passed easily to event loops: 

sub callback_fh( $dev ) {
    IO::Handle->new->fdopen( callback_fd( $dev ), 'r' );
}

Windows

Keen-eyed Windows API experts may have spotted an issue with the above. Windows does not have pipe(2). This winds up creating something of a mess, but the short version is for Windows, a socketpair is created in Perl, the write-end is passed to C, and the read end is returned to the calling program.

Examples

We should now have all the pieces to pass a filehandle to any event loop. As previous efforts used IO::Async, let's start there.

IO::Async

After MIDI device, decoder, and event loop setup, an IO::Async::Stream is created with the wrapped fd opened in our C code above. We can then attach a callback to the decoder instance to print incoming events, then pass incoming bytes to the decoder in async sub read_stream - as complete MIDI events are received, the decoder's callback will be invoked.

A ticking counter is added to demonstrate other activity - we are not blocked waiting for RtMidi, there is no need to sleep. Where previously we would have had to create a routine, and a channel for inter-process communication, here the decoding of incoming MIDI messages is taken care of by the stream's on_read handler within a single thread. 

use IO::Async::Loop;
use IO::Async::Stream;
use IO::Async::Timer::Periodic;
use MIDI::RtMidi::FFI::Device;
use MIDI::Stream::Decoder;

my $midi_in = RtMidiIn->new();
$midi_in->open_port_by_name( qr/sz|lkmk3/i );
my $fh = $midi_in->get_fh;

my $decoder = MIDI::Stream::Decoder->new;
$decoder->attach_callback( all => sub( $event ) {
    say join ' ', $event->dt, $event->as_arrayref->@*;
} );

my $loop = IO::Async::Loop->new;
my $stream = IO::Async::Stream->new(
    read_handle => $fh,
    on_read => sub( $self, $buffref, $eof ) {
        $decoder->decode( $$buffref );
        $$buffref = "";
    }
);
$loop->add( $stream );

my $tick = 0;
$loop->add( IO::Async::Timer::Periodic->new(
    interval => 1,
    on_tick => sub { say "Tick " . $tick++; },
)->start );

$loop->run;
Mojo::IOLoop

A very similar approach can be used with Mojo::IOLoop and Mojo::IOLoop::Stream: 

use Mojo::IOLoop;
use MIDI::RtMidi::FFI::Device;
use MIDI::Stream::Decoder;

my $midi_in = RtMidiIn->new();
$midi_in->open_port_by_name( qr/sz|lkmk3/i );
my $fh = $midi_in->get_fh;

my $decoder = MIDI::Stream::Decoder->new;
$decoder->attach_callback( all => sub( $event ) {
    say join ' ', $event->dt, $event->as_arrayref->@*
} );

my $stream = Mojo::IOLoop::Stream->new( $fh );
$stream->timeout( 0 );
$stream->on(
    read => sub ( $stream, $midi_bytes ) {
        $decoder->decode( $midi_bytes );
    }
);
$stream->start;

my $tick = 0;
Mojo::IOLoop->recurring( 1 => sub { say "Tick " . $tick++; } );

Mojo::IOLoop->start unless Mojo::IOLoop->is_running;
Future::IO

Future::IO is a solid option if you want you event-driven code to be loop-agnostic. The same Future::IO code can be integrated into programs using one of any number of event loops by loading an IO implementation, e.g. Future::IO::Impl::Glib. 

use Future::IO;
use Future::AsyncAwait;
use Time::HiRes qw/ time /;
use MIDI::RtMidi::FFI::Device;
use MIDI::Stream::Decoder;

my $midi_in = RtMidiIn->new();
$midi_in->open_port_by_name( qr/sz|lkmk3/i );
my $fh = $midi_in->get_fh;

my $decoder = MIDI::Stream::Decoder->new;
$decoder->attach_callback( all => sub( $event ) {
    say join ' ', $event->dt, $event->as_arrayref->@*
} );

async sub msg {
    my $size = $midi_in->bufsize;
    while ( my $midi_bytes = await Future::IO->read( $fh, $size ) ) {
        $decoder->decode( $midi_bytes );
    }
}

async sub tick {
    my $tick = 0;
    while ( 1 ) {
        await Future::IO->alarm( time() + 1 );
        say "Tick " . $tick++;
    }
}

Future->wait_all( tick, msg )->get;

This, to me, appears markedly simpler than the previous effort. There is no need to explicitly create routines, and no oddities around ceding control to RtMidi. It also has the benefit of being more robust, as Perl closures are not being called from external threads.

Sample output

The three examples above should produce equivalent output, which looks something like this: 

Tick 0
0 note_on 0 55 64
Tick 1
0.838393 note_off 0 55 0
Tick 2
0.018346 control_change 0 1 63
0.040585 control_change 0 1 62
0.053563 control_change 0 1 61
Tick 3
Tick 4
Tick 5
2.357593 control_change 0 1 62
0.004254 control_change 0 1 63
0.004569 control_change 0 1 64
Tick 6

We can see our ticking counter interleaved with incoming MIDI delta-times (time since previous event), and events from an external device.

Conclusion

After a review of previous work in integrating RtMidi with another event loop, we went through some of the shortcomings of this approach, before describing a new approach which uses pipes to pass MIDI bytes from RtMidi to a MIDI-processing program. We took a quick tour of the C code, which uses FFI::Platypus::Bundle to build and load a dynamic library to make itself available.

We took a look at some examples around how to integrate the fd created in C into event loops in Perl, which were marked by simpler implementation than previous async integration attempts.

The callback_fh mechanism and the examples in this post are now available in MIDI::RtMidi::FFI v0.10 and above.