Proposal for audio plugins

Flatpak doesn’t make it easy to have Audio plugins for Audio application. There are 4 formats at least: LADSPA, DSSI, LV and VST (native + windows). For now I’ll focus on the first 3, VST will come next.

Here is my proposal for audio plugins in Flatpak on flathub:

I created a base app used to build audio application AND plugins. This will provide the base libraries built to support the plugins: See https://github.com/flathub/flathub/pull/1460 (current status: Pull Request)
It is based off the KDE 5.14 runtime as to have Qt5 and has gtk2 built, as well as common modules needed for the audio plugins. One of the issues I have run into is that if GNOME 3.36 is required, it’s a bit more complicated.

Please, feel free to submit feedback on the pull request.

Once it is available on Flathub, plugins shall be submitted (they can’t build otherwise). I already have a few that I did package to actually test this concept, in all 3 formats, with the first being So-SynthLV2: https://github.com/flathub/flathub/pull/1459

Also update the following Flatpaks to build against the base app and add the extension point (I have already done the work locally for some):

  • Muse3 sequencer (LADSPA, DSSI)
  • Hydrogen (LADSPA)
  • gsequencer (LADSPA, DSSI and LV2) that ship with some plugins that instead should be build as Flatpak extensions first.
  • Audacity (LADSPA, LV2)
  • Ardour (current status: Pull Request https://github.com/flathub/flathub/pull/831) (LADSPA, LV2) - this one was actually the motivation to push this effort.

Possibly other apps I haven’t listed here yet.

Update:

The base app is necessary for plugins, not the apps. It only contains DSSI and LV2, but is used for all the plugin types supported. If is now base in freedesktop sdk only, with caveat. Just updated the PR for the base app:

Update2:

The BaseExtension app now contain nothing, it is just necessary to declare the extension points so that extensions can be built. This is all that is needed. It also handle the runtime versioning.

And this PR is now what’s blocking any further submissions.

1 Like

Hi @hub - thanks for your work on this. Can you confirm whether all of these plugins /themselves/ need all the dependencies you mention (Qt5, Gtk2, Gtk3, …) or are you trying to capture the dependencies of every program that needs these plugins so they can share the same app?

The reason I ask is that you don’t need to use a base app just to inherit an extension point - you can also just manually duplicate the extension point - declare it with the same name in another unrelated app - and Flatpak will pick up the matching named extensions. This means if your plugins have more simple/modest requirements - eg they could be built in a freedesktop.org runtime, your base app could be a stub, that exists only to build the plugins, and the apps can use any ABI compatible (eg, GNOME or KDE deriving from the same fd.o) runtime to provide their extensions, not use the base app, and then the apps can just import the plugin extension point by name.

If most/all of these extensions do need these dependencies, then I am inclined to agree this is the least bad approach. But it’s also a little sad because it will mean this base app has to keep piling up all of the dependencies that any audio app might need.

1 Like

Hey, it’s really cool that you’re working on this!

Audio plugins have to be shared objects that are loaded into a host process, so that processing can be as low-latency as possible. With that in mind, why do audio plugins have a separate runtime? Surely they need their dependencies to be present in the runtime of whatever the host app is?

edit:

I created a base app used to build audio application AND plugins.

OK, so if I understand, the idea is to make “one Pro Audio runtime to rule them all” .

1 Like

@ramcq

Plugins usually have the latitude to use the toolkit they want. In general I have seen Gtk2, Qt5 and sometime Gtk3. With Qt4 that I have excluded, they are the 4 toolkits that are “supported” by LV2. Gtk3 being in the Freedesktop SDK and Qt5 in the KDE sdk, it felt only natural to use the KDE sdk as a base. Gtk2 is very popular amongst plugins, and with Ardour (the main motivation to get this going) it becomes a no brainer. gsequencer which is on Flathub also uses Gtk2.

As for the dependencies, a lot of plugins want liblo to implement OSC (as do the apps). LV2 plugins need the lv2 package (so do the apps), LADSPA need the SDK, so does DSSI, etc. In addition jack seems to often be a hard require as well as lrdf.
(the PR needs an update, I forgot to push it)

For the record, I have run through a few apps and almost a dozen plugins in the 3 formats to refine the list of dependencies. There was already a lot of things in common.

Things I decided against including are fltk (ZynAddSubFx and Yoshimi seems to like it), python-2.7 (required by a few never updated packages and / or waf), and other audio wrapper like portaudio or rtaudio.

I thought this was the best approach to minimize redundancy, as I could figure out if I could have a BaseApp using freedesktop sdk but the actual app building with GNOME or KDE, and while just using shared-modules for the manifest to build the various dependencies modules could be another option I was also trying to avoid the bloat. Maybe I should still do that.

And if the app needs the GNOME Platform it can just build independently, still.

I’ll update the PR later tonight.

This is actually no true. gsequencer uses Gtk3.

I dug a bit more with the corner cases, and here is why this is justified, albeit problematic:

Carla require Qt5 and is a LV2 plugin. If you build the app that mount LV2Plugins without using the base app but instead using the GNOME runtime, there is no Qt5 and the UI will not run. This is the kind of problem we’ll encounter, and the failure is very silent. While most plugin might not require the BaseApp, several make it necessary.

It would be simpler if Qt5 was a SDK extension for Freedesktop.

Suggestions welcome, off course.

I have submitted some of the modules to shared-modules:

I think this is a good solution, the most important is that things “just work” even if that does mean a bit of extra disk space and build time usage. I don’t have a better idea :slight_smile:

The PR above has been merged.

I think I’ll try with a mostly empty BaseApp. It is still required to actually build plugins since the extension point have to be defined. But the hosting application doesn’t to and the shared-modules will help building stuff.

Few plugins can be build with an empty BaseApp. This include LADSPA plugins, VST plugins and plugins built using JUCE.

DSSI and LV2 plugin (that require the respective packages) and plugins that require Gtk2 are more tricky to build.

So now the base app is strictly limited to LV2 and DSSI to provide the headers. The plugins don’t link against it.

It could be treated as a SDK.

Another update. It got thinned down to just the extension points and renamed “BaseExension”.

lv2 and dssi have to be build by the respective plugins.

They really should not use any of those toolkits. Plugins must be self-contained (statically linked).
gtk2/3/4 cannot co-exist in the same memory space, nor can QT4/5.

Most LV2 plugins have moved away from gtk and QT in recent years. Also because those toolkits don’t offer any relevant widgets for pro-audio.

When a plugin host loads a plugin binary, all of the plugin’s public symbols are mapped into the memory space of the host. If the plugin depends on a shared libs, the symbols of those libs are also inherited, and there may be conflicts.

This is not limited to GUI libs alone. Say you load a plugin that dynamically links against libaubio2, then you cannot load a different plugin that is dynamically linked against libaubio3 in the same session.

1 Like

This a problem we can try to mitigate. The good news is that anything in the Freedesktop-sdk is shared, so we already have that.

Some update: The base extension, that is needed to build the extensions, is now part of flathub repository.

I have updated LMMS, Hydrogen, gsequencer to support plugins.

I have also submitted the LADSPA version of swh and PR for Kdenlive, Flowblade and Shotcut that ship them (as a dependency).

I now have Ardour 6 ready (no Jack) and it was the big driver for this feature. While it’s not yet approved I think it is in a state that is functional, granted we need also plugins to land.

Now one thing is the discoverability of plugins.