Extension's files are not accessible by apps on the host?

Hi, I am packaging org.kicad.KiCad on flathub.

TL;DR: My question is: Is there a way to make an extension’s files accessible on the host? If not, it seems that it’s currently impossible to split the KiCad documentation out into a .Docs extension?

In detail:

Currently we include the HTML documentation in the main flatpak, which – as of today – is 118 MB of data for everyone to download with every new install:

$ flatpak run --command=sh org.kicad.KiCad
[📦 org.kicad.KiCad ~]$ du -hs /app/share/doc/kicad/
118M	/app/share/doc/kicad/

So, I was looking into splitting the HTML documentation off into a dedicated .Docs extension, which would be not installed by default. This seemed straight-forward after I learned about bundled extensions. So I went ahead and added a new .Docs extension to add-extensions, bundling /app/share/doc/kicad:

add-extensions:
  org.kicad.KiCad.Docs:
    version: stable
    directory: share/doc/kicad
    bundle: true
    autodelete: true
    no-autodownload: true

Running the newly built flatpak locally, by default the extension directory is empty, as expected:

$ flatpak run --user --command=sh org.kicad.KiCad
[📦 org.kicad.KiCad ~]$ ls -lh /app/share/doc/kicad/
total 0

And, after installing the .Docs extension, the above path contains the help subdirectory. Again, as expected:

$ flatpak install --user --reinstall --assumeyes "$(pwd)/.flatpak-builder/cache" org.kicad.KiCad.Docs
Looking for matches…


        ID                              Branch           Op           Remote                Download
 1. [✓] org.kicad.KiCad.Docs            stable           i            docs-origin           1.0 kB / 310.3 MB

Installation complete.
$ flatpak run --user --command=sh org.kicad.KiCad 
[📦 org.kicad.KiCad ~]$ ls -lh /app/share/doc/kicad/
total 0
drwxr-xr-x. 1 me me 44 Jan  1  1970 help
[📦 org.kicad.KiCad ~]$ ls /app/share/doc/kicad/help
ca  de	en  es	fr  id	it  ja	pl  ru	zh
[📦 org.kicad.KiCad ~]$ ls -lh /app/share/doc/kicad/help/en
total 2.3M
-rw-r--r--. 3 me me 244K Jan  1  1970 cli.html
-rw-r--r--. 3 me me 745K Jan  1  1970 eeschema.html
-rw-r--r--. 3 me me  46K Jan  1  1970 gerbview.html
-rw-r--r--. 3 me me 121K Jan  1  1970 getting_started_in_kicad.html
drwxr-xr-x. 1 me me  31K Jan  1  1970 images
-rw-r--r--. 3 me me  50K Jan  1  1970 introduction.html
-rw-r--r--. 3 me me 140K Jan  1  1970 kicad.html
-rw-r--r--. 3 me me  41K Jan  1  1970 pcb_calculator.html
-rw-r--r--. 3 me me 861K Jan  1  1970 pcbnew.html
-rw-r--r--. 3 me me  66K Jan  1  1970 pl_editor.html

Great! So everything’s fine, let’s push and merge!

But, wait! The help file is now not opening in the host browser anymore as it used to do!? Literally nothing happens when you click on Help → Help from KiCad (or any of the other editors) with the .Docs extension installed.

Without the .Docs extension installed, KiCad detects the missing files and offers to open the online help instead. But with .Docs installed, nothing happens. The Help → Help (or any other help) menu entry seems to be dead.

So I checked what was going on here, and it seems to be that the files from extensions are not appearing on the host file system. With the .Docs extension installed (and the being accessible from within the flatpak, see above), the path on the host is empty:

$ ls -lh ~/.local/share/flatpak/app/org.kicad.KiCad/x86_64/stable/97c4601cfe60b3554deff34f40dbfb04297ad4c231b36891c8f68d6da7dfde6f/files/share/doc/kicad
total 0

For contrast, the current stable installation of org.kicad.KiCad from flathub still includes the docs in the main flatpak directly, and has them mapped to the host file system:

$ ls -lh /var/lib/flatpak/app/org.kicad.KiCad/x86_64/stable/ab46313fe580ec39a6a93e2058668a21728e1689b929bc5ea7b7f0c25da6e88b/files/share/doc/kicad/help/en
total 2.3M
-rw-r--r--. 2 root root 228K  Jan 1 1970  cli.html
-rw-r--r--. 2 root root 736K  Jan 1 1970  eeschema.html
-rw-r--r--. 2 root root  46K  Jan 1 1970  gerbview.html
-rw-r--r--. 2 root root 121K  Jan 1 1970  getting_started_in_kicad.html
drwxr-xr-x. 1 root root  31K  Jan 1 1970  images
-rw-r--r--. 2 root root  50K  Jan 1 1970  introduction.html
-rw-r--r--. 2 root root 138K  Jan 1 1970  kicad.html
-rw-r--r--. 2 root root  41K  Jan 1 1970  pcb_calculator.html
-rw-r--r--. 2 root root 853K  Jan 1 1970  pcbnew.html
-rw-r--r--. 2 root root  66K  Jan 1 1970  pl_editor.html

This is the path, I see in my browser URL bar after opening Help → Help:

Firefox showing the help file from the path mapped to the host1695×837 151 KB

If you want to reproduce the issue locally, feel free to install KiCad and the .Docs extension from the PR (assuming that the test build suceeds. The above tests and screenshots were done with a local build), and compare it to the currently available stable flathub version of org.kicad.KiCad.

Extensions are a kind of runtime & mounted at the correct path when the application is run via Flatpak.

The actual location on the host should be /var/lib/flatpak/runtime/org.kicad.KiCad.Docs/…. I don’t know if there’s an API to provide file location of the host.

Thanks for the hint. At least I have found the files on the host fs now.

But, shouldn’t flatpak then hand this file path over to the app you are starting on the host? In case of KiCad (which uses wxWidgets), “opening a HTML help file” means “start a web browser to display this file:// URL”.

The file:// URL KiCad tries to open is file:///app/share/doc/kicad/help/en/kicad.html when you click on Help → Help from the KiCad project manager (i.e. “KiCad”). In the currently published flatpak, this URL gets correctly translated, or re-mapped, to the /var/lib/flatpak/app/... location on the host fs.

But using an extension, flatpak doesn’t seem to do this translation correctly (or at all?). Is this a bug that should be reported or is this by design? To me it feels like a bug.

Maybe just to underline my point: I am not asking for API to get this file’s location on the host from within the app running as flatpak. I would have expected flatpak to translate a file path from within the sandbox to the host fs path automatically, just as it does it already (as long as it’s part of the app). But this assumption broke with a file coming from an extension (a.k.a. /var/lib/flatpak/runtime/...) and not from the main app (a.k.a. /var/lib/flatpak/app/...).