Help with pip dependencies when Flatpaking Python app

1

Hello!

I am attempting to create my first Flatpak, which is an existing Python app that runs fine on my machine:

(To install and run the app, I start a virtual env, pip install the requirements, and run main.py.)

The steps I have taken so far for flatpaking are:

# this works
flatpak install flathub org.gnome.Sdk/x86_64/47
# this seems to work
python3 flatpak-pip-generator --build-isolation --requirements-file=requirements.txt --runtime=org.gnome.Sdk/x86_64/47

Create org.curiositry.selfctrlpanel.json with the contents:

{
  "app-id": "org.curiositry.selfctrlpanel",
  "runtime": "org.gnome.Platform",
  "runtime-version": "47",
  "sdk": "org.gnome.Sdk",
  "command": "python3 main.py",
  "modules": [
    "python3-requirements.json",
    {
      "name": "what-am-i-doing",
      "buildsystem": "simple",
      "build-commands": [
        "pip3 install --prefix=/app --no-deps ." 
      ],
      "sources": [
        {
          "type": "git",
          "branch": "main",
          "url": "https://gitlab.com/GitFr33/what-am-i-doing.git"
        }
      ]
    }
  ],
  "finish-args": [
    "--share=ipc",
    "--socket=x11",
    "--socket=wayland",
    "--device=dri"
  ]
}

And then run the builder:

flatpak-builder --force-clean --install-deps-from=flathub flatpak-build org.curiositry.selfctrlpanel.json

However, this always throws pip dependency errors, so I suspect I’m doing something horribly wrong. Currently, I’m getting:

ERROR: No matching distribution found for Cython>=3.0.10

Previously, it complained about meson-python, and then various dependencies of dbus-python. I’m pretty sure that adding more things to the requirements.txt is not the answer…

Could anyone who is familiar with flatpacking python apps give me a pointer about where I have made a wrong turn?

Thanks so much!

1 Like
2

Well, Flatpak requires all dependencies to be declared in the manifest. If something is missing on build or after installation, it is most likely because you didn’t add an dependency to the manifest.

A few things I can point out:

First, the requirements.txt seems not to be the simplest way of creating all pip dependencies for Flatpak. In this case, the requirements.txt does have to include all dependencies, including dependencies of dependencies.
In the Flatpak Docs the normal command omits the argument for the requirements file, but instead specifies the module name of the app, so the dependencies are gathered for this module. May be worth trying.

See also the flatpak-pip-generator ReadMe for further details.

Secondly, a note on app name and app id:
The name “What am I doing?” is an… let’s say unique name. May be worth a change.
Also, in the app id (org.curiositry.selfctrlpanel), the app is called “selfctrlpanel”. That part should match the app name.
Also, the domain in the app id org.curiositry appears to not be owned by you. If you intend to publish your app, the app id needs to be owned by you as per Flathub Policy. Refer to their documentation for further details.

2 Likes
3

@CodedOre Thank you so much for the help!

I assumed that when the dependencies were collected from the python-pip-generator output, their dependencies were fetched as well. This makes a lot of sense out of what was happening!

Two questions:

  1. Is there a simple way to collect all the dependencies recursively into the requirements file, or at some other step of this process? My Googling of the question hasn’t turned up any simple solutions.
  2. Is there a way to flatpak “pile of files” style apps, or do they pretty much need to be turned into a module first?

I’m not trying to push it to Flathub myself. It’s a project I use and have contributed to, but I’m just trying to submit a patch preparing it for flatpaking, not be the Flatpak maintainer. But you’re right that it should be com.curiositry rather than org, and I should update it to reflect the app’s current name.

1 Like
4

Like I mentioned before, try to run flatpak-pip-generator without the reference to the requirements file. In this case, it should find the dependencies and their dependencies for the application module.

You can use the simple buildsystem and define the build-commands yourself. In the case of pre-build apps, you need to copy all files into the appropriate folders inside the /app prefix.

2 Likes
5

Thanks so much @CodedOre. I think I am making progress, but I’m still a bit lost.

I have (as far as I can tell) successfully turned the project into a local module, and built it into a wheel using Setuptools as the backend (and tested installing it with Pip).

However, when you say “run flatpak-pip-generator without the reference to the requirements file”, I’m unclear what to point it at, since the module isn’t on PyPI yet. What is the correct way to run flatpak-pip-generator against a local module (built or unbuilt)?

6

@CodedOre I also now have the Flatpak building, and am able to run it and it runs main.py, using simple build system like you suggested, with a modules manifest inspired by this KDE project:

modules:
  # - python3-pyproject.toml.json
  - name: what-am-i-doing
    buildsystem: simple
    build-commands:
      - install -D runner.sh /app/bin/runner.sh
      # - ls -la
      - pip3 install --no-build-isolation --prefix=/app . 
      # - ls -la
    sources:
      - type: script
        commands:
          - exec python3 /app/lib/python3.12/site-packages/what-am-i-doing/main.py
        dest-filename: runner.sh
      - type: archive
        path: dist/what_am_i_doing-0.0.1.tar.gz

Now all I’m missing is … all the dependencies :slight_smile:

I’m guessing that transferring the dependencies from requirements.txt to pyproject.toml won’t help with fetching them recursively? I did notice that I could point flatpak-pip-generator at my pyproject.toml and it would collect things from there.

I submitted a patch, and the developer made a test PyPi package, which I was able to install; however, I haven’t figured out how to point flatpak-pip-generator at a test PyPi package (it doesn’t take the --index-url flag that allows me to pip install --index-url https://test.pypi.org/simple/ packagename).

Any advice would be much appreciated!

1 Like
7

The project itself is now on PyPI:

https://pypi.org/project/what-am-i-doing/

However, when I run:

python3 flatpak-pip-generator --runtime=org.gnome.Sdk/x86_64/47 what-am-i-doing

It produces this:

{
    "name": "python3-what-am-i-doing",
    "buildsystem": "simple",
    "build-commands": [
        "pip3 install --verbose --exists-action=i --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} \"what-am-i-doing\" --no-build-isolation"
    ],
    "sources": [
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/68/9b/e77c19b23da869416f49f46904c4eefe9ccf0b12b46da4b2d90f638e2c55/what_am_i_doing-0.0.2-py3-none-any.whl",
            "sha256": "7e4968f20901a958b3f66627e98cb976e5d5ad396d0628ef9773475f63016202"
        }
    ]
}

And when I include that in the manifest, and build and run the flatpak, I get:

ModuleNotFoundError: No module named 'requests'

So I’m obviously not doing it right…

1 Like