How to increase the file descriptor limit (NOFILE/ulimit -n) for a Flatpak app? VKD3D shader cache issue

1

Hello,

I’m facing a problem with the Heroic Games Launcher (com.heroicgameslauncher.hgl) and a game (Horizon Forbidden West) running via Wine/Proton. The game uses VKD3D-Proton to cache DX12 shaders.

The Problem:
Every time I exit the game, the shader cache merge process (from vkd3d-proton.cache.write to vkd3d-proton.cache) seems to fail. I suspect this is because the file descriptor limit (ulimit -n) inside the Flatpak sandbox is only 1024. The process likely needs to open thousands of files to merge the cache and hits this limit.

  • Proof: flatpak run --command=sh com.heroicgameslauncher.hgl -c "ulimit -n" returns 1024.

  • Result: The .write file is left behind, the main .cache doesn’t update, and the game recompiles all shaders from scratch on every launch (taking several minutes).

My Question:
Is there an official or recommended way to increase this RLIMIT_NOFILE for a specific Flatpak application?

What I’ve tried (unsuccessfully):

  1. Creating a systemd user override for the app’s scope (e.g., app-flatpak-com.heroicgameslauncher.hgl-*.scope). This didn’t work because the limit seems to be enforced earlier by the sandbox (bwrap).

  2. Setting the WINE_FD_LIMIT=1048576 environment variable inside the sandbox. This does not affect the actual OS-level limit.

Is this the right approach?
Would a flatpak override command like the one below work? If so, is there a specific parameter that directly controls the file descriptor limit for the sandboxed process?

flatpak override --user com.heroicgameslauncher.hgl --env=WINE_FD_LIMIT=1048576 # (plus other necessary permissions)

Or is there another mechanism I should use? If possible, could you provide an example of a working configuration?

System Information:

  • Distribution: Fedora Linux 42 (Rawhide)

  • Flatpak version: flatpak --version → 1.16.2

  • Affected application: com.heroicgameslauncher.hgl

Any guidance would be greatly appreciated. This issue severely impacts the usability of demanding DX12 games through Flatpak.

Thank you for your help!

2

I have updated my approach. It prevents data loss, but I am not seeing cache growth yet.

1. Cache Saving (Stalemate):

I implemented a “Smart Wrapper” script that compares file sizes before overwriting to prevent data loss.

Result: The main cache file is preserved (no longer overwritten by empty files), BUT it is not growing.

I restored a backup cache (~22MB). After a 20+ minute session, the file size remained exactly the same (22849104 bytes).
My wrapper log reports: INFO: No .write file found. on exit.
This implies that while VKD3D_CONFIG=pipeline_library_app_cache stopped the errors in the log, the game is still not flushing new compilation data to the disk on exit.

2. The Audio Issue:
To address the CPU starvation hypothesis, I increased PULSE_LATENCY_MSEC.

Change: Increased from 90 to 300.
Status: Audio worked in the last session, but given the randomness of this issue, I need more runs to confirm stability.

3. Settings:
Disabled GameMode and EAC in Heroic settings.

CURRENT CONFIGURATION

A. The “Smart” Wrapper Script:
Currently testing this script in Heroic’s “Wrapper Command”.

#!/bin/bash

ulimit -n 524288

export PROTON_LOG=1
export PROTON_LOG_DIR="$HOME/Games/Heroic/HorizonForbiddenWestCE"
export VKD3D_DEBUG=warn

echo "--- NEW SESSION: $(date) ---" >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"

"$@"

echo "Game closed. Checking cache files..." >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"

if [ -f "vkd3d-proton.cache.write" ]; then
    
    if [ -f "vkd3d-proton.cache" ]; then
        # Get file sizes
        SIZE_OLD=$(stat -c%s "vkd3d-proton.cache")
        SIZE_NEW=$(stat -c%s "vkd3d-proton.cache.write")
        
        echo "Size OLD: $SIZE_OLD | Size NEW: $SIZE_NEW" >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"

        if [ $SIZE_NEW -gt $SIZE_OLD ]; then
            mv -f "vkd3d-proton.cache.write" "vkd3d-proton.cache"
            echo "SUCCESS: New cache is bigger. Updated." >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"
        else
            rm "vkd3d-proton.cache.write"
            echo "IGNORE: New cache is smaller. Kept the old big file." >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"
        fi
    else
        # No old cache, create first one
        mv -f "vkd3d-proton.cache.write" "vkd3d-proton.cache"
        echo "START: Created first cache file." >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"
    fi
    
    chmod 666 "vkd3d-proton.cache" 2>/dev/null

else
    echo "INFO: No .write file found." >> "$HOME/Games/Heroic/HorizonForbiddenWestCE/fix_status.txt"
fi

B. Environment Variables:

PULSE_LATENCY_MSEC = 300

VKD3D_CONFIG = no_upload_hvv,pipeline_library_app_cache

__GL_SHADER_DISK_CACHE = 1

__GL_SHADER_DISK_CACHE_SIZE = 10000

WINE_RT_AUDIO = 1

WINE_FD_LIMIT = 524288

Summary:
Wrapper prevents data loss, but the game is not updating the cache file (no .write file generated on exit). Audio seems stable with higher latency.