Out of the box, OBS’s VA-API hardware encoding performance with AMD GPUs is rather abysmal. Even on the newest generation of AMD GPUs (RDNA2 architecture, e.g. 6900XT), OBS can do 1440p 30fps max. 1440p 60fps, 4k 30fps and of course 4k 60fps recording typically looks like a slideshow.
I was searching for a solution to this, and most commonly suggested workarounds involved:
-
Using AMD AMF. I did not want to go this route as I did not want to use AMD’s installer script, since it is not integrated with the distro package managers very well, does not work with Flatpak and is proprietary software.
-
Using a custom fork of obs, specifically obs-hevc-vaapi-git which has some patches unlocking HEVC VAAPI recording in OBS. I did not really want to go down this path either, as I do not use Arch and building it myself would be quite difficult.
Enter the obs-gstreamer plugin. With this plugin, all you need to do is:
-
In OBS settings > Output > Streaming choose “GStreamer Encoder” as the encoder and “VAAPI” as the encoder type.
-
For recording, go to the Recording tab and choose “Use stream encoder” as the encoder
With this, coupled alongside the obs-vkcapture plugin, my 6900XT is capable of doing 100% smooth 4K 60fps gameplay recording.
To install, you need the following flatpak packages:
com.obsproject.Studiocom.obsproject.Studio.Plugin.Gstreamerorg.freedesktop.Platform.GStreamer.gstreamer-vaapi- Optionally install
com.obsproject.Studio.Plugin.OBSVkCapturefor capturing game footage^1
[1] You will additionally have to:
-
Install
org.freedesktop.Platform.VulkanLayer.OBSVkCapturewhen running games e.g. Steam via flatpak -
Install the capture tools natively when e.g. running games via Lutris installed through apt/pacman etc
(Originally posted here)