[winswitch] Fedora33 xpra h264 encoding support not working

[Antoine Martin]

On 26/05/2021 14:52, Terry Barnaby via shifter-users wrote:
> Thanks for the reply and your work on xpra. I certainly looks a good
> remote desktop access system especially with the individual application
> support.
> 
> On 26/05/2021 05:39, Antoine Martin via shifter-users wrote:
>> (..)
>>> I much prefer to use base system packages wherever possible.
>> Me too, normally.
>> Unfortunately, it's just painful to use xpra with the system packages.
> 
> Is there any reason why the Fedora system packages should be so
> different ? I can see they wouldn't be so up-to-date but I would have
> thought the Fedora maintainer would take the upstream as close as
> possible. It would be a better user experience and probably less support
> if they were basically the same.
Agreed. It's really disappointing that this is not the case.

(..)
>>
>> --encodings vs --encoding
> 
> Ah, one thing that was missing is that you can set a list of allowed
> encodings like "--encodings=h264,rgb".
> 
> I am, perhaps, using a difficult application to handle, Firefox. As that
> gives me the ability to experiment with widgets, images, videos etc
> easily.
Yes, browsers are the most difficult type of application to handle as
the same window, with the same metadata, can contain text or pictures or
even video, often all of them at once.
The ability to scroll up and down makes this even more difficult to handle.

> In the end am experimenting to see what is possible for remote
> access to a set of seismic data processing servers with GUI applications
> to show data in graphical form and interact with these. I am trying to
> see how near local application latency/quality is possible these days.
Users of such applications usually have CPU and bandwidth to spare, and
they achieve near local latency and quality. Xpra is quite popular for
this type of application.
You may want to add your application and / or specific windows to the
mappings in:
/etc/xpra/content-*

>> (..)
>>> xpra now works with me when "--encodings=h264" is set. Generally the
>>> latancy/flickering is better than in auto.
>> That's odd. Using "h264" alone will waste a lot of CPU and bandwidth.
>> (assuming that it is actually used)
>> The latency really depends on a large number of factors so I can't
>> comment without knowing more.
>> But there should be no flickering no matter which encoding is used.
> 
> In this case the flickering is during a screen update of Firefox. Click
> on a link and the screen updates but then various areas update again a
> few times (probably the images). With the h264 only encoding this was
> much less pronounced (maybe as a whole of screen update was used and
> perhaps there is a delay before this is sent after the application
> updates the window buffer).
Right, that makes sense.

(..)
>>> Is there anyway to determine if xpra is using hardware h264 if
>>> I have managed to configure for that ?
>> In theory, hardware acceleration should be used automatically if it is
>> available.
>> The only acceleration that is well tested is with NVENC.
>>
>> You can see which encoders are available on your system with:
>> xpra encoding
>> xpra video
>>
>> Again, just because hardware acceleration is available on your system
>> does not mean that it should be used for every screen updates.
>> In many cases, the server engine will prefer software encoding because
>> it is more suitable.
> 
> Ok on both systems there is the line "found 3 vaapi codecs: h264, hevc,
> mpeg2", so it looks like they are available. Is there anyway of forcing
> hardware encode (just for experimentation) ?
--video-encoders=ffmpeg
Will only load the ffmpeg encoder, which is the one that does vaapi.
(but that's not ideal - see below)
This is a fairly generic API and I only have one active system that
supports it, so I cannot guarantee that other cards / CPU will work well
with it.
NVENC is proprietary, but it works fairly consistently across a wide
range of cards, and it is much faster too.

Hardware encoders score higher than software encoders and will be used
automatically unless the stream is brand new and instantiating a
hardware encoder may take too long.

> Where I am coming from is to see how possible it is to have a low
> latency, good quality remote GUI program access these days for a
> scientific data processing server as mentioned above and was wondering
> how well using modern hardware h264/h265 encoding would work for this
> particular usage. As a background I have tried using gstreamer to
> simulate this usage between two machines connected using OpenVPN over a
> 30 MBits/s Internet connection. With this I could run 3 x 1024x768 at 30Hz
> h264 encoded video streams across the network with the following CPU usage:
(..)
30Mbps should be enough for 4K30Hz.
> 
> So very low CPU usage and reasonable network bandwidth (default h264
> encoder options used). Maybe naive, as I haven't looked into the issues
> in detail, but from this it seems it might be possible to simply send
> the whole application window at, say, 30 frames per second, stopping
> when there has been no activity for a time and continuing h264 encoding
> where it left off so using the h264 compression to handle the minor
> screen changes on its own (I know not the best for a GUI type update but
> simple and in hardware). I was hoping to simulate such a mode in xpra to
> see how well this might work.
The devil is in the details.
You can also use hints to tell the xpra server to prefer video encoders.
But:
* it should not be used for all the popup windows, menus and tooltips
that the application may use - those are small and short lived
* it cannot be used for windows that use transparency - and some
applications use transparency despite being completely opaque..
* lossy screen updates from video codecs trigger an automatic lossless
refresh, which you can turn off if you wish: --auto-refresh-delay=0
* when there aren't many screen updates for a while, it may choose to
use a lossless picture codec instead (which avoids the refresh)
* hardware accelerated encoders take longer to initialize, so the server
may start with a software encoder (x264) to prevent the screen updates
from stuttering, or use jpeg until the video encoder is ready
* if the server decides that the performance is inadequate, it may
choose to subsample the data used as input to the encoder, or even
downscale it
etc..
Almost all of these characteristics can be tuned, but that's not
normally needed.

Cheers
Antoine


> 
> Terry
> 
> 
> 
> _______________________________________________
> shifter-users mailing list
> shifter-users at lists.devloop.org.uk
> https://lists.devloop.org.uk/mailman/listinfo/shifter-users