[winswitch] Fedora33 xpra h264 encoding support not working

[Terry Barnaby]

Thanks for the reply and your work on xpra. I certainly looks a good 
remote desktop access system especially with the individual application 
support.

On 26/05/2021 05:39, Antoine Martin via shifter-users wrote:
> (..)
>> I much prefer to use base system packages wherever possible.
> Me too, normally.
> Unfortunately, it's just painful to use xpra with the system packages.

Is there any reason why the Fedora system packages should be so 
different ? I can see they wouldn't be so up-to-date but I would have 
thought the Fedora maintainer would take the upstream as close as 
possible. It would be a better user experience and probably less support 
if they were basically the same.


>
>>>> Both show the same issue.
>>>>
>>>> If I use "--encodings=h264" on a manually started server and client, the
>>>> server messages have the error listed:
>>> This is the wrong switch to use.
>>> You need more than one 'encodings' enabled to be able to run correctly.
>> Not sure what you mean by this ?
> The xpra server needs more encodings enabled to be able to send screen
> updates.
> For example, it is not always possible to send screen updates using
> `h264". There are size constraints, rounding errors, etc.
>
>> I am just experimenting to see what the capabilities/performance options
>> are and so was trying to force full h264 encoding to try this out.
> Without knowing exactly what application you are testing with, I can't
> be certain, but forcing "h264" is almost always a bad idea.
>
>> I assume from what you say it is possible to define multiple encodings
>> that xpra can choose from somehow ?
> I have updated the manual to try to clarify :
> https://github.com/Xpra-org/xpra/commit/174056958a44d25d95bc5f8d659a5e916016620c
>
> --encodings vs --encoding

Ah, one thing that was missing is that you can set a list of allowed 
encodings like "--encodings=h264,rgb".

I am, perhaps, using a difficult application to handle, Firefox. As that 
gives me the ability to experiment with widgets, images, videos etc 
easily. In the end am experimenting to see what is possible for remote 
access to a set of seismic data processing servers with GUI applications 
to show data in graphical form and interact with these. I am trying to 
see how near local application latency/quality is possible these days.


>
> (..)
>> xpra now works with me when "--encodings=h264" is set. Generally the
>> latancy/flickering is better than in auto.
> That's odd. Using "h264" alone will waste a lot of CPU and bandwidth.
> (assuming that it is actually used)
> The latency really depends on a large number of factors so I can't
> comment without knowing more.
> But there should be no flickering no matter which encoding is used.

In this case the flickering is during a screen update of Firefox. Click 
on a link and the screen updates but then various areas update again a 
few times (probably the images). With the h264 only encoding this was 
much less pronounced (maybe as a whole of screen update was used and 
perhaps there is a delay before this is sent after the application 
updates the window buffer).


>
>> One thing that is interesting
>> is the CPU usage on the server side. Using firefox as a test client and
>> clicking on different web pages and waiting the server's CPU usage can
>> be around 15%
> Do you have audio forwarding running?
> If there are no screen updates, the server CPU usage should be very
> close to zero.
>
>> after than or sometimes 100% or 200% and then sticks at
>> those values even though the page looks static after that.
> You can turn on "damage" and "compress" debugging on your server to see
> if it is receiving / sending screen updates:
> xpra start -d damage,compress
> * "damage" corresponds to application paint events
> * "compress" logs the type of picture compression used
>
> You can also see which encoding is used for each screen update using the
> colour coded client debug switch:
> xpra attach --no-mmap --env=XPRA_PAINT_BOX=1
> The color codes are defined here:
> https://github.com/Xpra-org/xpra/blob/master/xpra/client/paint_colors.py

Thanks for the information. I will play with those options next time I 
see this. I suspect it is the use of Firefox on a page with something 
being updated all of the time (although I didn't see anything) and with 
h264 lots of whole window screen updates are happening for small areas.


>
>> I assume that my xpra server here is using software h264 from those
>> figures.
> Assuming that it is actually using "h264" for all the screen updates -
> which is not guaranteed.
>
>> Is there anyway to determine if xpra is using hardware h264 if
>> I have managed to configure for that ?
> In theory, hardware acceleration should be used automatically if it is
> available.
> The only acceleration that is well tested is with NVENC.
>
> You can see which encoders are available on your system with:
> xpra encoding
> xpra video
>
> Again, just because hardware acceleration is available on your system
> does not mean that it should be used for every screen updates.
> In many cases, the server engine will prefer software encoding because
> it is more suitable.

Ok on both systems there is the line "found 3 vaapi codecs: h264, hevc, 
mpeg2", so it looks like they are available. Is there anyway of forcing 
hardware encode (just for experimentation) ?

Where I am coming from is to see how possible it is to have a low 
latency, good quality remote GUI program access these days for a 
scientific data processing server as mentioned above and was wondering 
how well using modern hardware h264/h265 encoding would work for this 
particular usage. As a background I have tried using gstreamer to 
simulate this usage between two machines connected using OpenVPN over a 
30 MBits/s Internet connection. With this I could run 3 x 1024x768 at 30Hz 
h264 encoded video streams across the network with the following CPU usage:

Server: 4 Cores, 4 HyperThreads, Intel(R) Xeon(R) CPU E3-1245 v5 @ 3.50GHz
Client: 6 Cores, 0 HyperThreads, Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz

1 Application

Host        Process    ssh    System
=======================================
CPU Beam:    18%    1%    3%
CPU Study    7.6%    4%    5%

3 Applications

Host        Process    ssh    OveralSystem
=======================================
CPU Beam:    3*18%    3*1%    9%
CPU Study    3*7.6%     3*4%    12%

NetworkRate:    3.2 MByte/s

So very low CPU usage and reasonable network bandwidth (default h264 
encoder options used). Maybe naive, as I haven't looked into the issues 
in detail, but from this it seems it might be possible to simply send 
the whole application window at, say, 30 frames per second, stopping 
when there has been no activity for a time and continuing h264 encoding 
where it left off so using the h264 compression to handle the minor 
screen changes on its own (I know not the best for a GUI type update but 
simple and in hardware). I was hoping to simulate such a mode in xpra to 
see how well this might work.

Terry