Heavy screen flickering VirtualBox 6.1.48 / 7.0.12 Ubuntu Wayland guest kernels 6.4 / 6.5

[Erwin]

Heavy screen flickering occurs with Ubuntu Wayland guest after installing guest Ubuntu updates. Forum: ​https://forums.virtualbox.org/viewtopic.php?t=110882

Reproducible on AMD Ryzen 7/9 Windows/Linux host computers as all guests running Ubuntu (Wayland) flavors are affected after updated kernel versions 6.4/6.5:

• Ubuntu 22.04 / 23.10 desktops running kernel versions 6.4 and 6.5
• Fedora 39, kernel 6.6.9-200.fc39
• Hosts Windows 10 and any Linux
• VirtualBox versions 6.1.48 and 7.0.12
• Corresponding Guest additions installed
• Guest Display settings: 128MB, 3D acceleration unchecked (as 3D graphics acceleration is not supported by VirtualBox)
• Latest Windows 10 host NVidia driver v546.33

Problems:

• Screen flickering occurs especially in full screen of the guest and maximized guest window.
• Additionally, the Ubuntu 23.10 / 24.04 installation window does not start or is white (popup asks to quit as there is no GUI response).
• Very slow GUI guest response.

[pm4492]

Getting this error in Xorg.0.log with kernel 6.6 not with kernel 6.1

[    53.021] (EE) vmwgfx_scanout_update: failed to send dirty (-12, Cannot allocate memory)
[    53.021] (EE) vmwgfx_scanout_update: failed to send dirty (-12, Cannot allocate memory)

[pm4492]

Among the kernels that I can find, 6.3.9 is the last one that works and 6.4 is the first one that has this issue.