Wayland vs X11: A Deep Dive

Wayland and X11 have coexisted on Linux desktops for over a decade, but the balance has shifted decisively. Major distributions now ship Wayland sessions by default, and most daily-use applications work seamlessly under it. Still, X11 is not gone, and the architectural differences between the two systems have real, practical consequences for security, latency, screen sharing, and legacy software. This guide cuts through the noise and explains how each system works, where one beats the other, and how to navigate a world where both are still present.

Architecture: How X11 Works

X11 is a networked windowing protocol created at MIT in 1984. Its design separates the X server (which owns the display hardware) from clients (applications). A compositor such as Mutter or KWin sits on top as just another client, asking the X server to composite windows and hand output to the display. This layering made remote display over a network elegant and powerful for its era.

The downside is that every window on screen can, by default, spy on every other window. Any client can read keystrokes sent to a different application, capture pixel data from arbitrary windows, and inject synthetic input events—all without special privilege. This is not a bug; it is how the protocol was designed when multi-user networked workstations were the threat model. That threat model is no longer valid for a personal desktop in 2024.

Rendering has also grown baroque. Applications draw using a mix of toolkits (GTK, Qt, SDL, Electron), each talking to the X server via Xlib or XCB, while a compositor uses extensions like COMPOSITE, DAMAGE, and GLX to intercept that output and apply effects. The result is extra copies of framebuffer data, additional round-trips, and tearing artifacts when synchronization is missed.

Architecture: How Wayland Works

Wayland is not a server-client protocol in the X11 sense. It is a protocol specification that defines how a Wayland compositor communicates directly with clients. The compositor is simultaneously the display server, the window manager, and the compositing engine—there is no separate X server layer. Each client renders into its own buffer (typically via GPU-accelerated EGL surfaces), then hands that buffer to the compositor via the wl_surface protocol. The compositor decides what to show on screen and when.

The security posture is radically different. Under Wayland, an application can only see its own pixels and receive only the input events the compositor explicitly delivers to it. There is no ambient access to other applications' windows or input streams. This isolation is enforced at the protocol level, not by optional access controls bolted on afterward.

Network transparency—the ability to run a remote application and display it locally—is not built into Wayland the way it is into X11. Remote rendering requires higher-level protocols such as RDP, VNC, or Pipewire-based screen capture. For most desktop users this is not a loss; for administrators managing remote GUI applications it requires an adjusted workflow.

Security Implications in Practice

The X11 isolation problem is not theoretical. The xinput utility can log every keystroke typed in any window while X11 is running—no root required, no special capability needed. A malicious application or a compromised dependency running in your session has the same access.

Under Wayland, that attack surface is eliminated by design. Clipboard sharing, screenshot capture, and global hotkeys all require explicit compositor-mediated mechanisms: xdg-desktop-portal, the zwp_input_method protocol, or compositor-specific extensions. Applications must request these capabilities, and compositors can prompt the user for permission.

One important caveat: when you run an X11 application inside a Wayland session via XWayland, that application reverts to X11 security rules within the XWayland server's scope. Other XWayland clients can spy on each other. Native Wayland clients remain isolated from this. You can audit which of your applications still use XWayland with:

xlsclients -display :0

An empty result or only compositor-internal entries means your session is running mostly native Wayland clients.

Latency and Frame Timing

Wayland's direct compositor model eliminates a round-trip that X11 always had: application draws → X server receives → compositor re-draws → display. Under Wayland the path is: application draws into buffer → compositor presents buffer. Fewer copies, fewer context switches.

More importantly, Wayland compositors implement presentation feedback (the wp_presentation protocol), which lets clients know the exact timestamp at which a frame was shown. This enables tear-free, low-latency rendering that adapts to the actual display refresh cycle. X11 compositors rely on heuristics and extension workarounds to achieve similar results, and they frequently miss.

For gaming, the story is nuanced. Wayland now supports VRR (Variable Refresh Rate) and explicit sync via the linux-drm-syncobj protocol in modern kernels and compositors (GNOME 46+, KDE Plasma 6+). If you are on an older compositor or kernel, X11 may still deliver lower perceived latency for fast-paced games. Check your compositor's release notes before assuming Wayland is faster for your specific GPU and driver combination.

What Still Requires X11

Not everything works natively on Wayland yet. The following categories still present real friction:

• Screen recording and capture: Older tools like scrot, xwd, and ffmpeg's x11grab do not work under Wayland. Use Pipewire-based alternatives: wf-recorder, OBS (with the Pipewire screen capture plugin), or GNOME/KDE's built-in screen recorders.
• Global hotkey daemons: Tools such as xbindkeys and sxhkd depend on X11 global input grab. Under Wayland you need compositor-native solutions (kglobalaccel on KDE, xdg-desktop-portal keybindings, or compositors that expose inhibit protocols).
• Accessibility tools: AT-SPI2 is the Linux accessibility bus. Wayland support has improved substantially in GTK4 and Qt6, but some assistive technologies still rely on X11 for event injection.
• Wine and legacy games: Many Windows applications running under Wine still assume X11 semantics. Wine has Wayland driver support as of Wine 9.0, but compatibility is not universal—test per application.
• Remote desktop targets: If your machine is the target of an RDP or VNC session, X11 sessions are far simpler to set up with tools like x11vnc. Wayland remote desktop requires gnome-remote-desktop, wayvnc, or RDP-capable compositors.
• Specific GPU drivers: Proprietary NVIDIA drivers prior to driver version 545 had significant Wayland issues. On driver 545+ with GBM support, most problems are resolved, but check your specific driver version.

Mixed Sessions and XWayland

XWayland is a compatibility bridge: a special X11 server that itself runs as a Wayland client. It intercepts X11 protocol requests from legacy applications and translates them into Wayland surface operations. This is transparent to most applications—they believe they are talking to a real X server.

You can check whether XWayland is running in your session:

ps aux | grep Xwayland

To force a specific application to prefer Wayland (where it has a native Wayland backend), set the appropriate environment variable before launching it:

# For GTK applications
GDK_BACKEND=wayland myapp

# For Qt applications
QT_QPA_PLATFORM=wayland myapp

# For SDL applications
SDL_VIDEODRIVER=wayland myapp

To make these permanent for all GTK or Qt apps in your session, add the exports to ~/.profile or your compositor's environment file. On systemd-managed desktops:

systemctl --user import-environment GDK_BACKEND QT_QPA_PLATFORM

Conversely, if a native Wayland app is broken and you need it to fall back to XWayland temporarily:

GDK_BACKEND=x11 myapp

Switching Sessions at Login

Most display managers let you choose between Wayland and X11 sessions at login. On GDM (GNOME):

# To disable Wayland and force X11 for all users, edit:
sudo nano /etc/gdm3/custom.conf   # Debian/Ubuntu
sudo nano /etc/gdm/custom.conf    # Fedora/RHEL
# Set: WaylandEnable=false under [daemon]

On SDDM (KDE Plasma), select "Plasma (X11)" or "Plasma (Wayland)" from the session menu at the login screen. No config file edit is needed for per-session choice.

On Arch with any display manager, session files live in /usr/share/wayland-sessions/ and /usr/share/xsessions/. Install both packages for your desktop environment to have the choice available.

Verifying Your Active Session

Check which display protocol your current session is using:

echo $XDG_SESSION_TYPE

This will print wayland, x11, or mir. A quick sanity check that also shows the Wayland compositor name:

echo $WAYLAND_DISPLAY

Under a Wayland session this will be something like wayland-0. Under X11 it will be empty.

Troubleshooting Common Issues

Screen sharing does not work in browser or video call app

Install and enable the Pipewire and xdg-desktop-portal packages for your desktop environment. On Debian/Ubuntu:

sudo apt install xdg-desktop-portal xdg-desktop-portal-gnome pipewire pipewire-pulse wireplumber

On Fedora these are typically pre-installed. Confirm the portal is running:

systemctl --user status xdg-desktop-portal

Application renders as a tiny window or wrong scale on HiDPI

XWayland does not support fractional scaling natively until compositor-side fractional scaling protocols are used. Force the application to use native Wayland rendering with the backend environment variables shown above. For Electron apps, add --ozone-platform=wayland to the launch flags.

Cursor disappears or flickers in games

This is frequently an explicit sync issue between the GPU driver and compositor. Update to the latest mesa and kernel. For NVIDIA, confirm you are on driver 545 or later and that __GL_SYNC_TO_VBLANK is not set to 0.