Architecture

Stack Overview

DisplayXR sits between the OpenXR API and vendor-specific display hardware. Applications write to the standard OpenXR interface. DisplayXR handles session management, compositor orchestration, and extension dispatch. Vendor-specific processing — weaving, interlacing, calibration — happens below, in the display processor layer. The same runtime ships on Windows, macOS, and Android.

Each graphics API gets its own native compositor. No cross-API interop or Vulkan intermediary is required.

Native Compositor Model

Most XR runtimes use a single graphics API internally and translate submitted textures as needed. DisplayXR takes a different approach: each supported graphics API has its own dedicated compositor implementation.

Platforms

DisplayXR runs on three platforms from one codebase. On Windows it drives LeiaSR displays through the D3D11/D3D12/Vulkan/OpenGL compositors; on macOS it ships the Metal, OpenGL, and MoltenVK paths against the simulation backend.

On Android, the same OpenXR runtime drives integrated 3D tablets and handhelds — such as ZTE's Nubia Pad 2 and Red Magic Explorer 3D — through the native Vulkan compositor. The vendor display processor runs out-of-process as a service, so the runtime stays vendor-neutral and apps connect over IPC with a zero-copy buffer handoff. Rendering is orientation-aware — portrait and landscape share one worst-case atlas, with no stall on live rotation — and the same display-zone and see-through transparency model used on the desktop lets a weaved 3D object sit beside a flat 2D HUD or float over the live screen.

One Model for 2D and 3D Regions

A spatial-display window is rarely all-3D. A weaved 3D object often needs to sit beside a flat 2D HUD, a toolbar, or live screen content. DisplayXR expresses every such layout through a single mechanism — XR_EXT_display_zones: an app declares any number of 3D zones (each a rectangle with its own view rig and swapchain) alongside any number of 2D zones, plus a per-pixel wish mask that tells the panel which areas should be physically 3D and which should stay flat. There is now exactly one way to say "this region is 3D, that region is 2D."

This replaces an earlier, narrower mechanism. The original 2D-surround / output-rect path could only express a single 3D rectangle surrounded by one monolithic 2D fill — a strict special case of the zone model, where the output rect is just one 3D zone and the surround one 2D zone. It was retired in runtime v1.25.0, folding both 2D and 3D region expression onto the same compositing path and removing the redundant per-API fill code.

Crucially, how an app owns its output surface is orthogonal to how it expresses regions. Whether the app draws into its own window, shares a texture with the runtime, or lets the runtime host a window for it, zones are how all of them carve up 2D and 3D. A plain full-window app is simply the degenerate one-zone case — it needs no explicit zones at all.

Shipping Components

A DisplayXR install delivers four cooperating pieces. Most applications only interact with the first; the others come into play when apps are sandboxed, when the shell is running, or when the web is the target surface.

How the Pieces Ship

DisplayXR is deliberately split across repositories so each piece evolves on its own cadence and the runtime stays vendor-neutral — the runtime binary carries no vendor SDK and no shell code. You can install everything at once with the meta-installer, or add components individually.

Workspace Controllers

The runtime is useful on its own. A bare install gives you a standards-compliant OpenXR + WebXR surface for a 3D display — full-screen apps, simulation backend, native compositors, the WebXR bridge. No spatial desktop, no windowing, no launcher.

A workspace controller is an optional process that adds spatial-desktop features on top: multi-app composition, window chrome, layout presets, an app launcher. The runtime exposes two extensions for this: XR_EXT_spatial_workspace for window pose / hit-test / capture, and XR_EXT_app_launcher for tile registration and lifecycle. Anything that speaks them is a first-class controller.

Activation is gated by orchestrator-PID match: the runtime trusts the binary it was configured to spawn, not a brand string. OEMs point a single service.json field at whichever controller the SKU should run.

Spec details: workspace-controller-registration.md.

AI Control Surface

DisplayXR exposes live spatial state and control to AI agents over the Model Context Protocol. The framework is a separate, embeddable library at displayxr-mcp — JSON-RPC 2.0 over a unix-socket / Windows-named-pipe transport, with a stdio bridge for any MCP client (Claude Code, voice CLI, custom agent).

The library has no runtime or shell coupling — any C project can consume it via CMake FetchContent and register its own tools. Every tool call is audit-logged and gated by a per-client allowlist.

End users opt in by installing DisplayXR MCP Tools ( releases) — an optional third installer alongside the runtime and the shell. It writes HKLM\Software\DisplayXR\Capabilities\MCP\Enabled, a registry capability flag that the runtime and the shell read at startup to spawn their MCP server thread. The DISPLAYXR_MCP=1 environment variable still works as a process-local override (CI / dev / quick-disable).

Two Compositor Paths

The runtime chooses at session creation whether to composite inside the application process or delegate to the service. This decision is transparent to the application — it just uses OpenXR as normal.

The runtime picks IPC automatically when it detects a sandboxed process (Chrome AppContainer, UWP) or a shell-managed session; otherwise it composites in-process.

Per-Graphics-API Design

This per-API design means no texture copies between APIs, no translation overhead, and no dependency on a single "blessed" graphics backend. The compositor that runs is the one that matches the application's chosen API.

The runtime selects the correct compositor at session creation time based on the graphics binding the application provides. This is transparent to the application — it simply uses OpenXR as normal.

Separation of Concerns

DisplayXR draws a clean boundary between two responsibilities:

Simulation Driver

DisplayXR includes a simulation display processor (sim_display) that allows development and testing without physical 3D display hardware. It provides the same interface as a hardware-backed display processor but renders to a standard window.

This means developers can build, test, and iterate on spatial display applications using any standard monitor. The simulation path supports all graphics APIs and all application classes.