Enable propolis to generate ACPI tables

[glitzflitz]

Background

As per #695
currently propolis relies on edk2-stable202105 version of EDK2 OVMF to provide the ACPI tables to the guest as it was the last version that has included static tables.
Another limitation is the guest only sees whatever OVMF decided to generate rather than what the hypervisor knows about the virtual/emulated hardware.

In newer versions, OVMF expects the VMM to generate a set of ACPI tables and expose them via the fw_cfg table-loader interface. Being able to generate ACPI tables also unlocks other opportunities for features like being able to chose which tables and control methods to expose, PCIe host bridge and switch emulation, supporting native PCIe hotplug etc.

This PR addresses that limitation and adds mechanism to let propolis generate its own ACPI tables.

Implementation

Oveview

The series starts with implementing fw_cfg's table-loader mechanism to enable passing static tables to guest firmware(OVMF). Then the basic static tables like RSDT, XSDT and RSDP etc are added.
After that we reach to second milestone that is generating the AML bytecode. This where some technical decision need to be made after evaluating different options and tradeoffs along with use case for how to go about generating bytecode without introducing too much complexity.
At the end everything is wired up to switch to using propolis generated tables.

Details

The fw_cfg Interface

• https://www.qemu.org/docs/master/specs/fw_cfg.html
• https://qemu.googlesource.com/qemu/+/refs/heads/staging-8.2/hw/acpi/bios-linker-loader.c

QEMU's fw_cfg interface provides a mechanism for the hypervisor to expose files to guest firmware. Propolis already had basic fw_cfg support for the e820 memory map and bootrom. The ACPI implementation builds on that foundation.

OVMF expects three specific fw_cfg files for ACPI tables:

etc/acpi/tables    Combined ACPI table data
etc/acpi/rsdp      Root System Description Pointer
etc/table-loader   Linker/loader commands

The table-loader file contains a sequence of fixed-size commands that instruct OVMF to allocate memory, patch pointer fields and compute checksums. This is necessary because the tables contain absolute addresses that are only known after OVMF allocates memory for them.

In the proposed implementation in Add fw_cfg table-loader helpers for ACPI table generation , TableLoader generates three command types:

ALLOCATE - reserves memory for a fw_cfg file with specified alignment in a given zone

ADD_POINTER - patches an address field in one file to point at another file's allocated location. The command specifies source file, destination file, offset within source and pointer size

ADD_CHECKSUM - computes a checksum over a byte range and writes it to a specified offset. ACPI tables use a simple byte sum that must equal zero.

The commands are used in Prepare the ACPI tables for generation

Static Table Generation

The simpler static tables that don't require AML bytecode are implemented first.

• Add RSDT, XSDT and RSDP tables
• Add FADT and DSDT table generation
• Add MADT table
• Add HPET table
• Add FACS table

Since propolis does not have hotplug support yet SSDT is not required as of now.

AML generation and usage

• Define AML opcode constants
• Add ACPI name encoding utilities
• Introduce AML bytecode generation
• Support resource construction for ACPI methods
• Wire up firmware/acpi module exports
• Generate DSDT with PCIe host bridge and serial ports
• Add PS/2 controller in DSDT
• Add PCIe _OSC method for OS capability negotiation

The DSDT contains AML bytecode for describing devices, methods and resources. AML has a hierarchical structure with scopes containing devices which contain named objects and methods. The encoding uses variable length packages.

Possible approaches

QEMU uses a C based approach with GArray buffers. Each AML construct is a function returning an Aml pointer that must be explicitly appended to its parent. The design is flexible but also has caveats for example, forgetting manual aml_append call silently drops content and there is no type safety around what can be nested. Since we are not bound my limitations of C and have borrow checker with us, we can do better.

crosvm defines a single Aml trait with many implementing types. Each construct is a separate struct collecting children in a Vec. The usage pattern is usually a macro followed by to_aml_bytes() which recursively serializes the tree. Although this provides strong typing, its bit more complex and requires constructing the entire tree in memory before serialization. Package lengths use a two pass approach of first measuring then writing.

Firecracker also follows a same pattern to crosvm with trait methods along with some additional error handling.

acpi_tables crate used by cloud-hypervisor: uses a dual trait design to split the problem into two traits: Aml for things that can be serialized and AmlSink as the destination. The sink abstraction is nice because the same tree can write to a Vec or feed a checksum calculator without changing the serialization code. Its structurally similar to crosvm and the same two pass length encoding which gets bit complex when building nested hierarchies.

Approach in this series

Introduce AML bytecode generation adds RAII guards that automatically finalize package lengths when dropped.
The core abstraction is an AmlBuilder that owns a single byte buffer plus guard types for Scope, Device and Method. Each guard holds a mutable borrow on the builder so we have compile time scope safety through the borrow checker. This way its impossible to miss closing any scope.
Also using single buffer from AmlBuilder avoids the overhead of dynamic dispatch as in crosvm and acpi_tables approach.

Guards borrow the builder mutably and write content directly to its buffer. When a guard is created it writes the opcode, reserves 4 bytes for the package length (the maximum encoding size) and writes the name. When the guard drops it calculates the actual package length, encodes it in 1-4 bytes and splices out the unused reserved bytes.

Usage looks like

let mut builder = AmlBuilder::new();
{
    let mut sb = builder.scope("\\_SB_");
    {
        let mut pci0 = sb.device("PCI0");
        pci0.name("_HID", &EisaId::from_str("PNP0A08"));
    }  // DeviceGuard drops
}  // ScopeGuard drops
let aml = builder.finish();

which looks structurally similar to ASL code that is compiled to AML bytecode.
The conditional content is simply an if statement due to RAII guards which avoids complexity of Option wrappers as needed in other cases mentioned above. The limitation in this design is that its less composable. There is no easy way to return a "partial device tree" from a function or store AML fragments for later use.

Note about Package Length Encoding

The ACPI specification Section 20.2.4 defines a variable length encoding for package sizes. A package length includes itself in the count which creates a circular dependency: the length must be known to encode it but the encoding affects the length. That is why two pass approach is often used as done by others.

The implementation in Introduce AML byte generation, simply reserves max 4 bytes when opening any scope and splices in the actual encoded length when the scope closes. This produces minimal output with a single pass through the data.

I'd be open to new ideas or going with another approach mentioned above as well :)

Wiring up new tables

• Wire up ACPI table generation via fw_cfg
  The new table generation is controlled by a native_acpi_tables flag in the Board spec. Newly launched VMs have this set to true and get new generated tables via fw_cfg. VMs migrating from older propolis versions won't have this field in their spec so it defaults to false and they keep using OVMF tables.
  So existing VMs can safely migrate to propolis generated tables without any guest visible changes to their ACPI tables. Only VMs launched with new version of propolis will use the new tables.

Testing

This is the dmesg of linux when using new tables. Now the standard OVMF bootrom can be used.

[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
...
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] APIC: Static calls initialized
[    0.000000] efi: EFI v2.7 by EDK II
[    0.000000] efi: SMBIOS=0xbf9a9000 ACPI=0xbfb7e000 ACPI 2.0=0xbfb7e014 MEMATTR=0xbe3f5298 INITRD=0xbe564f98
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: Oxide OxVM, BIOS v0.0.1-alpha 1 Bureaucracy 41, 3186 YOLD
...
[    0.000000] ACPI: RSDP 0x00000000BFB7E014 000024 (v02 OXIDE )
[    0.000000] ACPI: XSDT 0x00000000BFB7D0E8 00004C (v01 OXIDE  PROPOLIS 00000001      01000013)
[    0.000000] ACPI: FACP 0x00000000BFB7B000 000114 (v06 OXIDE  PROPOLIS 00000001 OXDE 00000001)
[    0.000000] ACPI: DSDT 0x00000000BFB7C000 000833 (v02 OXIDE  PROPOLIS 00000001 OXDE 00000001)
[    0.000000] ACPI: FACS 0x00000000BFBFC000 000040
[    0.000000] ACPI: APIC 0x00000000BFB7A000 000072 (v05 OXIDE  PROPOLIS 00000001 OXDE 00000001)
[    0.000000] ACPI: MCFG 0x00000000BFB79000 00003C (v01 OXIDE  PROPOLIS 00000001 OXDE 00000001)
[    0.000000] ACPI: HPET 0x00000000BFB78000 000038 (v01 OXIDE  PROPOLIS 00000001 OXDE 00000001)
[    0.000000] ACPI: BGRT 0x00000000BFB77000 000038 (v01 INTEL  EDK2     00000002      01000013)
[    0.000000] ACPI: Reserving FACP table memory at [mem 0xbfb7b000-0xbfb7b113]
[    0.000000] ACPI: Reserving DSDT table memory at [mem 0xbfb7c000-0xbfb7c832]
[    0.000000] ACPI: Reserving FACS table memory at [mem 0xbfbfc000-0xbfbfc03f]
[    0.000000] ACPI: Reserving APIC table memory at [mem 0xbfb7a000-0xbfb7a071]
[    0.000000] ACPI: Reserving MCFG table memory at [mem 0xbfb79000-0xbfb7903b]
[    0.000000] ACPI: Reserving HPET table memory at [mem 0xbfb78000-0xbfb78037]
[    0.000000] ACPI: Reserving BGRT table memory at [mem 0xbfb77000-0xbfb77037]

• Now we can easily use HPET in the guest. There are TSC calibration failures which were there when using OVMF as well

[    0.000000] ACPI: Core revision 20250404
[    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 113919999973 ns
[    0.000000] APIC: Switch to symmetric I/O mode setup
[    0.000000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.008000] tsc: Unable to calibrate against PIT
[    0.008000] tsc: HPET/PMTIMER calibration failed
[    0.008000] tsc: Marking TSC unstable due to could not calculate TSC khz

• We also have support for PCIe ECAM to use PCIe host bridges

[    0.313000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.316000] PCI: ECAM [mem 0xe0000000-0xefffffff] (base 0xe0000000) for domain 0000 [bus 00-ff]
[    0.316000] PCI: ECAM [mem 0xe0000000-0xefffffff] reserved as E820 entry
[    0.316000] PCI: Using configuration type 1 for base access

• The GlobalLock is not supported by propolis yet so the warning appears with OVMF tables as well

[    0.337000] ACPI: Added _OSI(Module Device)
[    0.337000] ACPI: Added _OSI(Processor Device)
[    0.337000] ACPI: Added _OSI(Processor Aggregator Device)
[    0.338000] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.340000] ACPI Error: Could not enable GlobalLock event (20250404/evxfevnt-182)
[    0.341000] fbcon: Taking over console
[    0.341000] ACPI Warning: Could not enable fixed event - GlobalLock (1) (20250404/evxface-618)
[    0.341000] ACPI Error: No response from Global Lock hardware, disabling lock (20250404/evglock-63)
[    0.342000] ACPI: Interpreter enabled

• Support for power state

[    0.342000] ACPI: PM: (supports S0 S3 S4 S5)
[    0.342000] ACPI: Using IOAPIC for interrupt routing

• Support for _OSC which enables use PCIe host bridges and PCIe Native Hotplugging in future

[    0.342000] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.342000] PCI: Using E820 reservations for host bridge windows
[    0.343000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.343000] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3]
[    0.343000] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER]
[    0.343000] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability LTR DPC]
[    0.343000] acpi PNP0A08:00: host bridge window expanded to [mem 0xc0000000-0xfbffffff]; [mem 0xc0000000-0xfbffffff window] ignored
[    0.343000] acpi PNP0A08:00: host bridge window [mem 0x240000000-0xfffffffffff window] ([0x10000000000-0xfffffffffff] ignored, not CPU
addressable)
[    0.343000] PCI host bridge to bus 0000:00
[    0.343000] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfbffffff]
[    0.343000] pci_bus 0000:00: root bus resource [mem 0x240000000-0xffffffffff window]
[    0.343000] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.343000] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.343000] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.344000] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint
[    0.350000] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint
[    0.357000] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint
[    0.362000] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
[    0.362000] pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB

• HPET from bhyve could be used

[    2.588000] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    2.588000] hpet0: 8 comparators, 32-bit 16.777216 MHz counter
[    2.591000] clocksource: Switched to clocksource hpet
[    2.588000] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    2.588000] hpet0: 8 comparators, 32-bit 16.777216 MHz counter
[    2.591000] clocksource: Switched to clocksource hpet

• PS/2 keyboard is also supported. Note that support can be easily extended to PS/2 mouse and AUX port but to maintain same behaviour as OVMF table only keyboard is exposed for now

[    3.047234] i8042: PNP: PS/2 Controller [PNP0303:KBD] at 0x60,0x64 irq 1
[    3.047234] i8042: PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
[    3.050217] serio: i8042 KBD port at 0x60,0x64 irq 1

• Then some more errors due to better emulate PM1 registers #837

[    3.063171] ACPI Error: Could not enable RealTimeClock event (20250404/evxfevnt-182)
[    3.075329] ACPI Warning: Could not enable fixed event - RealTimeClock (4) (20250404/evxface-618)
[

TODO:

• Test *bsd/illumos
• Test Windows
• Compare dump of actual tables from OVMF and new tables extracted from the guest?