Continuing from Part 1.

Let’s look at what this looks like in practice, and where QEMU’s handling of it actually falls over. The latest good breakpoint was alloc_page_freelists+0x54 (I spent some time catching it):

[0]> alloc_page_freelists+0x54::bp
[0]> :c
kmdb: stop at alloc_page_freelists+0x54
kmdb: target stopped at:
alloc_page_freelists+0x54:      stx       %i0, [%l6 + %l7]
[0]> mmu_exported_pagesize_mask/X
mmu_exported_pagesize_mask:
mmu_exported_pagesize_mask:     2b
[0]> kmem64_szc/X
kmem64_szc:
kmem64_szc:     5 - <this is 256M page for this specific allocation >
[0]> kmem64_base/J
kmem64_base:
kmem64_base:    70000000000
[0]> kmem64_aligned_end/J
kmem64_aligned_end:
kmem64_aligned_end:             70010000000
[0]> 70000000000/J
0x70000000000:  0
[0]> 7000000F000/J
0x7000000f000:  0
[0]> 7000000FF00/J
0x7000000ff00:  0
[0]> 7000000FFF0/J
0x7000000fff0:  0
[0]> 7000000FFFE/J -<hang where we try to look the code out of 64K>

Here, kmem64_szc is 5, which corresponds directly to TTE256M from the table in Part 1 - confirming that the kernel is indeed trying to map the kmem64 segment using 256M pages, driven by the broken_md_flag logic, but it can’t do that.

The hang happens exactly at the boundary where we step outside the 64K range - right where QEMU would need to service a translation for the 256M-sized mapping. This lines up with the suspicion from Part 1: QEMU’s MMU emulation may not correctly handle the 256M page size that the guest kernel enables as a side effect of the missing domaining-enabled property. The guest isn’t doing anything unusual - it’s following the exact logic in niagara.c - but QEMU’s handling of that particular [sz] encoding looks like the likely point of failure, producing the endless stream of Hyperprivileged Trap Instruction / Data Access MMU Miss entries seen in the log at the start of this investigation. This remains a suspicion rather than a confirmed root cause.

When we tried to suppress the pagesize to 0x9 (8K+4M), like so:

[0]> ::bp alloc_kmem64
[0]> :c
[0]> mmu_exported_pagesize_mask/W b

the kernel booted successfully, but without 256MB pages.

I then started looking at the QEMU code. The TTE page-size field is defined in QEMU’s /target/sparc/cpu.h as:

#define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)

and the sun4v_tte_to_sun4u() conversion inside QEMU’s target/sparc/ldst_helper.c only preserves 2 bits for the page size:

    sun4u_tte |= (sun4v_tte & 3ULL) << 61; /* TTE_PGSIZE */

It seems the logic only preserves two bits (61/62), giving a value of 0-3, when the highest bit - bit 48 (szh) - is also supposed to be part of it, which makes the operation a bit more involved: we need to reconstruct it as

| szh[48] | szl[62] | szl[61] | |—|—|—|

to build the table described below in the UltraSPARC T1 Supplement draft: to hold all the properties in our case.

Interestingly, when I tried to figure out when this extension was introduced into the CPU spec, I found the Panther/USIV CPU listed as the first to support it, then T1 supported the same TTE pagesize bits, but T2 changed the format:

When I started prototyping the fix with Claude, it took five iterations to land on this patch and send it to the qemu-devel community. Each of the first four had something wrong: they used the wrong bitfields for storing the [szh] bit - the first time we built it on unused bit 42, which is out of UltraSparc spec :), and it even worked, sort of; or used suboptimal extra operations to store this bit in the TTE. All of these worked, but…

With it applied, the guest started to boot:

[0]> alloc_page_freelists+0x54::bp
[0]> :c
kmdb: stop at alloc_page_freelists+0x54
kmdb: target stopped at:
alloc_page_freelists+0x54:      stx       %i0, [%l6 + %l7]
[0]> kmem64_szc/X
kmem64_szc:
kmem64_szc:     5
[0]> kmem64_base/J
kmem64_base:
kmem64_base:    70000000000
[0]> kmem64_aligned_end/J
kmem64_aligned_end:
kmem64_aligned_end:             70010000000
[0]> mmu_exported_pagesize_mask/X
mmu_exported_pagesize_mask:
mmu_exported_pagesize_mask:     2b
[0]> 7000000FFFE/J <- it's survived!
0x7000000fffe:  0
[0]> 70000010000/J
0x70000010000:  0
[0]> 70000100000/J <- also survived
0x70000100000:  0
[0]> :c
kmdb: stop at alloc_page_freelists+0x54
kmdb: target stopped at:
alloc_page_freelists+0x54:      stx       %i0, [%l6 + %l7] - <then it stopped the second time
[0]> 70000100000/J
0x70000100000:  0
[0]> mmu_exported_pagesize_mask/X
mmu_exported_pagesize_mask:
mmu_exported_pagesize_mask:     2b
[0]> :c
SunOS Release 5.11 Version snv_151a 64-bit
Copyright (c) 1983, 2010, Oracle and/or its affiliates. All rights reserved.

And here’s how it looks under the QEMU monitor:

(qemu) info tlb
(qemu) info tlb
MMU contexts: Primary: 0, Secondary: 0
DMMU Tag Access: 180a000, TSB Tag Target: 0
DMMU dump
[00] VA: 70000000000, PA: d0000000, 256M, priv, RW, unlocked, ie no, ctx 0 local <<<<<-- there's the first 256M chunk, visible to QEMU!
[01] VA: ffffffffffffc000, PA: 801ee000,   8k, priv, RW, unlocked, ie no, ctx 0 local
[02] VA: ffffffffff300000, PA: f1f0c000,   8k, priv, RW, unlocked, ie no, ctx 0 local
[03] VA: fffffffcc000a000, PA: f1394000,   8k, priv, RW, unlocked, ie no, ctx 0 local
[04] VA: 1930000, PA: ff530000,   8k, priv, RW, unlocked, ie no, ctx 0 local