Boxes Of Tat: NetBSD/sparc64 9.2 on QEMU

NetBSD

A member of the open source family of Berkeley Software Distribution (BSD) UNIX-like operating systems, along with FreeBSD and OpenBSD, NetBSD is available for a wide range of hardware platforms (Platforms supported by NetBSD), including support for many systems of historical interest.

SPARC64

Released in 1993 the SPARC Version 9 specification added 64-bit support to the SPARC processor architecture. The first workstations to market were the HAL SPARC64 (Wikipedia) based Fujitsu HALstation Model 300 series in October 1995, followed by the Sun UltraSPARC based Sun Ultra workstations (Wikipedia) in November 1995.

QEMU and SPARC64

As well as the 32-bit 'sun4m' emulations QEMU also provides 64-bit UltraSPARC emulation (sparc64), providing 'sun4u' (UltraSPARC) and 'sun4v' (UltraSPARC T1) based platforms. This is newer than the mature sun4m emulations, so don't expect the sparc64 emulation to be as stable and well featured.

The default QEMU machine for UltraSPARC is a generic PCI-based 'sun4u' system:

CPU: TI UltraSPARC IIi (SUNW,UltraSPARC-IIi)
RAM: 128 MiB default, max. > 4 GiB
Disk controller: PATA
Network controller: Sun "Happy Meal" Ethernet (hme0)
Graphics: generic VGA with VESA support

The use of the UltraSPARC IIi makes this most similar to the Sun Ultra 5 and Ultra 10 workstations, which were very popular during the late 1990s.

The emulated 'sun4u' system is limited to a single processor. It appears that it can support more memory than any of the late 1990s UltraSPARC based Sun Ultra workstations (max. 4 GiB in Sun Ultra 80). The revival of the Ultra brand in the mid-2000s featured a line of UltraSPARC based workstations which had support for more memory (max. 16 GiB in Ultra 45).

Graphics

While the graphics work for the OpenBIOS firmware and the early stages of booting, it fails when the operating system tries to switch into graphics mode.

The popular Ultra 5 and Ultra 10 workstations used integrated Sun PGX graphics, based on the ATI 3D Rage II+DVD (PGX8, 2MiB) or ATI 3D Rage Pro (PGX24, 4 MiB) chipsets, depending on the mainboard. For the other UltraSPARC systems Sun sold PCI PGX graphics cards to add graphics output or for multi-monitor configurations: PGX8, PGX32 (3Dlabs Permedia 2) and PGX64 (ATI Rage XL, 8 MiB). Later models would use Sun XVR based graphics cards based on the ATI Redeon chipsets. Sadly QEMU does not, currently, provide emulations of those chipsets, so operating systems require the -nographic option to setup.

QEMU Command

For those interested, the emulated system run here was started using the command:

$ qemu-system-sparc64 -m size=1024M -hda hda_NetBSD_sparc64.qcow -cdrom NetBSD-9.2-sparc64.iso -boot c -net nic -net user -nographic

For installation the firmware was told to boot from the installation ISO with -boot d instead of the boot from the system disk option (-boot c) used here. The -nographic option is used to avoid problems with the emulated graphics card in NetBSD/sparc64.

System Information

So lets see what NetBSD has to say about itself and the emulated system it is running on.

uname

Operating system name, version, revision and platform information.

$ uname -a
NetBSD qnetbsd9.home 9.2 NetBSD 9.2 (GENERIC) #0: Wed May 12 13:15:55 UTC 2021  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/sparc64/compile/GENERIC sparc64

That is a NetBSD, version 9.2, release "NetBSD 9.2 (GENERIC) #0: Wed May 12 13:15:55 UTC 2021 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/sparc64/compile/GENERIC" on a 64-bit SPARC platform (sparc64), with the node name "qnetbsd9.home".

dmesg

System boot messages.

$ dmesg
[     1.000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[     1.000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[     1.000000]     2018, 2019, 2020 The NetBSD Foundation, Inc.  All rights reserved.
[     1.000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[     1.000000]     The Regents of the University of California.  All rights reserved.
[     1.000000] NetBSD 9.2 (GENERIC) #0: Wed May 12 13:15:55 UTC 2021
[     1.000000]         mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/sparc64/compile/GENERIC
[     1.000000] total memory = 1024 MB
[     1.000000] avail memory = 987 MB
[     1.000000] timecounter: Timecounters tick every 10.000 msec
[     1.000000] Kernelized RAIDframe activated
[     1.000000] running cgd selftest aes-xts-256 aes-xts-512 done
[     1.000000] mainbus0 (root): OpenBiosTeam,OpenBIOS: hostid 80123456
[     1.000000] cpu0 at mainbus0: SUNW,UltraSPARC-IIi @ 100 MHz, CPU id 0
[     1.000000] cpu0: manuf 17, impl 12, mask 91
[     1.000000] cpu0: 256K instruction (64 b/l), 16K data (32 b/l), 256K external (64 b/l)
[     1.000000] psycho0 at mainbus0
[     1.000000] psycho0: SUNW,sabre: impl 0, version 0: ign 7c0 bus range 0 to 2; PCI bus 0
[     1.000000] pci0 at psycho0
[     1.000000] pci0: i/o space, memory space enabled
[     1.000000] ppb0 at pci0 dev 1 function 1: Sun Microsystems Simba PCI Bridge (rev. 0x11)
[     1.000000] pci1 at ppb0 bus 1
[     1.000000] pci1: i/o space, memory space enabled
[     1.000000] ebus0 at pci1 dev 1 function 0: Sun Microsystems PCIO Ebus2, revision 0x01
[     1.000000] clock0 at ebus0 addr 2000-3fff: mk48t59
[     1.000000] power at ebus0 addr 7240-7243 ipl 1 not configured
[     1.000000] fdthree at ebus0 addr 0-ffffffff not configured
[     1.000000] com0 at ebus0 addr 3f8-3ff ipl 2b: ns16550a, working fifo
[     1.000000] com0: console
[     1.000000] pckbc0 at ebus0 addr 60-67 ipl 29autoconfiguration error: : no intr 1: selftest ok
[     1.000000] pckbd0 at pckbc0 (kbd slot)
[     1.000000] wskbd1 at pckbd0 mux 1
[     1.000000] pms0 at pckbc0 (aux slot)
[     1.000000] wsmouse0 at pms0 mux 0
[     1.000000] hme0 at pci1 dev 1 function 1: Sun Happy Meal Ethernet, rev. 0
[     1.000000] hme0: interrupting at ivec 3021
[     1.000000] hme0: Ethernet address 52:54:00:12:34:56
[     1.000000] nsphy0 at hme0 phy 1: DP83840 10/100 media interface, rev. 1
[     1.000000] nsphy0: 100baseTX-FDX, auto
[     1.000000] genfb0 at pci1 dev 2 function 0: vendor 1234 product 1111 (rev. 0x02)
[     1.000000] cmdide0 at pci1 dev 3 function 0: CMD Technology PCI0646 (rev. 0x07)
[     1.000000] cmdide0: bus-master DMA support present
[     1.000000] cmdide0: primary channel configured to native-PCI mode, channel non-independant
[     1.000000] cmdide0: using ivec 1820 for native-PCI interrupt
[     1.000000] atabus0 at cmdide0 channel 0
[     1.000000] cmdide0: secondary channel configured to native-PCI mode, channel non-independant
[     1.000000] atabus1 at cmdide0 channel 1
[     1.000000] ppb1 at pci0 dev 1 function 0: Sun Microsystems Simba PCI Bridge (rev. 0x11)
[     1.000000] pci2 at ppb1 bus 2
[     1.000000] pci2: i/o space, memory space enabled
[     1.000000] pcons at mainbus0 not configured
[     1.000000] timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
[     1.000003] timecounter: Timecounter "tick-counter" frequency 100000000 Hz quality 100
[     1.000003] No counter-timer -- using %tick at 100MHz as system clock.
[     4.143247] wd0 at atabus0 drive 0
[     4.163424] wd0: <QEMU HARDDISK>
[     4.163424] wd0: drive supports 16-sector PIO transfers, LBA48 addressing
[     4.163424] wd0: 6144 MB, 12483 cyl, 16 head, 63 sec, 512 bytes/sect x 12582912 sectors
[     4.173378] wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
[     4.183315] wd0(cmdide0:0:0): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA)
[     4.183315] atapibus0 at atabus1: 2 targets
[     4.193752] cd0 at atapibus0 drive 0: <QEMU DVD-ROM, QM00003, 2.5+> cdrom removable
[     4.203927] cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
[     4.203927] cd0(cmdide0:1:0): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA)
[     4.243486] WARNING: 1 error while detecting hardware; check system log.
[     4.253250] root on wd0a dumps on wd0b
[     4.263063] root file system type: ffs
[     4.272781] kern.module.path=/stand/sparc64/9.2/modules

Lots of stuff here, so I'll only pick out some highlights, if you want to work through it you may find the NetBSD man pages for section 4 "Special Files and Hardware Support" helpful (NetBSD/sparc64 man 4 intro).

Since modern operating systems tend to be a little more memory hungry than those the original UltraSPARC systems were released with, I've upped the memory from the default 128 MiB to a more comfortable 1.0 GiB (1024 MiB). The processor is reported as an UltraSPARC IIi at 100 MHz. The disk drives show the QEMU finger print in the reported manufacturer and device name strings.

sysctl

A tool to manipulate system information variables in the kernel. Here I've limited the output to only the hardware related items (hw group).

$ sysctl -a | grep '^hw'
hw.machine = sparc64
hw.model = OpenBiosTeam,OpenBIOS (SUNW,UltraSPARC-IIi @ 100 MHz)
hw.ncpu = 1
hw.byteorder = 4321
hw.physmem = 1073741824
hw.usermem = 1057128448
hw.pagesize = 8192
hw.disknames = wd0 cd0
hw.machine_arch = sparc64
hw.alignbytes = 15
hw.cnmagic = \x27\x01
hw.physmem64 = 1073741824
hw.usermem64 = 1057128448
hw.iostatnames = wd0 cd0
hw.ncpuonline = 1
hw.uhso.autoswitch = 1
hw.fwohci.nocyclemaster = 0
hw.fwohci.phydma_enable = 1
hw.ieee1394if.try_bmr = 1
hw.ieee1394if.hold_count = 0
hw.ieee1394if.ieee1394_debug = 0
hw.fwmem.eui64_hi = 0
hw.fwmem.eui64_lo = 0
hw.fwmem.speed = 2
hw.fwmem.fwmem_debug = 0
hw.fwip.rx_queue_len = 256
hw.fwip.if_fwip_debug = 0
hw.sbp.auto_login = 1
hw.sbp.max_speed = -1
hw.sbp.exclusive_login = 1
hw.sbp.login_delay = 1000
hw.sbp.scan_delay = 500
hw.sbp.use_doorbell = 0
hw.sbp.tags = 0
hw.sbp.sbp_debug = 0
hw.ath.dwell = 200
hw.ath.calibrate = 30
hw.ath.outdoor = 1
hw.ath.countrycode = 0
hw.ath.regdomain = 0
hw.ath.debug = 0
hw.ath.rxbuf = 40
hw.ath.txbuf = 200
hw.ath.hal.dma_brt = 2
hw.ath.hal.sw_brt = 10
hw.ath.hal.swba_backoff = 0
hw.wsevent.default_version = 1
hw.firmware.path = /libdata/firmware:/usr/libdata/firmware:/usr/pkg/libdata/firmware:/usr/pkg/libdata
hw.ubsec.maxbatch = 1
hw.ubsec.maxaggr = 1
hw.cpu0.name = SUNW,UltraSPARC-IIi
hw.cpu0.id = 0
hw.cpu0.clock_frequency = 100000000
hw.cpu0.ver = 6474004206454023
hw.wd0.use_ncq = 1
hw.wd0.use_ncq_prio = 0

Again, a bit too much information here, most of which repeats the stuff in the dmesg output, so just the highlights...

The hw.model variable contains the hardware model name, which is a generic OpenBIOS name for this generic 'sun4u' system. This also includes the processor identification as an UltraSPARC IIi at 100 MHz, which is also covered in the CPU section (hw.cpu0.*)

pcictl

The PCI bus control application, can get a report of the devices on the bus:

# pcictl pci0 list
000:01:0: Sun Microsystems Simba PCI Bridge (PCI bridge, revision 0x11)
000:01:1: Sun Microsystems Simba PCI Bridge (PCI bridge, revision 0x11)
001:01:0: Sun Microsystems PCIO Ebus2 (miscellaneous bridge, revision 0x01)
001:01:1: Sun Microsystems PCIO Happy Meal Ethernet (ethernet network)
001:02:0: vendor 1234 product 1111 (VGA display, revision 0x02)
001:03:0: CMD Technology PCI0646 (IDE mass storage, interface 0x8f, revision 0x07)
NULL phandle
Unexpected client interface exception: -1

Things don't seem quite right here, with a recurring error appearing after listing the devices. But the device information reported shows the expected devices installed.

The UltraSPARC IIi processor includes a PCI bus controller, which gives the main PCI bus ("Sabre" bus). The "Sabre" bus connects to two PCI bridge devices, one for the mainboard devices, and the other for the PCI slots. Knowing this the reported PCI configuration makes sense, with the two "Simba" bridge devices at the top on bus '000', followed by the mainboard devices on bus '001' supporting the I/O device, ethernet, VGA graphics and Parallel ATA controller.

The reporting of the 'vendor' and 'product' identifiers instead of their associated strings for the VGA controller, indicates that there is no matching information in NetBSD's hardware database.

atactl

Using sysctl we can get a list of the disks installed in the system:

# sysctl hw.disknames
hw.disknames = wd0 cd0

The wd prefix (NetBSD man) indicates that the drive is on an ATA bus, so we can use atactl to get more information about it (if it was a SCSI disk (sd) we would use scsictl instead):

# atactl wd0 identify
Model: QEMU HARDDISK, Rev: 2.5+, Serial #: QM00001
Device type: ATA, fixed
Capacity 6442 Mbytes, 12582912 sectors, 512 bytes/sector
Cylinders: 12483, heads: 16, sec/track: 63
Physical sector size: 512 bytes
Device capabilities:
        DMA
        LBA
        IORDY operation
Device supports following standards:
ATA-4 ATA-5 ATA-6 ATA-7 
Command set support:
        NOP command (enabled)
        Write cache (enabled)
        SMART feature set (enabled)
        FLUSH CACHE EXT command (enabled)
        FLUSH CACHE command (enabled)
        48-bit Address feature set (enabled)
TRIM supported
# atactl cd0 identify
Model: , Rev: , Serial #: 
Device type: ATA, removable
Capacity 0 bytes, 0 sectors, 512 bytes/sector
Cylinders: 0, heads: 0, sec/track: 0
Device capabilities:

The cd devices (NetBSD man) are a bit special, since we can also look at them as SCSI devices:

# scsictl cd0 identify
/dev/rcd0: scsibus0 target 0 lun 0 >QEMU, QEMU DVD-ROM, 2.5+>

Where we see the manufacturer and drive name strings, which were missing from the ATA version of the information.

Benchmark

Since this is an emulated machine an obvious question is, how well does it perform compared to the real thing? To find out we're going to need a benchmark.

BogoMips

The BogoMips (Wikipedia) pseudo-benchmark has a behavior where the score is related to the processor clock speed, and has an easily available collection of results from a range of systems (see BogoMips mini-Howto). This is helpful in this case since the performance of processors of this era typically scaled with clock speed. So let's get some results using the standalone BogoMips 1.3 program and see what it says:

Calibrating delay loop.. ok - 214.74 BogoMips
Calibrating delay loop.. ok - 218.24 BogoMips
Calibrating delay loop.. ok - 216.48 BogoMips
Calibrating delay loop.. ok - 214.74 BogoMips
Calibrating delay loop.. ok - 216.48 BogoMips

On UltraSPARC systems the BogoMips score is approx. twice the processor clock speed, so the maximum score of about 220 BogoMips suggests performance equivalent to an UltraSPARC running at 110.0 MHz. Checking online the slowest released Sun Ultra workstations had a processor clock of 143 MHz (giving around 286 BogoMips), so this emulation is a little on the slow side when compared to Sun hardware. However the first 64-bit SPARC processors were used in the Fujitsu HALstation Model 330 and Model 350 workstations, these use a SPARC64 processor (Wikipedia) running at 101 Mhz and 118 MHz respectively, and ran a fork of Solaris 2.4 called SPARC64/OS. These workstations would be around the performance level we're seeing.

OpenSSL

The OpenSSL library of cryptographic methods provides a means to test the performance of the methods. Using this we can get a feel for the computational performance of a system, and comparisons with real hardware and other emulations can show how well a particular emulation performs. Since we are mainly interested in old hardware, we'll test using two of the older methods: MD5 and RSA.

$ openssl speed md5
Doing md5 for 3s on 16 size blocks: 812486 md5's in 3.02s
Doing md5 for 3s on 64 size blocks: 659360 md5's in 3.01s
Doing md5 for 3s on 256 size blocks: 342044 md5's in 3.02s
Doing md5 for 3s on 1024 size blocks: 153124 md5's in 3.01s
Doing md5 for 3s on 8192 size blocks: 24631 md5's in 3.00s
Doing md5 for 3s on 16384 size blocks: 12456 md5's in 3.00s
OpenSSL 1.1.1k  25 Mar 2021
NetBSD 9.2
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr) 
gcc version 7.5.0 (NetBSD nb4 20200810) 
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
md5               4304.56k    14019.61k    28994.46k    52092.68k    67259.05k    68026.37k
$ openssl speed rsa
Doing 512 bits private rsa's for 10s: 1275 512 bits private RSA's in 10.01s
Doing 512 bits public rsa's for 10s: 14227 512 bits public RSA's in 10.00s
Doing 1024 bits private rsa's for 10s: 229 1024 bits private RSA's in 10.04s
Doing 1024 bits public rsa's for 10s: 4879 1024 bits public RSA's in 10.00s
Doing 2048 bits private rsa's for 10s: 37 2048 bits private RSA's in 9.68s
Doing 2048 bits public rsa's for 10s: 1496 2048 bits public RSA's in 10.01s
Doing 3072 bits private rsa's for 10s: 13 3072 bits private RSA's in 10.37s
Doing 3072 bits public rsa's for 10s: 649 3072 bits public RSA's in 10.04s
Doing 4096 bits private rsa's for 10s: 6 4096 bits private RSA's in 10.00s
Doing 4096 bits public rsa's for 10s: 422 4096 bits public RSA's in 10.01s
Doing 7680 bits private rsa's for 10s: 1 7680 bits private RSA's in 10.97s
Doing 7680 bits public rsa's for 10s: 113 7680 bits public RSA's in 10.03s
OpenSSL 1.1.1k  25 Mar 2021
NetBSD 9.2
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr) 
gcc version 7.5.0 (NetBSD nb4 20200810) 
                  sign    verify    sign/s verify/s
rsa  512 bits 0.007851s 0.000703s    127.4   1422.7
rsa 1024 bits 0.043843s 0.002050s     22.8    487.9
rsa 2048 bits 0.261622s 0.006691s      3.8    149.5
rsa 3072 bits 0.797692s 0.015470s      1.3     64.6
rsa 4096 bits 1.666667s 0.023720s      0.6     42.2
rsa 7680 bits 10.970000s 0.088761s      0.1     11.3

Extracting the relevant figures for comparisons (see OpenSSL Speed Results):

OpenSSL speed MD5 8,192 bytes: 67,259.05k
OpenSSL speed RSA 4,096 bytes sign/s: 0.6
OpenSSL speed RSA 4,096 bytes verify/s: 42.2

These will be roughly indicative of computational workload performance on this machine.

Thoughts

The speed of the processor being slower than any of the actual Sun Ultra workstations may place some restrictions on roles that can be fulfilled by the emulated machine. However performance of other emulated hardware elements (e.g. disk and network) may offset the slower processor, and make this a viable environment for more tasks than would be initially expected.

Sources

Update 24-Jun-2021: added OpenSSL benchmark results

Boxes Of Tat

Friday, 28 May 2021

NetBSD/sparc64 9.2 on QEMU - System Information