SPARC V9 Boot Process – Part 1: bootblk

Introduction

SPARC V9 is a 64-bit Reduced Instruction Set Computer (RISC) architecture introduced in the mid-1990s that was widely used in servers and workstations manufactured by Sun Microsystems. As its popularity gradually declined, the hardware lost the performance advantage it once held in its early years and is now considered a legacy architecture.

Despite its age and competitive obsolescence, SPARC V9 is an interesting architecture to study. It has several unique characteristics, and some of us who lived through the dot-com boom still enjoy using Sun hardware for nostalgic reasons.

This article takes a closer look at the overlooked elements of the SPARC V9 boot process. It explores the OpenBoot firmware and demonstrates how to implement a first-stage bootloader from scratch.

OpenBoot

OpenBoot represents Sun Microsystems’ implementation of the IEEE 1275 Open Firmware standard, providing facilities for hardware configuration, system diagnostics, and operating system initialisation on SPARC V9 architectures. After completing the Power-On Self-Test (POST), OpenBoot firmware displays the ok prompt (provided the autoboot property is set to false), allowing the boot process to be started from a disk or over a network.

The OpenBoot PROM includes an FCode interpreter, which enables the execution of code written in a machine-independent interpreted language known as FCode. Firmware for pluggable storage adapters or even the first-stage bootloader can be written in FCode, allowing it to remain portable across different CPU architectures that implement a compatible FCode interpreter.

During initialisation, OpenBoot constructs the device tree — a hierarchical data structure that represents the system’s hardware and configuration parameters, and may also include firmware device drivers. The device tree consists of device nodes, where each node may have properties, methods, data, and other child nodes.

There are several standard system nodes; however, the /chosen node includes a number of useful properties that can be accessed by the first-stage bootloader. This node contains parameters chosen or specified at runtime:

Property Name	Encoding	Value
name	string	"chosen"
stdin	integer	Ihandle for the console input.
stdout	integer	Ihandle for the console output.
bootpath	string	The device path for the last boot device.
bootargs	string	The arguments to the last boot command.
memory	integer	Ihandle for the package that describes physical memory.
mmu	integer	Ihandle for the package that describes memory management unit.

Below is an example of what it looks like on a Sun T5220:

{0} ok dev /chosen
{0} ok .properties
bootargs                 
bootpath                 
mmu                      fff74080 
memory                   fff74290 
stdout                   feb51d88 
stdin                    feb51f90 
stdout-#lines            ffffffff 
name                     chosen

For example, the first-stage bootloader can use the hexadecimal value of the stdin ihandle to read text from the console, and the hexadecimal value of the stdout ihandle to write text to the console.

OpenBoot provides two primary interfaces: the user interface and the client interface. The user interface presents the ok prompt and offers a command-line environment for user interaction. The client interface supplies a set of services that can be invoked by a client program. A client program may be the first-stage bootloader, known as bootblk, and can comprise either FCode bytecodes or 64-bit SPARC V9 instructions packaged within an ELF64 binary.

In the following sections, I describe how a 64-bit ELF64 binary can call the OpenBoot client interface handler and invoke a range of services.

OpenBoot Client Interface Services

The OpenBoot firmware provides a number of services that can be invoked by the first-stage bootblk bootloader. The services are accessed via the client interface handler (a function pointer), with arguments and return values passed through an array of fixed size cells. A cell is a unit of storage in OpenBoot and on SPARC V9 its size is 8 bytes (64 bits).

The array passed to the OpenBoot client interface handler comprises the following cells:

Cell Name	Description
service	Pointer specifying the address of a '\0' terminated string of the client interface service
Num args	Integer specifying the number of input arguments to the client interface service
Num rets	Integer specifing the number of return values from the client interface service
Arg1	...
Arg2	...
ArgN	Input arguments to the client interface service
Ret1	...
Ret2	...
RetN	Return values from the client interface service

In the C programming language, a call to the write service can be implemented as shown in the following example:

/* 64-bit Open Firmware uses 64-bit storage cells */
typedef uint64_t ofw_cell_cdt;

/*
* Open Firmware ihandle is a handle/reference identifying a package instance.
* The ihandle always seems to be a 32-bit integer.
*/
typedef uint32_t ihandle_cdt;
#define OFW_IHANDLE_INVALID ((ihandle_cdt)-1)

/* Open Firmware Client Interface Function handler */
static int (*ofw_cif_handler)(void *);

/* Open Firmware ihandles for stdin and stdout */
static ihandle_cdt ihandle_stdin    = OFW_IHANDLE_INVALID;
static ihandle_cdt ihandle_stdout   = OFW_IHANDLE_INVALID;
static ihandle_cdt ihandle_bootpath = OFW_IHANDLE_INVALID;

/*
* Write to Open Firmware stdout.
*
* Arguments:
* strbuf     (in): String buffer.
* strbuf_len (in): String buffer length in chars.
*
* Pre-cond:
* 1. ofw_init() was executed earlier.
* 2. strbuf points to a valid string buffer.
* 3. strbuf_len indicates the number of string chars to write to stdout.
*
* Post-cond:
* 1. Characters written from strbuf to stdout.
*
* Return:
* On success: number of bytes copied from strbuf to stdout.
* On failure: (uint32_t)-1.
*/
uint32_t ofw_write_stdout(const char *strbuf, uint32_t strbuf_len)
{
    ofw_cell_cdt cif_arg[7];

    /* If stdout ihandle is not initialised, return -1 */
    if (ihandle_stdout == OFW_IHANDLE_INVALID) { return (uint32_t)-1; }

    cif_arg[0] = (ofw_cell_cdt)"write";        /* Service */
    cif_arg[1] = (ofw_cell_cdt)3;              /* Number of arguments */
    cif_arg[2] = (ofw_cell_cdt)1;              /* Number of return values */
    cif_arg[3] = (ofw_cell_cdt)ihandle_stdout; /* Arg1: stdout ihandle */
    cif_arg[4] = (ofw_cell_cdt)strbuf;         /* Arg2: strbuf address */
    cif_arg[5] = (ofw_cell_cdt)strbuf_len;     /* Arg3: strbuf length */
    cif_arg[6] = (ofw_cell_cdt)-1;             /* Ret1: return value */

    /* Call into cif handler */
    ASSERT(ofw_cif_handler(cif_arg) == 0);
    return (uint32_t)cif_arg[6];
}

For a specification of the client interface services, consult the IEEE 1275-1994 standard.

Sun VTOC Disk Label and Bootblk Requirements

When OpenBoot attempts to boot from a disk drive, it checks for a valid Sun VTOC disk label in the first 512-byte sector of the boot device. If a valid disk label is not found, OpenBoot displays an error message and aborts the boot process.

Once the Sun disk label has been checked, OpenBoot reads the next 16 512-byte sectors and loads bootblk first-stage bootloader. Since the number of sectors read during this stage is hard-coded into the firmware, the size of bootblk is limited to approximately 8 KiB and should fit within the 16 sectors immediately following the disk label. In my testing on a Sun T5220, I was able to load a bootblk slightly over 10 KiB in size; however, larger bootblk binaries resulted in the firmware error: "ERROR: Last Trap: Fast Data Access MMU Miss".

I developed a utility called vtoc_label, which can write a fake Sun VTOC disk label to a file or disk. The integer values within the disk label must be stored in big-endian byte order; therefore, the utility employs byte-order conversion functions to translate between host and network byte order, allowing it to generate correct disk labels even on little-endian architectures.

The bootblk is primarily written in the C programming language, with a small section containing the _start function implemented in SPARC V9 assembly. The boot stages are as follows:

1. Upon power-on, firmware begins to execute, probing devices, performing initialisation and running diagnostic routines.
2. Firmware enables the Memory Management Unit (MMU) and sets up a basic SPARC V9 trap table.
3. The boot command executes, specifying the boot device and boot arguments.
4. The first 512-byte sector is read from the boot device and the Sun VTOC disk label is checked for validity.
5. The bootblk binary is read from the next 16 512-byte sectors after the disk label.
6. The address of the OpenBoot client interface handler is copied to register %o4.
7. The bootblk binary format is checked (FCode, ELF64, etc) and then executed.
8. Control is transferred to the bootblk binary (i.e. first-stage bootloader), which can then load a larger second-stage bootloader or an operating system kernel.

Source Code and Boot Demo

The complete source code for the bootloaders is located here: sparcv9_bootloader.tar.gz. Below follows a demonstration of a simple bootblk first-stage bootloader.

Initially, I compile the vtoc_label utility and boot1.bin binary. Then, I use the vtoc_label and dd utilities to write the disk label followed by the bootloader:

# Include SPARC V9 cross-tools in current PATH
export PATH=${HOME:?}/gcc-15.2.0_sparc64-unknown-elf/bin:${PATH}

# Build native vtoc_label utility
cd ${SRC_DIR}/vtoc_label && gcc -O2 -std=c11 -Wall -pedantic \
  -D_FILE_OFFSET_BITS=64 -D_POSIX_C_SOURCE=200809L \
  -o ${BIN_DIR}/vtoc_label vtoc_label.c

# Build SPARC V9 stage-1 bootloader
cd ${SRC_DIR}/boot1 && sparc64-unknown-elf-gcc \
  -Os -std=c11 -Wall -pedantic -ffreestanding -nostdlib -nostartfiles \
  -o ${BIN_DIR}/boot1.bin start.s main.c ofw.c util.c && \
sparc64-unknown-elf-strip ${BIN_DIR}/boot1.bin

# Write Sun VTOC disk label and bootloader
${BIN_DIR}/vtoc_label write <device> &&
dd if=${BIN_DIR}/boot1.bin of=<device> bs=512 seek=1 &&
sync

Finally, I insert the USB flash drive into the Sun T5220 and boot from the drive containing my bootblk binary, which demonstrates basic input and output functionality on the system console:

{0} ok boot /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3/disk@0
Boot device: /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3/disk@0  File and args: 

Running SPARC V9 stage-1 bootloader

ihandle_stdin  0x100348
ihandle_stdout 0x100358

Type your name: root
Hello root. Press any key to exit: 
Program terminated
{0} ok

In the second article: SPARC V9 Boot Process – Part 2: sysboot, I will examine how the first-stage bootloader can load and execute larger binaries and overcome the 8 KiB size limitation imposed by the firmware.

References

IEEE 1275-1994 Standard for Boot Firmware
IEEE Draft Std P1275.1/D14a Standard for Boot Firmware