Lab 02 — ImageBuilder Firmware
Duration: 75 minutes
You met the Mango on stock GL.iNet firmware in Lab 01. This lab replaces that
firmware with a custom build: a sysupgrade .bin produced from
OpenWrt’s ImageBuilder and pinned to 23.05.3 / ramips / mt76x8.
The engineering exercise is package selection under a hard 16 MB NOR flash
ceiling. Tailscale, cloudflared, and python3 do not fit in NOR alongside a
kernel and bootloader. You confirm that by reading the package list, build
the image, and verify the resulting .bin against the size constraint.
Lab 03 will move those heavy packages onto a USB ExtRoot.
Reproducibility. The build sets
SOURCE_DATE_EPOCH=1714694400to strip squashfs timestamps. Two students building from the same commit on the same day produce identical SHA256 sums. “Is your firmware identical to mine?” becomes a one-line check.
Where each step runs
This lab runs entirely from the operator console (Codespace, local Dev
Container, or Minimal local; any path with Docker available).
Flashing the produced .bin to the Mango is Lab 03, not here. Lab 02
ends with a verified artifact on disk.
| Step | Operator console (any of: Codespace, Local Dev, Minimal local) |
|---|---|
| 1. Inspect contract | yes |
| 2. Build sysupgrade (ImageBuilder) | yes |
| 3. Verify size + manifest | yes |
| 4. Compare to upstream rootfs | yes |
| Validation | yes |
Learning objectives
- Understand the ImageBuilder model: pre-compiled packages assembled into a firmware image without a full source build.
- Read the
PACKAGESlist inbuild-drop-mango.shas the engineering document. Each addition or removal has a NOR-flash cost. - Use
SOURCE_DATE_EPOCHto produce bit-identical artifacts across hosts. - Read
build-manifest.jsonas a reproducibility audit trail. - Verify a squashfs size constraint programmatically. Know what to cut when it fails.
Pre-state
Before starting this lab confirm:
# Lab 01 is complete: SSH to the Mango works.
# (Skip this check if you are running Lab 02 ahead of bench access; the
# build itself does not need the Mango to be reachable.)
ssh -o ConnectTimeout=5 root@192.168.8.1 'echo ok' # stock GL.iNet; use 192.168.1.1 for pure-OpenWrt
# Docker is reachable from your operator console (this is the inner DinD
# daemon in Codespace / Local Dev Container, or your host docker for
# Minimal local).
docker info | grep 'Server Version'
# Pull the OpenWrt 23.05.3 ImageBuilder image (~800 MB; first run only).
docker pull openwrt/imagebuilder:ramips-mt76x8-23.05.3
# The lab's build inputs are checked into the repo. Confirm they exist.
ls courses/engagement-platform-labs/labs/Makefile
ls courses/engagement-platform-labs/labs/lab02-imagebuilder-firmware/build-drop-mango.sh
ls courses/engagement-platform-labs/labs/shared/files-mango/etc/banner
ls courses/engagement-platform-labs/labs/shared/build-manifest.schema.json
If any of these are missing, run git status. Restore with
git checkout HEAD -- <path>.
Optional pre-warm for Local Dev Container. Running
make engagement-platformfromlabs/builds the Debian operator-console Dev Container image and writes a manifest. This is orthogonal to ImageBuilder (it’s a normal Docker build, not an OpenWrt firmware build) but it is the same image VS Code triggers when you “Reopen in Container,” so pre-running it warms the cache. Codespace students skip this; the Codespace builds the image on launch.
Walkthrough
Spend five minutes reading the four files that define what goes into the firmware. The rest of the lab makes more sense once the package list is in your head.
# All commands run from the course root:
# cd courses/engagement-platform-labs
# The build script. The PACKAGES variable is the engineering document.
cat labs/lab02-imagebuilder-firmware/build-drop-mango.sh
# The shared files overlay baked into the squashfs:
cat labs/shared/files-mango/etc/banner
cat labs/shared/files-mango/etc/uci-defaults/99-enroll.sh.template
# Note: template placeholders ({{WORKER_URL}} etc.) are NOT substituted
# here. Lab 12 substitutes real secrets and rebuilds.
# The Makefile glue. `drop-mango` is the target you run in Step 2.
cat labs/Makefile
Read the PACKAGES list deliberately. The list is short and intentional:
- Base system:
base-files,busybox,fstools,libc,mtd,netifd,opkg,uci. The minimum you need for an OpenWrt boot. - Crypto + transport:
ca-bundle,ca-certificates,dropbear,libustream-mbedtls,urandom-seed,urngd. The Mango speaks TLS to the Cloudflare Worker and accepts SSH from the tailnet operator. - ExtRoot prerequisites:
block-mount,kmod-usb-storage,kmod-usb3,kmod-fs-ext4,e2fsprogs. These let Lab 03 mount a USB drive as the filesystem overlay. Without them in NOR, you couldn’t grow past 16 MB. - Minimal tooling:
curl,jsonfilter,tcpdump-mini. Just enough to talk to a Worker and capture a packet.
Read the negative entries (lines starting with -):
-luci,-luci-base,-luci-mod-admin-full,-luci-theme-bootstrap. No web UI. The drop device is operated over SSH and Tailscale, not a browser.-dnsmasq,-firewall4,-nftables,-kmod-nft-offload. No router-on-the-edge role. The Mango connects only to the tailnet; it does not provide DNS, NAT, or firewalling for downstream clients.-wpad-*,-odhcpd-ipv6only,-ppp*. Wi-Fi authenticator and IPv6/PPP daemons that ship in the default profile but are not used in the drop role.
Background reading. OpenWrt’s ImageBuilder docs describe the command-line flags this script is wrapping (
make image PROFILE= PACKAGES= FILES=). The package selection guide is helpful if you want to extend the list later.
The build runs inside the ImageBuilder Docker container. That image
ships with the cross-compilation toolchain and pre-built packages for
ramips/mt76x8, so you do not need a native MIPS toolchain on your
laptop. The docker-compose.yml file in labs/ wires the container
up; the Makefile target invokes it.
cd courses/engagement-platform-labs/labs
# The canonical entry point. Idempotent.
make drop-mango
The Makefile expands to:
docker compose run --rm imagebuilder \
/labs/lab02-imagebuilder-firmware/build-drop-mango.sh
Expected output (truncated):
>>> drop-mango build
PROFILE=glinet_gl-mt300n-v2
FILES_DIR=/labs/shared/files-mango
SOURCE_DATE_EPOCH=1714694400
OUTPUT_DIR=/labs/output
...
>>> built: bin/targets/ramips/mt76x8/openwrt-23.05.3-ramips-mt76x8-glinet_gl-mt300n-v2-squashfs-sysupgrade.bin
size: <N> bytes
sha256: <...>
squashfs rootfs: <M> bytes
>>> artifacts in /labs/output/
First-run timing. ImageBuilder downloads ~150 MB of package metadata
on the first invocation (the OpenWrt 23.05.3 feed for ramips/mt76x8).
Subsequent rebuilds reuse the container’s package cache. Allow 3–5
minutes the first time; under a minute thereafter.
If the build fails because the package list pushes the squashfs past the ceiling, the script exits with:
ERROR: squashfs rootfs N bytes exceeds ceiling 13631488
trim the PACKAGES list or move heavy packages to ExtRoot (Lab 03)
This is intentional. The constraint is the lesson.
# Confirm the artifact exists.
ls -lh labs/output/
# Expected:
# build-manifest.json
# openwrt-23.05.3-ramips-mt76x8-glinet_gl-mt300n-v2-squashfs-sysupgrade.bin
# ... and possibly the matching factory.bin and .manifest from ImageBuilder
# Locate the sysupgrade .bin and report its size in MB.
BIN=$(ls labs/output/*glinet_gl-mt300n-v2*sysupgrade.bin 2>/dev/null | head -1)
echo "$BIN"
wc -c < "$BIN" | awk '{printf "%.2f MB\n", $1/1024/1024}'
# Should be well under 16 MB. A typical clean build lands around 5–7 MB.
The 16 MB ceiling is the Mango’s NOR chip size. The build script enforces a tighter 13 MB cap on the rootfs alone, leaving headroom for U-Boot (~256 KB), the kernel (~1.5 MB), and a small reserved region. See the GL-MT300N-V2 device page for the partition layout.
# Read the build manifest. Required fields: role, openwrt_version, created_at.
# Optional but useful: openwrt_target, openwrt_profile, package_list_sha256,
# image_sha256, source_date_epoch.
cat labs/output/build-manifest.json
The manifest is the audit trail for “this exact firmware was built from
this exact source on this exact day.” package_list_sha256 is the SHA of
the resolved opkg package list (sorted, newline-delimited). When two
students compare builds:
- If the
image_sha256matches: identical bytes. Confidence high. - If
package_list_sha256matches butimage_sha256differs: the package list was the same, but a feed served different package metadata between the two builds. Investigate the feed mirror. - If
package_list_sha256differs: someone edited the PACKAGES list.
Compare with a peer:
sha256sum labs/output/*sysupgrade.bin
Two students who clone the same commit and build on the same day should see the same hash, byte for byte.
You met the upstream OpenWrt rootfs in Lab 01 Step 4: the
openwrt/rootfs:x86-64-23.05.3 sibling launched from the operator
console. That image is the unconstrained OpenWrt: ~127 packages,
busybox + opkg + dropbear + a kernel’s worth of modules + the standard
OpenWrt service plumbing, with no flash limit because it lives on a
container filesystem.
Your Mango sysupgrade is the constrained OpenWrt: a deliberately narrower package list selected to fit the 16 MB NOR chip. The interesting question is what survived the cut.
# 1) Read the package manifest ImageBuilder produced. Sorted, line-delimited:
ls labs/output/*glinet_gl-mt300n-v2*.manifest
cat labs/output/*glinet_gl-mt300n-v2*.manifest | sort
# 2) Count it.
wc -l labs/output/*glinet_gl-mt300n-v2*.manifest
# A clean drop-mango build lands around 90 packages (verified 2026-05-04
# at 93 lines). The PACKAGES list is small (~30 named entries), but opkg's
# dependency resolution pulls in libc, libgcc, mbedtls, kernel modules,
# wireless-regdb, and similar transitive deps, so the resolved manifest is
# roughly 3x the named-package count.
# 3) From inside the OpenWrt rootfs sibling (Lab 01 Step 4b), get its
# package count for comparison:
docker exec ep-devcontainer opkg list-installed | wc -l
# ~127 packages on the upstream x86_64 rootfs.
The point is not that the Mango image is smaller than the upstream rootfs. The point is that the Mango image carries a different kind of software:
| Category | Upstream rootfs (sibling) | Mango sysupgrade (this build) |
|---|---|---|
| Base + opkg | yes | yes |
| LuCI web UI | yes | no (drop role) |
| dnsmasq, firewall4 | yes | no (not a router) |
| ExtRoot prereqs | no (irrelevant on x86) | yes (USB grow path) |
| Tailscale / cloudflared | no | no (installed in Lab 03) |
Lab 03 takes this firmware, flashes it (sysupgrade -n), mounts a USB
drive as ExtRoot, and installs Tailscale and cloudflared on the overlay.
That sequence is only possible because Lab 02 reserved the
block-mount + kmod-usb-storage + kmod-fs-ext4 + e2fsprogs slots
in NOR.
Post-state
When this lab is complete you should be able to answer yes to all of the following:
-
labs/output/contains a*-glinet_gl-mt300n-v2-squashfs-sysupgrade.binfile produced by the latest build. -
wc -con that file reports a size below the 16 MB NOR ceiling (16,777,216 bytes). -
labs/output/build-manifest.jsonexists withrole: "drop-mango",openwrt_version: "23.05.3", and a recentcreated_attimestamp. -
validate.shexits 0. - You can name three packages that are present in the Mango image and
explain what each enables (canonical answer:
block-mountfor ExtRoot,dropbearfor SSH,ca-bundlefor outbound TLS). - You can name three packages that are deliberately absent from the
Mango image and explain why (canonical answer:
lucibecause no web UI is needed in the drop role;dnsmasqbecause the Mango is not a router;tailscalebecause it doesn’t fit in NOR and lives on ExtRoot in Lab 03).
Validation
Three validation paths. Pick whichever matches your setup. All three call
the same validate.sh and post the result to the same backend; only the
delivery channel differs.
Path 1: epl CLI (recommended from the operator console).
epl validate lab02-imagebuilder-firmware
The validator reads labs/output/build-manifest.json and the
*-sysupgrade.bin file; no Mango access required. Re-run as many times
as you like; only the most recent result is recorded.
Path 2: direct script.
bash courses/engagement-platform-labs/labs/lab02-imagebuilder-firmware/validate.sh
Or via the Makefile:
cd courses/engagement-platform-labs/labs
make validate-lab02-imagebuilder-firmware
Path 3: paste output into the widget.
Run validate.sh anywhere, copy the full output, paste it into the
widget below.
What the script checks:
labs/output/build-manifest.jsonexists.- The manifest contains the required fields (
role,openwrt_version,created_at). - The manifest validates against
labs/shared/build-manifest.schema.jsonwhenpython3is available. (Skipped with a[SKIP]note otherwise.) - A
*glinet_gl-mt300n-v2*sysupgrade.binfile exists inlabs/output/. - That
.binis smaller than 16 MB (the NOR chip size).
Take-home extension
See take-home/lab02-mt3000-build/ for
the parallel exercise on the cancelled MT3000 hardware
(mediatek/filogic, glinet_gl-mt3000). The MT3000 carries eMMC, so its
“drop firmware” has no 16 MB constraint. The contrast becomes a different
question: what does the MT3000 carry that the Mango cannot, and what
mission profile would use both?
The take-home directory is a stub during the May 2026 cohort and will be expanded post-workshop.
Primitives and drop-in substitutes
OpenWrt’s ImageBuilder is one implementation of a wider primitive: declarative firmware assembly. You name a target, a profile, a package list, and a file overlay; the toolchain produces a flashable artifact with a known size and a reproducible hash. The package selection is the engineering exercise; the toolchain that performs the assembly is interchangeable.
The table below maps that primitive to its closest equivalents across the embedded and systems-Linux landscape. Each tool accepts a declarative description of what should be in an image and outputs a bootable artifact. The variables are: what the “declaration” looks like, how the output is constrained, and who the upstream maintainer is.
| Name | Model | License | When to consider it |
|---|---|---|---|
OpenWrt full source build (make image) | Source-from-scratch; full toolchain compiles every package from C | GPL-2.0 | When ImageBuilder’s pre-compiled feed is missing a patch you need; when developing OpenWrt itself |
| Yocto Project / OpenEmbedded | Layered recipes assembled by BitBake; each layer can override any earlier layer | MIT (framework); various (generated binaries) | Industrial and commercial embedded products; vendors supply BSP layers for their SoCs |
| Buildroot | Declarative defconfig + Makefile; no package manager in the final image | GPL-2.0 | Smallest possible custom Linux; dev-board and appliance firmware with no runtime package management |
| Alpine Linux + apk + initramfs | apk packages assembled into a custom rootfs via mkimage/mkinitfs | MIT (apk-tools) | x86 / ARM appliance images; container base images where musl libc is acceptable |
| Debian debootstrap + custom kernel | debootstrap extracts a base, then on-host apt installs packages into a chroot | GPL/various | Targets with glibc and no flash constraint; common for SBCs (Raspberry Pi, BeagleBone) |
| mkosi | Declarative spec file (.conf) builds systemd-native images; outputs raw, GPT, or OCI | LGPL-2.1+ | Targets running systemd; the operator console Dev Container is a natural fit |
| OpenWrt-derivative distros (DD-WRT, FriendlyWrt, Padavan) | OpenWrt fork with vendor-specific patches and package sets | GPL-2.0 (mostly) | When the vendor’s tree carries drivers or features that mainline OpenWrt lacks for specific hardware |
package feed / sources
│
▼
assembly tool (ImageBuilder / Yocto / Buildroot / mkosi / debootstrap)
│
├── size constraint applied (NOR ceiling, eMMC budget, container layer limit)
│
▼
image artifact ──► SHA256 hash ──► reproducibility check
Take-home prompt. Pick a target on your shelf (RPi 4, an old PC with a 32 GB SSD, a dev board) and rebuild the equivalent of the drop-mango package list using one of the tools above. Compare the resulting artifact size, the hash-reproducibility properties, and the wall-clock time to first usable byte.
Further reading
ImageBuilder and reproducibility:
- OpenWrt ImageBuilder. Official documentation for the build system this lab wraps.
- Reproducible builds in OpenWrt.
The
SOURCE_DATE_EPOCHdiscipline and what it enforces. - opkg package management. How packages are resolved from feeds; useful when the manifest hash differs between students.
Hardware-specific:
- GL-MT300N-V2 (Mango) device page. Flash layout, supported releases, partition table.
ramips/mt76x8target. The MediaTek MT76x8 SoC family. The Mango is one of dozens of devices on this target.
Workshop-internal:
- Lab 03: Overlay Deployment. Flashes the
.binyou built here onto the Mango, mounts a USB drive as ExtRoot, and installs Tailscale on the overlay. - Lab 12: Drop Device. Re-runs this build with enrollment secrets baked into the firmware so the device self-registers on first boot.
View this page’s source:
labs/lab02-imagebuilder-firmware/README.mdx
Troubleshooting
make drop-mango: docker compose run fails (imagebuilder service not found)
The docker-compose.yml must be present in
courses/engagement-platform-labs/labs/. Verify:
ls courses/engagement-platform-labs/labs/docker-compose.yml
docker compose -f courses/engagement-platform-labs/labs/docker-compose.yml config
If the file is missing, check git status. It may be untracked or
deleted.
opkg download errors during build (mirror temporarily unavailable)
ImageBuilder fetches package metadata from downloads.openwrt.org on
first invocation. If a feed is temporarily unavailable, the build exits
non-zero. Retry:
make drop-mango
If retries also fail, check the OpenWrt mirror status at downloads.openwrt.org. Corporate networks sometimes block outbound on port 80; switching to a mobile hotspot for the first build is a fast workaround.
squashfs ceiling exceeded; build-drop-mango.sh exits with ERROR
You modified PACKAGES and pushed the rootfs past the 13 MB cap.
Identify the heaviest packages in the staged image:
docker compose run --rm imagebuilder \
find /home/buildbot/openwrt-imagebuilder-23.05.3-ramips-mt76x8.Linux-x86_64/bin/targets \
-name '*.ipk' | xargs ls -lS 2>/dev/null | head -20
Candidate packages to remove first: luci-* (large), nginx*, any
extra kmod-* for unused subsystems. Do not remove block-mount,
kmod-usb-storage, kmod-fs-ext4, or e2fsprogs. They are required
for ExtRoot in Lab 03.
build-manifest.json validation skipped (python3 not found)
validate.sh uses python3 to validate the manifest against the JSON
schema. If your host lacks Python 3, the validator emits a [SKIP]
notice and continues without that check. The other four assertions
still run.
To run the schema check, either install python3 on the host
(apt install python3 / brew install python3) or run validation from
inside the operator console (which has python3 baked in):
bash courses/engagement-platform-labs/labs/lab02-imagebuilder-firmware/validate.sh
SHA256 differs from a peer’s build, but PACKAGES is identical
Most often this means the OpenWrt package feed served slightly different
package metadata between the two builds. The package_list_sha256 in
build-manifest.json will agree, but image_sha256 will not.
Solutions, in order of effort:
- Re-run both builds at the same time so they hit the same feed snapshot.
- Pin a feed mirror by editing
/etc/opkg/distfeeds.confinside the ImageBuilder container. - Accept the drift and rely on
package_list_sha256as the reproducibility check; this is what the OpenWrt project itself does for non-strict reproducibility.
ValidateOutputPaster lab="lab02")