Replace PM3 compile-from-source in pi-gen with pre-built tarball extraction (saves 43-58 min). Merge stagePM3 into stageDangerousPi as 02-pm3-install substage, renumber all subsequent substages. Switch CI PM3 build to native ARM64 runner (ubuntu-24.04-arm64) eliminating QEMU overhead. Add weekly base-image workflow for pre-baking stages 0-2. Support PM3_TARBALL, BASE_IMAGE, and APT_PROXY env vars in build-image.sh. Also includes prior Phase 5 work: theme system, design system integration, component update system, OS updates, CI build pipeline, and test results. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9.8 KiB
On-Device Testing & Iteration Guide
How to build, deploy, and test Dangerous Pi components on a live Pi Zero 2 W from a development laptop.
Prerequisites
- Pi connectivity: WiFi (
dangerous-pi.local/ IP) or Ethernet - Serial console: USB-to-serial adapter at
/dev/ttyUSB0(115200 baud) - Docker: With QEMU binfmt support for
linux/arm64 - ARM toolchain:
arm-none-eabi-gcc(firmware only) - Python venv:
.venv/withhttpx,pyserial,websockets
Connectivity
| Method | Address | Use |
|---|---|---|
| HTTP API | http://dangerous-pi.local:8000/api/ |
Backend testing |
| HTTP UI | http://dangerous-pi.local/ |
Frontend (via nginx) |
| HTTPS | https://dangerous-pi.local/ |
If HTTPS enabled |
| Serial | /dev/ttyUSB0 @ 115200 |
System commands, logs, deployment |
| Bluetooth | Local adapter | BLE scan testing |
1. Cross-Compiling Proxmark3
The PM3 has two components that must match versions: firmware (ARM7TDMI, runs on PM3 hardware) and client (aarch64, runs on Pi).
Source Setup
cd /home/work/dangerous-pi/.pm3-test/proxmark3
# Keep in sync with upstream
git fetch origin
git pull origin master
# Configure platform
cat > Makefile.platform << 'EOF'
PLATFORM=PM3GENERIC
LED_ORDER=PM3EASY
EOF
Build Firmware (Native, Fast)
Firmware targets ARM7TDMI (the PM3's own chip), so arm-none-eabi-gcc builds it natively on x86:
cd /home/work/dangerous-pi/.pm3-test/proxmark3
SKIPQT=1 make -j$(nproc) bootrom fullimage
Output:
bootrom/obj/bootrom.elf(~10KB)armsrc/obj/fullimage.elf(~370KB)
Build time: ~1 minute on x86.
Build Client (Docker aarch64, Slow)
The client runs on the Pi (aarch64), so we cross-compile using Docker with QEMU emulation:
# Build the Docker image (one-time)
cd /home/work/dangerous-pi/.pm3-test
docker build --platform linux/arm64 -t pm3-client-builder -f Dockerfile.pm3-client .
# IMPORTANT: Clean x86 build artifacts first
cd proxmark3
make client/clean
# Build the aarch64 client
docker run --rm --platform linux/arm64 \
-v "$(pwd):/build/proxmark3" \
pm3-client-builder \
bash -c "cd /build/proxmark3 && SKIPQT=1 make client"
Output:
client/proxmark3(aarch64 ELF, ~5.7MB)
Build time: ~25 minutes (QEMU emulation is slow).
Verify architecture:
file client/proxmark3
# Should show: ELF 64-bit LSB pie executable, ARM aarch64
Dockerfile
Located at .pm3-test/Dockerfile.pm3-client:
FROM --platform=linux/arm64 debian:trixie-slim
RUN apt-get update && apt-get install -y \
build-essential pkg-config libreadline-dev libbz2-dev \
liblz4-dev libgd-dev libjansson-dev libbluetooth-dev \
python3-dev swig cmake \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /build/proxmark3
Common Pitfalls
- Stale x86 objects: If you built on the host first,
.ofiles in the volume mount will be x86_64. Docker can't link them. Alwaysmake client/cleanbefore the Docker build. - Merge conflicts with LED patch: When pulling upstream, our
cmdhw.candpm3_cmd.hchanges may conflict. Resolve by keeping ourCMD_LED_CONTROLdefine andledcommand while taking upstream's new code.
2. Deploying to Pi
Transfer Files (HTTP Server Method)
From the laptop, serve the built files:
cd /home/work/dangerous-pi/.pm3-test/proxmark3
python3 -m http.server 9999 --bind $(hostname -I | awk '{print $1}')
From the Pi (via serial console or SSH):
LAPTOP_IP=10.0.0.202 # adjust to your laptop's IP
mkdir -p ~/pm3-update
# Download client and firmware
curl -o ~/pm3-update/proxmark3 http://$LAPTOP_IP:9999/client/proxmark3
curl -o ~/pm3-update/fullimage.elf http://$LAPTOP_IP:9999/armsrc/obj/fullimage.elf
curl -o ~/pm3-update/bootrom.elf http://$LAPTOP_IP:9999/bootrom/obj/bootrom.elf
chmod +x ~/pm3-update/proxmark3
Deploy Binaries
# Stop the service first
sudo systemctl stop dangerous-pi
# Backup current binaries
cp ~/.pm3/proxmark3/client/proxmark3 ~/.pm3/proxmark3/client/proxmark3.bak
cp ~/.pm3/proxmark3/firmware/fullimage.elf ~/.pm3/proxmark3/firmware/fullimage.elf.bak
# Deploy new binaries
cp ~/pm3-update/proxmark3 ~/.pm3/proxmark3/client/
cp ~/pm3-update/fullimage.elf ~/.pm3/proxmark3/firmware/
cp ~/pm3-update/bootrom.elf ~/.pm3/proxmark3/firmware/
Flash Firmware
cd ~/.pm3/proxmark3
./pm3-flash-all
After flashing, the PM3 resets. If it doesn't re-enumerate on USB (/dev/ttyACM0), physically unplug and replug the USB cable.
Restart Service
sudo systemctl start dangerous-pi
3. Deploying Backend Changes
For Python backend changes (e.g., ble_manager.py, system.py):
Quick Inline Fix (via serial sed)
For small changes, use sed -i over serial:
# Example: fix a string match in system.py
sudo sed -i 's/"subjectAltName"/"Subject Alternative Name"/' \
/opt/dangerous-pi/app/backend/api/system.py
sudo systemctl restart dangerous-pi
Full File Transfer (HTTP method)
# On laptop - serve the file
cd /home/work/dangerous-pi
python3 -m http.server 9999 --bind $(hostname -I | awk '{print $1}')
# On Pi - download and deploy
curl -o /tmp/ble_manager.py http://$LAPTOP_IP:9999/app/backend/managers/ble_manager.py
sudo cp /tmp/ble_manager.py /opt/dangerous-pi/app/backend/managers/ble_manager.py
sudo systemctl restart dangerous-pi
Deploying nginx Config
# On Pi
curl -o /tmp/dangerous-pi-https.conf http://$LAPTOP_IP:9999/nginx/dangerous-pi-https.conf
sudo cp /tmp/dangerous-pi-https.conf /opt/dangerous-pi/nginx/
sudo /opt/dangerous-pi/scripts/configure-nginx.sh
sudo systemctl reload nginx
4. Running the Hardware Test Suite
The test suite at test_hardware_2026-03-03.py validates everything against the live Pi.
Setup
source .venv/bin/activate
pip install httpx pyserial websockets
Run All Tests
python test_hardware_2026-03-03.py
Run Specific Phase
python test_hardware_2026-03-03.py --phase 1 # Regression (Jan 23 issues)
python test_hardware_2026-03-03.py --phase 2 # Core health endpoints
python test_hardware_2026-03-03.py --phase 3 # PM3 hardware
python test_hardware_2026-03-03.py --phase 4 # WiFi manager
python test_hardware_2026-03-03.py --phase 5 # HTTPS/SSL
python test_hardware_2026-03-03.py --phase 6 # Auth & WebSocket
python test_hardware_2026-03-03.py --phase 7 # Plugins
python test_hardware_2026-03-03.py --phase 8 # BLE & UPS
python test_hardware_2026-03-03.py --phase 9 # Frontend & performance
Safe Mode (Skip Destructive Tests)
python test_hardware_2026-03-03.py --safe-only
Results are written to TEST_RESULTS_2026-03-03_vN.md.
Test Phases Overview
| Phase | Tests | What It Covers |
|---|---|---|
| 1 | 4 | Regression of Jan 2026 issues (WiFi, UPS, BLE, Pydantic) |
| 2 | 9 | Health, readiness, system info, config, response times |
| 3 | 14 | PM3 device discovery, sessions, command execution |
| 4 | 6 | WiFi status, scanning, saved networks |
| 5 | 8 | SSL cert info, HTTPS access, cert scripts |
| 6 | 10 | Auth enable/disable, token flow, WebSocket |
| 7 | 10 | Plugin discover, load, enable, hooks, widgets |
| 8 | 10 | BLE status/advertising, UPS battery/thresholds |
| 9 | 8 | Frontend pages, response times, memory, CPU temp |
5. Useful Serial Console Commands
# Service management
sudo systemctl status dangerous-pi
sudo systemctl restart dangerous-pi
journalctl -u dangerous-pi -f # Follow logs
# Check PM3
ls /dev/ttyACM* # USB device present?
dmesg | tail -20 # USB errors?
~/.pm3/proxmark3/client/proxmark3 -p /dev/ttyACM0 -c "hw version"
# Check networking
ip addr show wlan0
nmcli connection show --active
curl -s http://localhost:8000/api/health
# Check SSL
ls -la /opt/dangerous-pi/ssl/
openssl x509 -in /opt/dangerous-pi/ssl/dangerous-pi.crt -text -noout | head -20
# Check BLE
journalctl -u dangerous-pi --no-pager | grep -i "ble\|bluetooth"
# Check UPS
journalctl -u dangerous-pi --no-pager | grep -i "ups\|pisugar\|battery"
6. Known Issues & Workarounds
PM3 USB Enumeration After Flash
After pm3-flash-all, the PM3 sometimes fails to re-enumerate on USB (dmesg shows error -71). Fix: physically unplug and replug the USB cable.
PM3 Client Exit Code 1
When client and firmware versions don't match, pm3 exits with code 1 (despite producing valid output). The SubprocessPM3Worker treats this as failure, putting output in the error field instead of output. Fix: ensure client and firmware are built from the same commit.
SWIG Library on Pi Zero
Building the SWIG Python bindings (cmake -DBUILD_PYTHON_LIB=ON) can OOM on Pi Zero 2 W (512MB RAM). The pi-gen chroot build handles this with swap, but on-device rebuilds may fail. The SubprocessPM3Worker (used by default) doesn't need SWIG — it shells out to the pm3 CLI.
SSL Directory Permissions
/opt/dangerous-pi/ssl/ must be readable by the dt user (backend runs as dt):
sudo chmod 755 /opt/dangerous-pi/ssl/
7. Pi-gen vs Cross-Compilation
| Aspect | pi-gen (image build) | Cross-compile (dev iteration) |
|---|---|---|
| When | Building release images | Day-to-day development |
| Where | Docker chroot (aarch64) | Laptop + Docker for client |
| Speed | Full build: hours | Firmware: 1min, Client: 25min |
| SWIG | Built in chroot | Not needed (SubprocessPM3Worker) |
| LED Patch | Applied via led-pwm-control.patch |
Maintained in git (merge conflicts) |
| Deploy | Baked into .img |
HTTP transfer + manual copy |
The pi-gen pipeline (stagePM3/01-proxmark3/00-run-chroot.sh) builds everything inside an aarch64 chroot — firmware, client, and SWIG bindings. For development iteration, cross-compilation from the laptop is faster and doesn't require a full image rebuild.