pi-pm3/docs/COMPONENT_UPDATE_SYSTEM.md

# Component-Based Update System

## Context

The current update system is monolithic — a single `.tar.gz` downloaded from GitHub Releases, extracted to `/opt/dangerous-pi`, followed by `make clean && make` to rebuild PM3 on-device. This is slow, resource-heavy, and fragile on a Pi. The goal is to:

1. Ship **pre-built PM3 binaries** (cross-compiled in CI) instead of compiling on-device
2. Support **per-component updates** (update just the frontend, just the PM3 client, etc.)
3. Have the update system **auto-manage requirements** like Python version matching for SWIG `.so` files

## Component Model

Four independently-updateable components. 3rd-party themes use the plugin system instead.

| Component | Artifacts | Install Path | Platform Constraints |
|-----------|-----------|-------------|---------------------|
| `pm3` | `proxmark3` binary, `libpm3rrg_rdv4.so`, `_pm3.so`, `pm3.py`, `pm3-flash-*`, patches, `bootrom.elf`, `fullimage.elf` | `$HOME/.pm3/proxmark3/` | aarch64-linux, Python version must match |
| `frontend` | Remix build output (`build/client/`, `build/server/`), app components, routes | `/opt/dangerous-pi/app/frontend/build/` | None |
| `backend` | Python source, `requirements.txt` | `/opt/dangerous-pi/app/backend/` | Python 3.11+ |
| `theme` | Generated token CSS from `@dangerousthings/tokens` + `themes.json` registry | `/opt/dangerous-pi/app/frontend/themes/` | None |

The PM3 client and firmware are always distributed together — they're built from the same source tree with the same patches, and a version mismatch between client and firmware could cause flashing issues.

### Theme Distribution (two channels)

**Source of truth:** The `dt-design-system` monorepo at `/home/work/dt-design-system/` (Turborepo + npm workspaces).

**Monorepo packages:**
- `@dangerousthings/tokens` — TypeScript token definitions → generates `dist/css/{dt,classic,supra}.css`
- `@dangerousthings/web` — Structural CSS (bevels, glows, elevation, fonts) + imports token CSS
- `@dangerousthings/react-native` — RN themed components (DTButton, DTCard, etc.)

**Currently:** The frontend imports `@dangerousthings/web/dist/index.css` which bundles *everything* (tokens + structural CSS) at Remix build time. This means a theme color change requires a full frontend rebuild.

**New approach:** Split token CSS (brand colors, typography, shapes) from structural CSS (bevels, glows, elevation):
- **Structural CSS** stays bundled with the frontend build (changes rarely, tied to component code)
- **Token CSS** (`dt.css`, `classic.css`, `supra.css`) is served dynamically from a `themes/` directory and loaded via `<link>` tags — can be updated without rebuilding the frontend

**Built-in themes** (dt, classic, supra) are distributed as the `theme` component via the update system. They ship with tested releases and are always available.

**3rd-party themes** are distributed as plugins via the existing plugin system. A theme plugin provides token CSS + `theme.json` and registers via the `theme_register` plugin hook.

Both channels write to the same dynamic theme registry, which replaces the current hardcoded `themes[]` array in `ThemeContext.tsx`.

**Theme package structure** (same format for both channels):
```
{theme-id}/
  theme.json          # id, name, description, supportsModes, defaultMode, author
  tokens.css          # CSS custom properties (--color-*, --radius-*, --font-*, --shape-*)
```

## Implementation Phases

### Phase 1: Manifest System (Foundation)

Add component tracking without changing update behavior yet.

**Files to modify:**
- `app/backend/managers/update_manager.py` — Add dataclasses and manifest I/O

**New dataclasses:**
```python
class ComponentId(str, Enum):
    PM3 = "pm3"
    FRONTEND = "frontend"
    BACKEND = "backend"
    THEME = "theme"

@dataclass
class ComponentInfo:
    component_id: str
    version: str
    installed_at: str
    checksum: Optional[str] = None
    install_path: str = ""
    platform: Optional[str] = None
    python_version: Optional[str] = None
    metadata: Dict[str, Any] = field(default_factory=dict)

@dataclass
class SystemManifest:
    schema_version: int = 1
    system_version: str = "0.1.0"
    components: Dict[str, ComponentInfo] = field(default_factory=dict)
    last_updated: str = ""
```

**New file:** `/opt/dangerous-pi/component-manifest.json` — replaces the single `VERSION` file.

**Migration:** On first startup with no manifest, `_migrate_from_legacy()` reads the existing `VERSION` file and PM3 `VERSION.txt` to populate a manifest with current component versions.

---

### Phase 2: CI/CD Build Pipeline

Cross-compile PM3 and package all components in GitHub Actions.

**New files:**
- `.github/workflows/build-release.yml` — Main workflow, triggered on tag push (`v*`)
- `ci/build-pm3.sh` — Runs in aarch64 Docker container (QEMU), reproduces the build from `pi-gen/stagePM3/01-proxmark3/00-run-chroot.sh`:
  - Clone RfidResearchGroup/proxmark3
  - Apply `led-pwm-control.patch` + `hf-booster-detection.patch`
  - Apply DangerousPi branding
  - Set `PLATFORM=PM3GENERIC`, `LED_ORDER=PM3EASY`
  - Build firmware (`bootrom.elf`, `fullimage.elf`) + client + SWIG bindings in one pass
  - `cmake -DBUILD_PYTHON_LIB=ON`, SWIG bindings
  - Matrix build for Python 3.11 + 3.12
- `ci/build-frontend.sh` — `npm ci && npm run build`, package `build/`
- `ci/build-backend.sh` — Package `app/backend/` + `requirements.txt`
- `ci/build-theme.sh` — Builds `@dangerousthings/tokens` in the `dt-design-system` monorepo, packages the generated `dist/css/{dt,classic,supra}.css` + a `themes.json` registry
- `ci/generate-manifest.py` — Generates `release-manifest.json` with checksums

**Release asset naming:**
```
release-manifest.json
pm3-{ver}-aarch64-linux-cp312.tar.gz
pm3-{ver}-aarch64-linux-cp311.tar.gz
frontend-{ver}.tar.gz
backend-{ver}.tar.gz
theme-{ver}.tar.gz
```

**`release-manifest.json` structure:**
```json
{
  "schema_version": 1,
  "release_version": "0.2.0",
  "components": {
    "pm3": {
      "version": "4.19544-dp1",
      "assets": {
        "aarch64-linux-cp312": {
          "filename": "pm3-4.19544-dp1-aarch64-linux-cp312.tar.gz",
          "checksum_sha256": "...",
          "size": 12345678,
          "python_version": "3.12"
        }
      },
      "changelog": "..."
    }
  }
}
```

---

### Phase 3: Component-Aware Update Logic

Refactor `UpdateManager` to download/install individual components.

**Files to modify:**
- `app/backend/managers/update_manager.py`

**Key changes:**
1. `_fetch_latest_release()` → fetch and parse `release-manifest.json` from GitHub release assets (fall back to legacy `.tar.gz` scanning for old releases)
2. `_select_pm3_asset()` → auto-detect Pi's Python version (`sys.version_info`) and architecture (`platform.machine()`) to pick the correct pm3 variant
3. `check_for_updates()` → return per-component availability with compatibility info
4. New `download_component(component_id)` → download a single component tarball
5. New `install_component(component_id)` → component-specific install logic:
   - `pm3`: copy client + firmware to `$HOME/.pm3/proxmark3/`, create `_pm3.so` symlink, verify arch
   - `frontend`: replace `build/` directory
   - `backend`: replace source, run `pip install -r requirements.txt` in venv
   - `theme`: extract to `/opt/dangerous-pi/app/frontend/themes/`, update theme registry
6. **Delete `_rebuild_pm3_client()`** — no more on-device compilation
7. New `_backup_component()` / `rollback_component()` → per-component backups at `/opt/dangerous-pi/backups/{component_id}/`

**New status:** `ROLLING_BACK` added to `UpdateStatus` enum.

**New progress tracking:**
```python
@dataclass
class ComponentProgress:
    component_id: str
    status: UpdateStatus
    download_progress: float = 0.0
    error_message: Optional[str] = None
```

Added as `component_progress: Dict[str, ComponentProgress]` to `UpdateProgress`.

---

### Phase 4: Unified Plugin Update Checking

Integrate plugin update checks into the same update flow so "Check for Updates" covers everything.

**Files to modify:**
- `app/backend/managers/update_manager.py` — Add plugin update checking alongside component checks
- `app/backend/managers/plugin_manager.py` — Add `check_plugin_updates()` method that queries each installed plugin's GitHub repo for newer releases

**How it works:**
1. `check_for_updates()` calls both:
   - Core component check (from `release-manifest.json` on the main repo)
   - Plugin update check (queries each plugin's source repo for newer tags)
2. Plugin update info is returned alongside component updates in the same response
3. Plugin download/install still uses the existing `plugin_manager.py` install logic — the update manager just orchestrates the version check
4. Plugin versions are tracked in `component-manifest.json` under a `plugins` key (separate from `components`)

**Manifest extension:**
```json
{
  "components": { ... },
  "plugins": {
    "hello_world": {
      "version": "1.0.0",
      "source_repo": "dangerous-tacos/dp-plugin-hello-world",
      "installed_at": "2026-03-03T10:00:00Z"
    }
  }
}
```

**Update check response includes plugins:**
```python
class PluginUpdateInfo(BaseModel):
    plugin_id: str
    current_version: str
    available_version: str
    changelog: str
    download_size: Optional[int]
```

The `/api/updates/check` response adds `plugins: List[PluginUpdateInfo]` alongside `components`.

---

### Phase 5: API & Service Layer (includes plugin update endpoints)

**Files to modify:**
- `app/backend/services/update_service.py` — Add `download_component()`, `install_component()`, `rollback_component()`, `get_installed_components()`
- `app/backend/api/updates.py` — New endpoints + extended response models

**New endpoints:**
```
GET  /api/updates/components                    — installed component manifest
POST /api/updates/components/{id}/download      — download one component
POST /api/updates/components/{id}/install       — install one component
POST /api/updates/components/{id}/rollback      — rollback one component
```

**Modified endpoints (backward-compatible):**
- `GET /check` — adds `components: List[ComponentUpdateInfo]` to response
- `POST /download` — accepts optional `{"components": ["frontend"]}` body; no body = all
- `POST /install` — same optional body pattern
- `GET /progress` — adds `active_component` and `components` list to response

---

### Phase 5: Dynamic Theme Registry

Decouple token CSS from the frontend build so themes can be updated independently.

**Current flow (bundled):**
`@dangerousthings/web/dist/index.css` → imports token CSS + structural CSS → bundled by Vite into Remix build

**New flow (split):**
- Structural CSS (`base.css`, `bevels.css`, `glows.css`, `elevation.css`, fonts) stays in the Remix build via `@dangerousthings/web`
- Token CSS (`dt.css`, `classic.css`, `supra.css`) served from `/themes/{brand}/tokens.css` and loaded dynamically

**Files to modify:**
- `app/frontend/app/lib/ThemeContext.tsx` — Replace hardcoded `themes[]` array with dynamic loading from `GET /api/themes`; inject `<link>` tag for active brand's token CSS
- `app/frontend/app/root.tsx` — Remove token CSS from the static import of `@dangerousthings/web/dist/index.css`; keep only structural CSS. Add dynamic `<link>` for token CSS.
- `app/backend/api/system.py` — Add `GET /api/themes` endpoint that reads available themes from disk
- `app/backend/main.py` — Mount `/themes/` as a static file directory

**Files to create:**
- `app/frontend/themes/` — Directory for installed theme packages (in dev)
- `/opt/dangerous-pi/app/frontend/themes/` — Same, in production
- Each theme: `themes/{theme-id}/theme.json` + `tokens.css`

**How it works:**
1. Backend scans `themes/` directory + plugin-registered themes → serves merged list at `GET /api/themes`
2. `ThemeContext` fetches available themes on mount (with hardcoded fallback for SSR/offline)
3. On brand switch, `ThemeContext` updates `<link href="/themes/{brand}/tokens.css">` in the document head
4. Plugin themes register via `theme_register` hook, providing their CSS path + `theme.json`

**Migration from `@dangerousthings/web`:**
- `@dangerousthings/web` needs a new export that excludes token CSS: `@dangerousthings/web/dist/structural.css` (or the `index.css` is refactored to not import token CSS)
- The CI `build-theme.sh` step copies `@dangerousthings/tokens/dist/css/{dt,classic,supra}.css` into the theme tarball as `{brand}/tokens.css`

**Theme plugin hook** (added to plugin_manager.py):
```python
# Plugin calls this in on_enable():
self.register_hook("theme_register", self._register_theme)

def _register_theme(self):
    return {
        "id": "my-theme",
        "name": "My Custom Theme",
        "css_path": self.plugin_dir / "tokens.css",
        "theme_json": self.plugin_dir / "theme.json"
    }
```

---

### Phase 6: Frontend UI

**File to modify:**
- `app/frontend/app/routes/updates.tsx`

**Layout:**
1. **System overview card** — system version, last check, "Check for Updates" + "Update All" buttons
2. **Component cards** — one per component showing:
   - Name, installed version, available version
   - Compatibility badge (for pm3 Python version match)
   - Collapsible per-component changelog
   - Individual Download / Install / Rollback buttons
   - Per-component progress bar
3. **Plugin update cards** — shown for any installed plugins with updates available, using the same card layout. Download/install delegates to the existing plugin manager.
4. **Status states per card:** up-to-date (muted), available (accent), downloading (progress bar), installing (spinner), complete (success), failed (danger + rollback button), incompatible (warning + reason)

---

### Phase 7: Testing & Polish

- Test legacy migration (no manifest → auto-generated manifest)
- Test partial update (update only frontend, leave PM3 alone)
- Test Python version mismatch (pm3-client shows incompatible)
- Test rollback per component
- Test disk space check before download
- Test backward compat with a legacy monolithic release (no `release-manifest.json`)
- Test theme update (update `theme` component without touching frontend)
- Test 3rd-party theme plugin (install, register, appear in theme selector, uninstall)
- Test plugin update check (installed plugin with newer release shows in update check response)
- Test plugin update install (update delegates to plugin_manager install logic)

## Verification

1. **Manifest migration**: Start backend with no `component-manifest.json` — verify it auto-creates from `VERSION` files
2. **CI pipeline**: Push a test tag — verify all component tarballs + `release-manifest.json` are produced
3. **Component check**: `GET /api/updates/check` — verify per-component version comparison
4. **Selective update**: Download + install only `frontend` — verify only frontend files change
5. **PM3 binary**: Install `pm3-client` — verify binary runs, `_pm3.so` symlink works, no compilation triggered
6. **Rollback**: Install a component, then `POST /rollback` — verify files restored and manifest updated
7. **Python version matching**: On a Pi with Python 3.12, verify `cp312` asset is selected; `cp311` shown as incompatible
8. **Theme update**: Update `theme` component — verify new CSS applied without frontend rebuild
9. **Theme plugin**: Install a theme plugin — verify it appears in theme selector alongside built-in themes
10. **Plugin updates**: Install a plugin, publish a newer release to its repo — verify `GET /api/updates/check` includes it in the response and install works via existing plugin manager