otp-import/docs/design.md

Apex OTP Importer — Design

Date: 2026-07-08 Status: Draft for review

Goal

A single runnable Python script that reads the curated OTP export (otp-keys-export.xlsx, sheet OTP Keys) and writes every credential to a VivoKey Apex's OTP (OATH) applet over PC/SC, quickly and repeatably. It must support password-protected OATH applets (authenticate before writing) and, optionally, profile per-credential on-chip memory consumption.

Context and constraints

• Host: Windows 11, Python 3.11. Installed and used: pyscard, cryptography, openpyxl. No Yubico libraries (ykman/yubikit) — the OATH protocol is implemented directly, so there is no added dependency and no reliance on Yubico device enumeration (Apex is not a Yubico device).
• Transport: PC/SC. Multiple contactless readers are typically present (ACR1252 CL, OMNIKEY 5022, Identiv uTrust 3700 F CL, VivoKey Smart Reader).
• Input source of truth: the curated columns of the spreadsheet (Issuer, Account, Secret, Type, Algorithm, Digits, Period) — NOT the otpauth:// URI column. The curated Account values were cleaned during export (e.g. Heroku → amal@vivokey.com, not the QR label "Amal Graafstra").

Verified protocol facts

Verified directly against the applet sources in the local repos apex-totp (ykneo-oath, com.vivokey.otp.YkneoOath / OathObj) and javacard-memory (de.chrz.jcmemory.JCMemoryApplet), plus apex-totp/doc/Protocol.txt.

OATH applet

• Applet AID (from apex-totp/build.xml): A0 00 00 05 27 21 01 01 41 50 45 58 01 (41 50 45 58 = "APEX"). SELECT strategy: try this full AID first, then fall back to A0 00 00 05 27 21 01 01 (8 B, per repo docs/tests) and A0 00 00 05 27 21 01 (7 B, canonical Yubico). Use the first that returns a valid SELECT response (version tag 0x79). Prefix selection works because Yubico Authenticator already talks to Apex with the short AID.
• INS: PUT 0x01, DELETE 0x02, SET CODE 0x03, RESET 0x04 (P1P2 must be 0xDEAD), RENAME 0x05, LIST 0xA1, CALCULATE 0xA2, VALIDATE 0xA3, CALCULATE ALL 0xA4, SEND REMAINING 0xA5.
• Tags: NAME 0x71, NAME_LIST 0x72, KEY 0x73, CHALLENGE 0x74, RESPONSE 0x75, T_RESPONSE 0x76, NO_RESPONSE 0x77, PROPERTY 0x78, VERSION 0x79, IMF 0x7A, ALGORITHM 0x7B.
• Type|algo byte: type in high nibble (HOTP 0x10, TOTP 0x20), algo in low nibble (SHA1 0x01, SHA256 0x02). SHA512 is rejected by this applet. All our credentials are TOTP/SHA1 → 0x21.
• SELECT response TLV: VERSION(0x79) + NAME(0x71, 8-byte device id/salt) and, if a password is set, CHALLENGE(0x74, 8 bytes) + ALGORITHM(0x7B, 1 byte). Presence of the CHALLENGE tag ⇒ applet is locked.
• PUT data: NAME(0x71,len,name) + KEY(0x73, len=2+keylen, [typeAlgo, digits, key…]). PROPERTY/IMF omitted (no touch, TOTP only). Name ≤ 64 B, key ≤ 64 B. The applet XORs exactly keylen bytes into the 64-byte HMAC ipad/opad, so a short secret yields the same HMAC as a zero-padded one; we still right-pad short secrets to 14 B to match the Yubico host (HMAC-identical).
• LIST response (repeats): NAME_LIST(0x72, len=namelen+1, [type, name…]).
• Long responses (LIST/CALCULATE ALL) chunk with SW 61xx; fetch the rest with SEND REMAINING (0xA5) until 9000.
• Response codes: success 9000; applet full 6A84; auth required 6982; wrong syntax / wrong data (incl. wrong password on VALIDATE) 6A80; no such object 6984.
• Auth state is stored in a CLEAR_ON_DESELECT buffer: any deselect (e.g. a memory-applet SELECT) drops authentication and requires a fresh VALIDATE.

Password authentication (VALIDATE)

Per doc/Protocol.txt and OathObj:

1. key = PBKDF2-HMAC-SHA1(password_utf8, salt = 8-byte device id from SELECT, iterations = 1000, dklen = 16). PBKDF2 always uses HMAC-SHA1 here, independent of the credential/challenge algorithm.
2. The challenge-response HMAC uses the algorithm named by the SELECT ALGORITHM (0x7B) tag (low nibble: SHA1 0x01 / SHA256 0x02), defaulting to SHA1 — which is what the Yubico host sets in practice (20-byte response). deviceResponse = HMAC(key, deviceChallenge) where deviceChallenge is the SELECT 0x74 value.
3. clientChallenge = os.urandom(8).
4. Send VALIDATE (0xA3): RESPONSE(0x75, deviceResponse) + CHALLENGE(0x74, clientChallenge[8]). The response length must equal the applet's digest length or it is rejected — hence honoring the ALGORITHM tag.
5. Applet returns RESPONSE(0x75, HMAC(key, clientChallenge)); the host verifies it to confirm mutual auth. Wrong password → 6A80.

The derived key is stable across re-SELECTs (device id only changes on RESET), so it is derived once and reused when re-validation is needed during profiling.

Memory applet (optional)

• Applet AID: A0 00 00 08 46 6D 65 6D 6F 72 79 01 (46 6D…79 = "Fmemory").
• SELECT returns 12 bytes: availPersistent(4 B BE), totalBaseline(4 B BE), transientReset(2 B), transientDeselect(2 B). Apex Flex baseline = 84336 B.
• GET_BATCH = 00 01 00 00 00 → 4-byte batch id.
• Each SELECT runs requestObjectDeletion() (GC) then reports free memory. Because selecting it deselects OATH, profiling must re-VALIDATE afterward.

Architecture

One runnable file apex_otp_import.py, organized into focused units:

• Tlv helpers — encode/parse BER-TLV with 1/2/3-byte lengths (0x81/0x82).
• CardChannel — thin wrapper over a pyscard connection: transmit(apdu) plus send(cla, ins, p1, p2, data, le) that transparently follows 61xx with SEND REMAINING and returns (data, sw).
• OathSession — SELECT (with AID fallback), parse device id/challenge/locked, derive_key(password), validate(key), list(), put(cred), and status-word → exception mapping. Pure protocol; unit-testable with a fake channel.
• MemorySession — SELECT the memory applet, parse availPersistent + baseline, get_batch(). Absent applet handled gracefully.
• Credential — normalized record: issuer, account, secret(bytes), oath_type, algorithm, digits, period, and cred_id (issuer:account, with period/ prefix only if period ≠ 30).
• load_credentials(xlsx_path) — read the OTP Keys sheet from curated columns.
• validate_input(creds) — pre-flight checks; returns a list of problems.
• pick_reader(...) — enumerate readers, detect cards, SELECT OATH, choose.
• main() — CLI + orchestration in the order below.

Companion test_oath.py — hardware-free unit tests.

Execution flow

1. Load + validate input first (before any reader access). load_credentials → validate_input. If any problems, print them all and exit non-zero. No interactive fixing — the user edits the spreadsheet.
2. Reader/card detection. For each reader with a card inserted, try the OATH SELECT. 0 responders → error "Place your Apex on a reader and re-run", exit. 1 → use it. >1 → list reader names and prompt for the target.
3. Session. SELECT OATH. If locked, getpass the password and VALIDATE (map 6A80 → "incorrect password", exit). LIST existing credential names.
4. Optional capacity pre-check. If the memory applet is present, read availPersistent and warn (no halt) if availPersistent < ~250 B × N.
5. Import. For each credential: if its cred_id is already on the applet, ask per conflict (overwrite / skip; with all/none shortcuts). Otherwise PUT. Map 6A84 → "applet full" and stop further PUTs (report remainder as skipped). --profile-memory wraps each PUT in before/after memory reads (re-VALIDATE between, since the memory SELECT deselects OATH) and appends a log row.
6. Report. Per-credential OK / skipped / failed summary, plus mem-log path when profiling.

Pre-flight validation rules (halt on any)

• Duplicate cred_id within the input (e.g. the two SAW:michelle@dangerousthings.com entries with different secrets).
• Secret missing or not valid Base32 after uppercasing and removing spaces.
• Decoded key length > 64 bytes.
• cred_id (UTF-8) length > 64 bytes.
• Algorithm SHA512 (unsupported by the applet), or a non-TOTP/HOTP type.

Each problem is reported with the offending issuer/account and source row.

Conflict handling (on-device)

Interactive per name already present on the applet: [o]verwrite / [s]kip / [a] overwrite-all / [x] skip-all. Non-interactive overrides: --overwrite-all, --skip-existing.

Memory profiling (--profile-memory, opt-in)

Per credential: memory SELECT → availBefore; OATH SELECT + re-VALIDATE (if locked) → PUT; memory SELECT → availAfter; consumed = availBefore - availAfter. Append to apex-otp-memlog.csv with columns: timestamp_iso, card_uid, mem_batch, key_name, avail_before, avail_after, consumed_bytes. timestamp_iso is the host clock; card_uid from FF CA 00 00 00; the key secret is never logged. Block-granular allocation means consumed_bytes is a close estimate, not byte-exact.

CLI

apex_otp_import.py [--input PATH] [--reader SUBSTR] [--dry-run] [--list]
                   [--overwrite-all | --skip-existing] [--profile-memory]
                   [--memlog PATH] [--password PASS]

• --dry-run — load, validate, build every APDU, hex-print them; no card I/O.
• --list — read-only: connect, authenticate if needed, print on-applet creds.
• --password — supply the applet password non-interactively (else getpass).

Error handling

Central status-word → message map: 6A84 applet full, 6982 auth required, 6A80 wrong data / incorrect password, 6984 no such object, 61xx handled by SEND REMAINING, 6D00/6E00 wrong INS/CLA (wrong applet or AID). Card exceptions (no card, reader gone) surface as clear, actionable messages.

Testing (test_oath.py, no hardware)

• TLV encode/parse round-trips incl. multi-byte lengths.
• cred_id formatting (period prefix rule).
• Byte-exact PUT payload for a known credential (type/algo, digits, padded key).
• Byte-exact VALIDATE payload and mutual-response verification against a known key/challenge.
• Base32 decode + 14-byte right-pad; HMAC-equivalence of padded vs unpadded.
• PBKDF2/HMAC vectors; RFC 6238 TOTP HMAC sanity check.
• validate_input catches each halt condition (dup, bad Base32, oversize name, oversize key, SHA512).
• 61xx reassembly via a fake channel that chunks a long LIST response.

Password / reset admin (added after initial spec)

The importer also exposes two admin operations on the same protocol layer:

• --set-password PASS: after a fully successful import, set an access password on the applet only if it has none (SET CODE with a SHA1 access key derived as above). Runs post-import for efficiency (single session). Does not change or remove an existing password.
• --reset: standalone; sends RESET (P1P2 = 0xDEAD) to wipe all credentials and any password, then exits.

Note: the applet reports the full type|algo byte (e.g. 0x21) in the SELECT ALGORITHM tag; the low nibble is masked off to select the HMAC algorithm.

Out of scope (YAGNI)

Changing or removing an existing password; HOTP import; touch-required; RENAME/DELETE of individual credentials; CSV/text input; on-device code calculation (beyond ad-hoc post-import verification, not part of the tool).

Files

Code and docs live in the otp-import repo (C:\Users\amal\Documents\Repos\otp-import, remote git.dngr.us/VivoKey/otp-import): apex_otp_import.py, test_oath.py, docs/design.md, README.md, .gitignore.

The secret data stays out of the repo: the export spreadsheet (otp-keys-export.xlsx), the profiling CSV (apex-otp-memlog.csv, created only when profiling), and QR screenshots remain in the private Nextcloud folder and are excluded by .gitignore. The importer is pointed at the data with --input.

Risks / notes

• The instance AID on a Fidesmo-provisioned Apex may differ from build.xml; mitigated by the AID fallback list and validating the SELECT response shape.
• Contactless UID from FF CA may be random per tap on some configurations; it is logged as-is for correlation, not treated as a stable serial.
• Keep the card still: the whole import runs in one OATH session; a mid-import deselect drops auth and (without profiling's re-VALIDATE) would fail PUTs with 6982.