otp-import/docs/qr-to-csv-design.md

QR-to-CSV Utility — Design

Date: 2026-07-08 Status: Draft for review

Goal and workflow

A companion utility that turns saved OTP QR-code screenshots into rows of the canonical CSV that the importer consumes. The overall flow is a one-way pipeline, not a hand-maintained file:

1. QR -> CSV (qr_to_sheet.py): decode QR images, deduplicate on the otpauth:// data, and append only genuinely-new entries to the CSV.
2. Optional user edits: the user may adjust the label columns (issuer, account). They are instructed never to touch the otpauth:// data.
3. CSV -> applet (apex_otp_import.py): push to the Apex, validating for human-introduced problems (duplicate names, invalid characters, overlength).

The two stages guard different risks: stage 1 prevents re-importing the same QR; stage 3 catches bad edits. The otpauth:// string is the immutable source of truth throughout.

Canonical format: CSV

Replaces the .xlsx as the canonical format (open, plain-text, git-diffable, trivial to append and dedup). Minimal schema:

Column	Editable?	Purpose
issuer	yes (label)	Issuer shown on the applet; seeded from the otpauth.
account	yes (label)	Account shown on the applet; seeded from the otpauth label.
otpauth_uri	no	The raw decoded otpauth:// string. Dedup key (stage 1) and crypto source (stage 3).
source_file	informational	Filename the QR was decoded from.

The secret remains visible (inside otpauth_uri), so nothing is hidden. The credential name written to the applet is issuer:account from the editable columns; the secret/type/algorithm/digits/period come from parsing otpauth_uri.

A one-time conversion turns the existing otp-keys-export.xlsx "OTP Keys" sheet into otp-keys.csv (issuer, account, otpauth_uri, source_file), so no data is lost and the tools operate on the CSV going forward.

Shared module: otpauth.py

Used by both tools:

• parse_otpauth(uri) -> OtpAuth: parse otpauth://TYPE/LABEL?params into type (totp/hotp), issuer (from the issuer param, else the label prefix before :), account (label after the issuer: prefix, else the whole label), secret (uppercased, spaces removed), algorithm (default SHA1), digits (default 6), period (default 30). Percent-decodes label/issuer.
• canonical_key(otp) -> tuple: a normalized dedup key (type, issuer, account, secret, algorithm, digits, period) with defaults filled and the secret uppercased. Two scans of the same credential — even with the issuer given via the label vs. the issuer param — produce the same key; a different secret produces a different key.
• build_otpauth(...) -> str: reconstruct a URI (used by the xlsx conversion for any legacy row lacking a stored URI).

qr_to_sheet.py

CLI: qr_to_sheet.py [--images DIR] [--csv PATH] [--dry-run]

1. Enumerate QR images in --images (default: the data folder) — *.png, *.jpg.
2. Decode each with OpenCV using the robust multi-pass strategy (direct -> grayscale -> upscale 2x/3x/4x/0.5x -> Otsu/adaptive threshold).
3. parse_otpauth each decoded otpauth://totp/.... An otpauth-migration:// payload is reported as unsupported (out of scope), not parsed.
4. Load the existing CSV (if any); build the set of canonical_key values from every row's otpauth_uri.
5. For each decoded QR: if its canonical_key is already present (in the CSV or already added this run), skip it as a duplicate; otherwise append a row (issuer, account from the parse; otpauth_uri = the raw decoded string; source_file = image filename).
6. Never rewrite or reorder existing rows — append only.
7. Print a summary: added, skipped-as-duplicate, and undecodable images (stylized/colored QRs or text-only screenshots) listed by filename for manual entry. --dry-run reports without writing.

Importer changes (apex_otp_import.py)

• load_credentials accepts CSV or XLSX, chosen by file extension.
• Crypto (secret/type/algorithm/digits/period) is derived by parse_otpauth(otpauth_uri) when an otpauth_uri is present; issuer and account come from their columns. When no otpauth_uri is present (legacy xlsx), it falls back to the individual crypto columns as today.
• validate_input is unchanged in intent and still catches the stage-3 risks (duplicate issuer:account, control/:// characters, overlength names, bad digits/period, invalid Base32, unsupported algorithm).

Testing (hardware-free, TDD)

• parse_otpauth: issuer from param vs. label prefix; percent-decoding; defaults for algorithm/digits/period; account extraction.
• canonical_key: equal for label-issuer vs. param-issuer forms of the same credential; differs when the secret differs; secret case-insensitive.
• Dedup: given existing CSV rows and a batch of candidates, only the new canonical keys are appended; intra-run duplicates collapse.
• CSV read/append round-trip; append preserves existing rows verbatim.
• load_credentials from CSV: crypto comes from otpauth_uri, labels from the editable columns; legacy fallback path still works.

QR image decoding is the untestable I/O edge, validated against the real PNGs.

Out of scope (YAGNI)

otpauth-migration:// (Google Authenticator batch export protobuf) — reported as unsupported; OCR of text-only screenshots; editing or removing existing rows; backup/recovery .txt codes.

Files (in the otp-import repo)

New: qr_to_sheet.py, otpauth.py, test_qr.py (parse/canonical/dedup/CSV tests), docs/qr-to-csv-design.md. Changed: apex_otp_import.py (load_credentials CSV + otpauth-authoritative), test_oath.py (CSV-load cases), README.md. One-time output: otp-keys.csv (git-ignored, lives with the private data, not in the repo).