diff --git a/README.md b/README.md index 3e8ab17..477f84e 100644 --- a/README.md +++ b/README.md @@ -1,105 +1,294 @@ # otp-import -Turn saved OTP QR-code screenshots into credentials on a VivoKey Apex OTP -(ykneo-oath) applet, via a canonical CSV. +Move TOTP credentials from saved authenticator **QR-code screenshots** onto a +**VivoKey Apex** OTP applet (the ykneo-oath applet), by way of a plain, +human-editable **CSV**. -## Pipeline +Two small command-line tools: -1. **QR -> CSV** (`qr_to_sheet.py`): decode QR screenshots and append new - credentials to `otp-keys.csv`, deduplicated on the `otpauth://` data. -2. **Edit labels (optional)**: adjust the `issuer` / `account` columns to taste. - Never edit the `otpauth_uri` column - it is the immutable source of truth. -3. **CSV -> applet** (`apex_otp_import.py`): write to the Apex, validating for - anything you may have broken (duplicate names, invalid characters, overlength). +- `qr_to_sheet.py` — decode QR screenshots into the CSV (no hardware needed). +- `apex_otp_import.py` — write the CSV's credentials onto the applet over PC/SC. -The CSV has four columns: `issuer`, `account` (editable labels), `otpauth_uri` -(do not edit), and `source_file`. The secret and other crypto parameters live -inside `otpauth_uri` and are always taken from there. +--- + +## The pipeline ``` -# Populate/refresh the CSV from a folder of QR screenshots -python qr_to_sheet.py --images /path/to/qr-screenshots --csv otp-keys.csv -python qr_to_sheet.py --images ... --csv ... --dry-run # preview, no write + QR screenshots otp-keys.csv Apex OTP applet + ┌───────────────┐ ┌──────────────┐ ┌───────────────┐ + │ *.png / *.jpg │──(1)──▶ │ issuer │──(3)───────▶ │ TOTP creds │ + │ otpauth://... │ │ account │ PUT/auth │ (ykneo-oath) │ + └───────────────┘ │ otpauth_uri │ └───────────────┘ + │ source_file │ + (2) edit labels here + └──────────────┘ ``` -Undecodable images (stylized/colored QRs, or text-only screenshots) are listed -by filename so you can add them by hand. +1. **QR → CSV** (`qr_to_sheet.py`): decode each screenshot and append any + **new** credential to the CSV. Deduplication is on the `otpauth://` data, so + re-running never adds a credential that is already present. +2. **Edit labels (optional)**: adjust the `issuer` and `account` columns to + taste. **Never edit the `otpauth_uri` column** — it is the immutable source + of truth (secret and parameters live inside it). +3. **CSV → applet** (`apex_otp_import.py`): validate the CSV and write each + credential onto the applet, authenticating first if the applet has a + password. -## Importing +The two stages guard different things: stage 1 stops you re-importing the same +QR; stage 3 stops you writing something you broke while editing labels. -Import TOTP credentials from the CSV (or a legacy `.xlsx`) into the applet. +--- -The importer reads `otp-keys.csv` (or a legacy `.xlsx`), takes the credential -name from the `issuer`/`account` columns and the crypto from `otpauth_uri`, and -writes each credential using the standard YKOATH `PUT` command. It authenticates -first if the applet is password-protected, asks before overwriting credentials -that already exist, and can optionally profile per-credential on-chip memory -consumption via the `javacard-memory` applet. +## The CSV -The protocol layer is verified against the applet sources -(`com.vivokey.otp.YkneoOath` / `OathObj`) and `doc/Protocol.txt`. See -[docs/design.md](docs/design.md) for the full design and protocol notes. +Four columns: -## Security +| Column | Edit it? | Meaning | +| ------ | -------- | ------- | +| `issuer` | yes | Issuer shown on the applet (the "Issuer:" part of the name). | +| `account` | yes | Account shown on the applet (the part after "Issuer:"). | +| `otpauth_uri` | **no** | The raw `otpauth://` string from the QR. Dedup key and the source of the secret/algorithm/digits/period. | +| `source_file` | reference | The image filename the row was decoded from. | -This tool handles live TOTP secrets. **Keep your secrets out of this -repository.** The spreadsheet, any CSV, and QR screenshots are excluded by -`.gitignore`; keep the actual data in a private location (e.g. a synced -personal folder) and point the importer at it with `--input`. +The credential **name** written to the applet is `issuer:account` from the +editable columns. The **secret** and other crypto parameters are always taken +from `otpauth_uri`, so editing a label can never corrupt the code that gets +generated. The secret is still visible — it lives inside `otpauth_uri`. + +The CSV is disposable: it can be regenerated from the QR screenshots at any time +with `qr_to_sheet.py`. + +--- ## Requirements - Python 3.9+ -- `pyscard`, `cryptography`, `openpyxl` -- A PC/SC reader with the Apex (or a JavaCard carrying the applet) presented - -## Usage +- `pyscard`, `cryptography`, `openpyxl` (for reading legacy `.xlsx`), and + `opencv-python` (for QR decoding). +- For writing to a card: a PC/SC reader with the Apex (over NFC) or a JavaCard + carrying the applet presented to it. ``` -# 1. Validate the data and preview the APDUs (no card needed) -python apex_otp_import.py --input /path/to/otp-keys.csv --dry-run +pip install pyscard cryptography openpyxl opencv-python +``` -# 2. Read-only: list what is already on the applet -python apex_otp_import.py --input /path/to/otp-keys.csv --list +--- -# 3. Import (asks before overwriting existing credentials) -python apex_otp_import.py --input /path/to/otp-keys.csv +## Tool 1 — `qr_to_sheet.py` (QR → CSV, no card) -# Non-interactive conflict handling -python apex_otp_import.py --input ... --overwrite-all -python apex_otp_import.py --input ... --skip-existing +Decode a folder of QR screenshots and append new credentials to the CSV. This +tool never touches a reader, so run it any time — the Apex does not need to be +present. -# Target a specific reader, supply a password, and profile memory -python apex_otp_import.py --input ... --reader VivoKey --password '...' --profile-memory +``` +python qr_to_sheet.py --images --csv +``` -# One-shot pipeline: decode new QRs into the CSV (only new, deduped on -# otpauth://), then validate and import. QR folder defaults to the CSV's folder. +Options: + +| Option | Default | Meaning | +| ------ | ------- | ------- | +| `--images DIR` | `.` | Folder of QR images (`*.png`, `*.jpg`, `*.jpeg`). | +| `--csv PATH` | `otp-keys.csv` | The CSV to create or append to. | +| `--dry-run` | off | Report what would be added; write nothing. | + +What it does per run: + +- Decodes each image (a multi-pass decode copes with scaling/thresholding). +- Parses the `otpauth://totp/...` URI; seeds `issuer`/`account` from it. +- Skips any credential whose `otpauth://` data is already in the CSV + (deduplicated, existing rows never touched), and any duplicate within the same + run. +- Appends the genuinely-new rows and prints a summary of added / skipped / + undecodable. + +Example: + +``` +$ python qr_to_sheet.py --images ./screenshots --csv otp-keys.csv +Scanned 34 image(s): 32 otpauth QR(s) decoded. + 3 new, 29 duplicate(s) skipped. + could not be decoded (add manually): coinbase-...png, plaid-...png +Appended 3 row(s) to otp-keys.csv. +``` + +**Undecodable images** — stylized/colored QR codes, or screenshots that only +show the secret as text — are listed by filename. Add those by hand: create a +row with `issuer`, `account`, `source_file`, and an `otpauth_uri` you build from +the printed secret, e.g. +`otpauth://totp/Coinbase:you@example.com?secret=THESECRET&issuer=Coinbase`. + +`otpauth-migration://` payloads (Google Authenticator's batch-export QR) are +reported as unsupported, not decoded. + +--- + +## Tool 2 — `apex_otp_import.py` (CSV → applet) + +Validate the CSV and write its credentials onto the applet. Needs the Apex on a +reader (except `--dry-run`). + +``` +python apex_otp_import.py --input otp-keys.csv [options] +``` + +Options: + +| Option | Meaning | +| ------ | ------- | +| `--input PATH` | CSV (or legacy `.xlsx`) to import. Default `otp-keys.csv`. | +| `--reader SUBSTR` | Pick a reader by name substring (e.g. `VivoKey`, `3700 F CL`). Omit to auto-detect the single card holding an OATH applet; you are prompted if more than one is present. | +| `--dry-run` | Validate and print the APDUs that would be sent; no card I/O. | +| `--list` | Read-only: list the credentials already on the applet. | +| `--overwrite-all` | Overwrite any credential that already exists (non-interactive). | +| `--skip-existing` | Skip any credential that already exists (non-interactive). | +| `--password PASS` | Applet password (otherwise prompted if the applet is locked). | +| `--profile-memory` | Record per-credential EEPROM consumption to a CSV log (needs the `javacard-memory` applet). | +| `--set-password PASS` | After a successful import, set an access password — only if the applet has none. | +| `--reset` | Standalone: wipe all credentials and any password, then exit. | +| `--import-qr` | Run the QR → CSV step first (see below), then import. | +| `--qr-images DIR` | QR folder for `--import-qr` (default: the folder of `--input`). | +| `--memlog PATH` | Path for the `--profile-memory` log (default `apex-otp-memlog.csv`). | + +By default, when a credential name already exists on the applet you are asked +whether to overwrite or skip it (with all/none shortcuts). Use `--overwrite-all` +or `--skip-existing` for unattended runs. + +Recommended sequence for a first import: + +``` +python apex_otp_import.py --input otp-keys.csv --dry-run # (1) preview, no card +python apex_otp_import.py --input otp-keys.csv --list # (2) see what's on the applet +python apex_otp_import.py --input otp-keys.csv # (3) import for real +python apex_otp_import.py --input otp-keys.csv --list # (4) confirm +``` + +### Passwords + +Authenticating to a password-protected applet uses the standard YKOATH scheme: +PBKDF2-HMAC-SHA1 of the password with the applet's device id as salt, then a +mutual challenge-response. `--set-password` sets a password only on an applet +that has none; it does not change or remove an existing one. + +Note: unlike a YubiKey, this applet requires authentication before `--reset` +when a password is set. If you set a password and forget it, the OATH applet +cannot be reset over the OATH protocol — it must be deleted/reinstalled with +GlobalPlatform (card-manager keys). + +### Memory profiling + +`--profile-memory` measures how much persistent memory each credential consumes +by reading the `javacard-memory` applet before and after each `PUT`. Each row of +the log records the timestamp, card UID, memory-applet batch id, credential +name (never the secret), and bytes consumed. It is slower (a memory read per +credential) and off by default. + +--- + +## The three modes + +| Command | Does | Card needed | +| ------- | ---- | ----------- | +| `qr_to_sheet.py --images D --csv C` | QR → CSV | no | +| `apex_otp_import.py --input C` | CSV → applet | yes | +| `apex_otp_import.py --input C --import-qr` | QR → CSV → applet (one shot) | yes | + +`--import-qr` is the whole pipeline in one command: it decodes new QRs into the +CSV (deduped on `otpauth://`, existing entries silently skipped), then validates +and imports. The QR folder defaults to the CSV's folder; override with +`--qr-images`. If you only want to refresh the CSV, use `qr_to_sheet.py` — the +Apex does not need to be on a reader. + +``` +# whole pipeline, single command python apex_otp_import.py --input otp-keys.csv --import-qr --reader VivoKey -python apex_otp_import.py --input otp-keys.csv --import-qr --qr-images /qr/folder --overwrite-all -# Import, then set an access password (only if the applet has none yet) -python apex_otp_import.py --input ... --set-password 'yourpassword' - -# Reset: wipe all credentials and any password (standalone; no input needed) -python apex_otp_import.py --reset +# same, with an explicit QR folder and unattended overwrite +python apex_otp_import.py --input otp-keys.csv --import-qr --qr-images ./screenshots --overwrite-all ``` -Authenticating to a password-protected applet uses the standard YKOATH scheme -(PBKDF2-HMAC-SHA1 of the password with the device id as salt, then mutual -challenge-response). `--set-password` only sets a password on an applet that has -none; it does not change or remove an existing one. `--reset` clears everything. +--- -The input is validated before any reader is touched. The run halts with a -per-row report if it finds: duplicate credential names; invalid Base32; -over-long names or keys; an unsupported algorithm; control characters or a -`:`/`/` inside an issuer or account (they break the OATH name convention); or a -digit count outside 6-8 / a non-positive period. Fix the source data and re-run. +## Common workflows + +**Add newly-enrolled services.** Save the new QR screenshots into your folder, +then: + +``` +python qr_to_sheet.py --images ./screenshots --csv otp-keys.csv # appends only the new ones +# (edit issuer/account labels if you like) +python apex_otp_import.py --input otp-keys.csv --reader VivoKey --overwrite-all +``` + +**Re-provision a fresh/wiped Apex from scratch.** + +``` +python apex_otp_import.py --input otp-keys.csv --reader VivoKey +``` + +**Password-protect after loading.** + +``` +python apex_otp_import.py --input otp-keys.csv --reader VivoKey --set-password 'mypass' +``` + +**Wipe an applet (know the password if it has one).** + +``` +python apex_otp_import.py --reset --reader VivoKey [--password 'mypass'] +``` + +--- + +## Validation (stage 3) + +The CSV is validated before any reader is touched; the run halts with a per-row +report if it finds any of: + +- Duplicate credential name (`issuer:account`). +- A secret that is not valid Base32. +- A decoded key longer than 64 bytes, or a composed name longer than 64 bytes. +- An unsupported algorithm (the applet accepts SHA1/SHA256, not SHA512). +- Control characters, or a literal `:` or `/`, inside an issuer or account + (they collide with the OATH `issuer:account` separator and `period/` prefix). +- A digit count outside 6–8, or a non-positive period. + +Fix the offending cells and re-run. The applet itself only enforces the length +limits; the character and digit/period rules are host-side guards for +OATH-convention and authenticator-app compatibility. + +--- + +## Security + +This tooling handles live TOTP secrets. **Keep your secrets out of this +repository.** `otp-keys.csv`, any `.xlsx`, the memory log, and QR screenshots +are excluded by `.gitignore`; keep the real data in a private location (e.g. a +synced personal folder) and point the tools at it with `--csv` / `--input`. + +--- ## Tests -Hardware-free unit tests cover the protocol encoding, password derivation, -input validation, and response reassembly: +Hardware-free unit tests cover the OATH protocol encoding, password derivation, +input validation, response reassembly, otpauth parsing, dedup, and CSV I/O: ``` python test_oath.py +python test_qr.py ``` + +--- + +## How it works + +The OATH protocol layer is implemented directly over PC/SC (`pyscard`) with no +Yubico dependency, and is verified against the applet sources +(`com.vivokey.otp.YkneoOath` / `OathObj`) and `doc/Protocol.txt`. See +[docs/design.md](docs/design.md) for the importer/protocol design and +[docs/qr-to-csv-design.md](docs/qr-to-csv-design.md) for the QR-to-CSV design. + +- `otpauth.py` — parse/canonicalize `otpauth://` URIs (shared by both tools). +- `qr_to_sheet.py` — QR decode, dedup, CSV append. +- `apex_otp_import.py` — CSV/xlsx load, OATH SELECT/VALIDATE/LIST/PUT, password + auth, reset, memory profiling, and the CLI.