VivoKey/otp-import
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Expand README into full tool documentation

Browse Source
This commit is contained in:
Amal Graafstra
2026-07-08 15:44:31 -07:00
parent a8a405600c
commit 3560964188
1 changed files with 256 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files

323
README.md
View File

@@ -1,105 +1,294 @@
# otp-import
Turn saved OTP QR-code screenshots into credentials on a VivoKey Apex OTP
(ykneo-oath) applet, via a canonical CSV.
Move TOTP credentials from saved authenticator **QR-code screenshots** onto a
**VivoKey Apex** OTP applet (the ykneo-oath applet), by way of a plain,
human-editable **CSV**.
## Pipeline
Two small command-line tools:
1. **QR -> CSV** (`qr_to_sheet.py`): decode QR screenshots and append new
credentials to `otp-keys.csv`, deduplicated on the `otpauth://` data.
2. **Edit labels (optional)**: adjust the `issuer` / `account` columns to taste.
Never edit the `otpauth_uri` column - it is the immutable source of truth.
3. **CSV -> applet** (`apex_otp_import.py`): write to the Apex, validating for
anything you may have broken (duplicate names, invalid characters, overlength).
- `qr_to_sheet.py` — decode QR screenshots into the CSV (no hardware needed).
- `apex_otp_import.py` — write the CSV's credentials onto the applet over PC/SC.
The CSV has four columns: `issuer`, `account` (editable labels), `otpauth_uri`
(do not edit), and `source_file`. The secret and other crypto parameters live
inside `otpauth_uri` and are always taken from there.
---
## The pipeline
```
# Populate/refresh the CSV from a folder of QR screenshots
python qr_to_sheet.py --images /path/to/qr-screenshots --csv otp-keys.csv
python qr_to_sheet.py --images ... --csv ... --dry-run # preview, no write
QR screenshots otp-keys.csv Apex OTP applet
┌───────────────┐ ┌──────────────┐ ┌───────────────┐
│ *.png / *.jpg │──(1)──▶ │ issuer │──(3)───────▶ │ TOTP creds │
│ otpauth://... │ │ account │ PUT/auth │ (ykneo-oath) │
└───────────────┘ │ otpauth_uri │ └───────────────┘
│ source_file │
(2) edit labels here
└──────────────┘
```
Undecodable images (stylized/colored QRs, or text-only screenshots) are listed
by filename so you can add them by hand.
1. **QR → CSV** (`qr_to_sheet.py`): decode each screenshot and append any
**new** credential to the CSV. Deduplication is on the `otpauth://` data, so
re-running never adds a credential that is already present.
2. **Edit labels (optional)**: adjust the `issuer` and `account` columns to
taste. **Never edit the `otpauth_uri` column** — it is the immutable source
of truth (secret and parameters live inside it).
3. **CSV → applet** (`apex_otp_import.py`): validate the CSV and write each
credential onto the applet, authenticating first if the applet has a
password.
## Importing
The two stages guard different things: stage 1 stops you re-importing the same
QR; stage 3 stops you writing something you broke while editing labels.
Import TOTP credentials from the CSV (or a legacy `.xlsx`) into the applet.
---
The importer reads `otp-keys.csv` (or a legacy `.xlsx`), takes the credential
name from the `issuer`/`account` columns and the crypto from `otpauth_uri`, and
writes each credential using the standard YKOATH `PUT` command. It authenticates
first if the applet is password-protected, asks before overwriting credentials
that already exist, and can optionally profile per-credential on-chip memory
consumption via the `javacard-memory` applet.
## The CSV
The protocol layer is verified against the applet sources
(`com.vivokey.otp.YkneoOath` / `OathObj`) and `doc/Protocol.txt`. See
[docs/design.md](docs/design.md) for the full design and protocol notes.
Four columns:
## Security
| Column | Edit it? | Meaning |
| ------ | -------- | ------- |
| `issuer` | yes | Issuer shown on the applet (the "Issuer:" part of the name). |
| `account` | yes | Account shown on the applet (the part after "Issuer:"). |
| `otpauth_uri` | **no** | The raw `otpauth://` string from the QR. Dedup key and the source of the secret/algorithm/digits/period. |
| `source_file` | reference | The image filename the row was decoded from. |
This tool handles live TOTP secrets. **Keep your secrets out of this
repository.** The spreadsheet, any CSV, and QR screenshots are excluded by
`.gitignore`; keep the actual data in a private location (e.g. a synced
personal folder) and point the importer at it with `--input`.
The credential **name** written to the applet is `issuer:account` from the
editable columns. The **secret** and other crypto parameters are always taken
from `otpauth_uri`, so editing a label can never corrupt the code that gets
generated. The secret is still visible — it lives inside `otpauth_uri`.
The CSV is disposable: it can be regenerated from the QR screenshots at any time
with `qr_to_sheet.py`.
---
## Requirements
- Python 3.9+
- `pyscard`, `cryptography`, `openpyxl`
- A PC/SC reader with the Apex (or a JavaCard carrying the applet) presented
## Usage
- `pyscard`, `cryptography`, `openpyxl` (for reading legacy `.xlsx`), and
`opencv-python` (for QR decoding).
- For writing to a card: a PC/SC reader with the Apex (over NFC) or a JavaCard
carrying the applet presented to it.
```
# 1. Validate the data and preview the APDUs (no card needed)
python apex_otp_import.py --input /path/to/otp-keys.csv --dry-run
pip install pyscard cryptography openpyxl opencv-python
```
# 2. Read-only: list what is already on the applet
python apex_otp_import.py --input /path/to/otp-keys.csv --list
---
# 3. Import (asks before overwriting existing credentials)
python apex_otp_import.py --input /path/to/otp-keys.csv
## Tool 1 — `qr_to_sheet.py` (QR → CSV, no card)
# Non-interactive conflict handling
python apex_otp_import.py --input ... --overwrite-all
python apex_otp_import.py --input ... --skip-existing
Decode a folder of QR screenshots and append new credentials to the CSV. This
tool never touches a reader, so run it any time — the Apex does not need to be
present.
# Target a specific reader, supply a password, and profile memory
python apex_otp_import.py --input ... --reader VivoKey --password '...' --profile-memory
```
python qr_to_sheet.py --images <folder> --csv <path/to/otp-keys.csv>
```
# One-shot pipeline: decode new QRs into the CSV (only new, deduped on
# otpauth://), then validate and import. QR folder defaults to the CSV's folder.
Options:
| Option | Default | Meaning |
| ------ | ------- | ------- |
| `--images DIR` | `.` | Folder of QR images (`*.png`, `*.jpg`, `*.jpeg`). |
| `--csv PATH` | `otp-keys.csv` | The CSV to create or append to. |
| `--dry-run` | off | Report what would be added; write nothing. |
What it does per run:
- Decodes each image (a multi-pass decode copes with scaling/thresholding).
- Parses the `otpauth://totp/...` URI; seeds `issuer`/`account` from it.
- Skips any credential whose `otpauth://` data is already in the CSV
(deduplicated, existing rows never touched), and any duplicate within the same
run.
- Appends the genuinely-new rows and prints a summary of added / skipped /
undecodable.
Example:
```
$ python qr_to_sheet.py --images ./screenshots --csv otp-keys.csv
Scanned 34 image(s): 32 otpauth QR(s) decoded.
3 new, 29 duplicate(s) skipped.
could not be decoded (add manually): coinbase-...png, plaid-...png
Appended 3 row(s) to otp-keys.csv.
```
**Undecodable images** — stylized/colored QR codes, or screenshots that only
show the secret as text — are listed by filename. Add those by hand: create a
row with `issuer`, `account`, `source_file`, and an `otpauth_uri` you build from
the printed secret, e.g.
`otpauth://totp/Coinbase:you@example.com?secret=THESECRET&issuer=Coinbase`.
`otpauth-migration://` payloads (Google Authenticator's batch-export QR) are
reported as unsupported, not decoded.
---
## Tool 2 — `apex_otp_import.py` (CSV → applet)
Validate the CSV and write its credentials onto the applet. Needs the Apex on a
reader (except `--dry-run`).
```
python apex_otp_import.py --input otp-keys.csv [options]
```
Options:
| Option | Meaning |
| ------ | ------- |
| `--input PATH` | CSV (or legacy `.xlsx`) to import. Default `otp-keys.csv`. |
| `--reader SUBSTR` | Pick a reader by name substring (e.g. `VivoKey`, `3700 F CL`). Omit to auto-detect the single card holding an OATH applet; you are prompted if more than one is present. |
| `--dry-run` | Validate and print the APDUs that would be sent; no card I/O. |
| `--list` | Read-only: list the credentials already on the applet. |
| `--overwrite-all` | Overwrite any credential that already exists (non-interactive). |
| `--skip-existing` | Skip any credential that already exists (non-interactive). |
| `--password PASS` | Applet password (otherwise prompted if the applet is locked). |
| `--profile-memory` | Record per-credential EEPROM consumption to a CSV log (needs the `javacard-memory` applet). |
| `--set-password PASS` | After a successful import, set an access password — only if the applet has none. |
| `--reset` | Standalone: wipe all credentials and any password, then exit. |
| `--import-qr` | Run the QR → CSV step first (see below), then import. |
| `--qr-images DIR` | QR folder for `--import-qr` (default: the folder of `--input`). |
| `--memlog PATH` | Path for the `--profile-memory` log (default `apex-otp-memlog.csv`). |
By default, when a credential name already exists on the applet you are asked
whether to overwrite or skip it (with all/none shortcuts). Use `--overwrite-all`
or `--skip-existing` for unattended runs.
Recommended sequence for a first import:
```
python apex_otp_import.py --input otp-keys.csv --dry-run # (1) preview, no card
python apex_otp_import.py --input otp-keys.csv --list # (2) see what's on the applet
python apex_otp_import.py --input otp-keys.csv # (3) import for real
python apex_otp_import.py --input otp-keys.csv --list # (4) confirm
```
### Passwords
Authenticating to a password-protected applet uses the standard YKOATH scheme:
PBKDF2-HMAC-SHA1 of the password with the applet's device id as salt, then a
mutual challenge-response. `--set-password` sets a password only on an applet
that has none; it does not change or remove an existing one.
Note: unlike a YubiKey, this applet requires authentication before `--reset`
when a password is set. If you set a password and forget it, the OATH applet
cannot be reset over the OATH protocol — it must be deleted/reinstalled with
GlobalPlatform (card-manager keys).
### Memory profiling
`--profile-memory` measures how much persistent memory each credential consumes
by reading the `javacard-memory` applet before and after each `PUT`. Each row of
the log records the timestamp, card UID, memory-applet batch id, credential
name (never the secret), and bytes consumed. It is slower (a memory read per
credential) and off by default.
---
## The three modes
| Command | Does | Card needed |
| ------- | ---- | ----------- |
| `qr_to_sheet.py --images D --csv C` | QR → CSV | no |
| `apex_otp_import.py --input C` | CSV → applet | yes |
| `apex_otp_import.py --input C --import-qr` | QR → CSV → applet (one shot) | yes |
`--import-qr` is the whole pipeline in one command: it decodes new QRs into the
CSV (deduped on `otpauth://`, existing entries silently skipped), then validates
and imports. The QR folder defaults to the CSV's folder; override with
`--qr-images`. If you only want to refresh the CSV, use `qr_to_sheet.py` — the
Apex does not need to be on a reader.
```
# whole pipeline, single command
python apex_otp_import.py --input otp-keys.csv --import-qr --reader VivoKey
python apex_otp_import.py --input otp-keys.csv --import-qr --qr-images /qr/folder --overwrite-all
# Import, then set an access password (only if the applet has none yet)
python apex_otp_import.py --input ... --set-password 'yourpassword'
# Reset: wipe all credentials and any password (standalone; no input needed)
python apex_otp_import.py --reset
# same, with an explicit QR folder and unattended overwrite
python apex_otp_import.py --input otp-keys.csv --import-qr --qr-images ./screenshots --overwrite-all
```
Authenticating to a password-protected applet uses the standard YKOATH scheme
(PBKDF2-HMAC-SHA1 of the password with the device id as salt, then mutual
challenge-response). `--set-password` only sets a password on an applet that has
none; it does not change or remove an existing one. `--reset` clears everything.
---
The input is validated before any reader is touched. The run halts with a
per-row report if it finds: duplicate credential names; invalid Base32;
over-long names or keys; an unsupported algorithm; control characters or a
`:`/`/` inside an issuer or account (they break the OATH name convention); or a
digit count outside 6-8 / a non-positive period. Fix the source data and re-run.
## Common workflows
**Add newly-enrolled services.** Save the new QR screenshots into your folder,
then:
```
python qr_to_sheet.py --images ./screenshots --csv otp-keys.csv # appends only the new ones
# (edit issuer/account labels if you like)
python apex_otp_import.py --input otp-keys.csv --reader VivoKey --overwrite-all
```
**Re-provision a fresh/wiped Apex from scratch.**
```
python apex_otp_import.py --input otp-keys.csv --reader VivoKey
```
**Password-protect after loading.**
```
python apex_otp_import.py --input otp-keys.csv --reader VivoKey --set-password 'mypass'
```
**Wipe an applet (know the password if it has one).**
```
python apex_otp_import.py --reset --reader VivoKey [--password 'mypass']
```
---
## Validation (stage 3)
The CSV is validated before any reader is touched; the run halts with a per-row
report if it finds any of:
- Duplicate credential name (`issuer:account`).
- A secret that is not valid Base32.
- A decoded key longer than 64 bytes, or a composed name longer than 64 bytes.
- An unsupported algorithm (the applet accepts SHA1/SHA256, not SHA512).
- Control characters, or a literal `:` or `/`, inside an issuer or account
(they collide with the OATH `issuer:account` separator and `period/` prefix).
- A digit count outside 6–8, or a non-positive period.
Fix the offending cells and re-run. The applet itself only enforces the length
limits; the character and digit/period rules are host-side guards for
OATH-convention and authenticator-app compatibility.
---
## Security
This tooling handles live TOTP secrets. **Keep your secrets out of this
repository.** `otp-keys.csv`, any `.xlsx`, the memory log, and QR screenshots
are excluded by `.gitignore`; keep the real data in a private location (e.g. a
synced personal folder) and point the tools at it with `--csv` / `--input`.
---
## Tests
Hardware-free unit tests cover the protocol encoding, password derivation,
input validation, and response reassembly:
Hardware-free unit tests cover the OATH protocol encoding, password derivation,
input validation, response reassembly, otpauth parsing, dedup, and CSV I/O:
```
python test_oath.py
python test_qr.py
```
---
## How it works
The OATH protocol layer is implemented directly over PC/SC (`pyscard`) with no
Yubico dependency, and is verified against the applet sources
(`com.vivokey.otp.YkneoOath` / `OathObj`) and `doc/Protocol.txt`. See
[docs/design.md](docs/design.md) for the importer/protocol design and
[docs/qr-to-csv-design.md](docs/qr-to-csv-design.md) for the QR-to-CSV design.
- `otpauth.py` — parse/canonicalize `otpauth://` URIs (shared by both tools).
- `qr_to_sheet.py` — QR decode, dedup, CSV append.
- `apex_otp_import.py` — CSV/xlsx load, OATH SELECT/VALIDATE/LIST/PUT, password
auth, reset, memory profiling, and the CLI.