diff --git a/CLAUDE.md b/CLAUDE.md index 5a64c58..ba18b5a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -55,7 +55,17 @@ pm3py/ access_control/ # Wiegand, OSDP transponders/ # tag/transponder models (extracted from sim/) hf/iso14443a/ # Tag14443A_3/4, MifareClassic, DESFire, NfcType2/4 + # nxp/type2,ntag21x,ultralight,ntag_i2c + # (NTAG210-216, Ultralight/C/EV1, NTAG I2C plus) + # st/st25tn (Type 2), st/st25ta (Type 4) + # infineon/optiga_nbt (OPTIGA Authenticate NBT, Type 4) + hf/iso14443b/ # base.py: Tag14443B (SRIX slot-marker) + + # Tag14443B_4 (standard REQB/ATTRIB + ISO-DEP). + # Vendor models (organised by mfg, like the rest): + # st/st25tb, ti/rf430cl330h (NFC Type 4B) hf/iso15693/ # Tag15693, NfcType5, NXP ICODE/SLIX2/DNA/NTAG5 + # st/st25tv, st/st25dv, infineon/myd_vicinity + # ti/tagit (Tag-it HF-I), ti/rf430frl (sensor) lf/ # EM4100, HID, T5577 reader/ # higher-level reader modes by protocol/vendor (scaffold) ``` diff --git a/docs/NTAG5_SECURITY.md b/docs/NTAG5_SECURITY.md new file mode 100644 index 0000000..a8fcf2e --- /dev/null +++ b/docs/NTAG5_SECURITY.md @@ -0,0 +1,780 @@ +# NTAG5 Link (NTP5332) Security Configuration Reference + +This document captures key technical details for implementing ISO 29167-10 AES-128 authentication on NTAG5 Link chips. + +## Configuration Memory Map + +### Device Security Configuration (DEV_SEC_CONFIG) + +**Address:** Block 0x3F (NFC) / 0x103F (I2C) + +| Byte | Name | Description | +| ---- | --------------- | --------------------------- | +| 0 | DEV_SEC_CONFIG | Security mode and lock bits | +| 1 | SRAM_CONF_PROT | SRAM and config protection | +| 2 | PP_AREA_1 (LSB) | Protection pointer area 1 | +| 3 | PP_AREA_1 (MSB) | Protection pointer area 1 | + +**DEV_SEC_CONFIG Bit Definition:** + +| Bits | Name | Value | Description | +| ---- | ------------- | ----- | ------------------------ | +| 7-5 | Security Lock | 010b | Locked (cannot modify) | +| | | 101b | Writable (default) | +| 4 | RFU | 00b | Reserved | +| 3-0 | NFC_SEC_MODE | 0010b | AES mode (NTP5332 only) | +| | | 0101b | Plain password (default) | + +**Example Values:** +- `0xA5` = Writable + Plain password mode (factory default) +- `0xA2` = Writable + AES mode + +### NFC Global Crypto Header (NFC_GCH) + +**Address:** Block 0x0C (NFC) / 0x100C (I2C), **Byte 1** (other bytes are RFU) + +**What it controls:** NFC_GCH defines the status/access enforcement for passwords (in password mode), keys, protection pointer & conditions, key headers, key privileges, crypto configuration header, and EAS/AFI protection. + +**IMPORTANT:** NFC_GCH is **one-way programmable** (lower → higher values only) and **irreversible**. + +**Allowed Values (AES Mode):** All other values are invalid. + +| Value | Status | Description | +| ----- | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x81 | Deactivated (default) | Protection pointer/conditions not activated; user memory R/W possible without prior mutual auth; PP/PPC can be modified via PROTECT PAGE; keys & privileges are readable/writable via READ/WRITE CONFIG according to each key's KH status; EAS/AFI protection not activated | +| 0x87 | Deactivated & privileges locked | Same as 0x81, but Key Privileges are locked (cannot be modified). EAS/AFI still not protected | +| 0xC1 | Access right activated | Protection pointer/conditions enabled; user-memory R/W protection enforced per configuration; keys/privileges still readable/writable according to KH status; EAS/AFI protection enabled | +| 0xC7 | Access right activated & privileges locked | Same as 0xC1, but Key Privileges are locked | +| 0xE7 | Activated (final) | Protection enabled; all Key Headers/Privileges/Keys are locked (cannot be modified); EAS/AFI protection enabled | + +**Allowed Values (Plain Password Mode):** + +| Value | Status | Description | +| ----- | ------------------ | ------------------------------------------------------------------------------------------------------------------------- | +| 0x81 | Writable (default) | Passwords readable/writable via READ/WRITE CONFIG | +| 0xE7 | Locked | Passwords not readable/writable via READ/WRITE CONFIG (note: separate LOCK PASSWORD needed to lock passwords permanently) | + +### NFC Crypto Configuration Header (NFC_CCH) + +**Address:** Block 0x0D (NFC) / 0x100D (I2C), **Byte 1** (other bytes are RFU) + +**What it controls:** NFC_CCH controls whether the NFC Authentication Limit can be changed freely or only after authentication. This is the "global knob" for Authentication Limit behavior (not general access-right enforcement). + +**Allowed Values:** All other values are invalid. + +| Value | Status | Description | +| ----- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------ | +| 0x81 | Unlocked (default) | Authentication limit can be modified freely | +| 0xE7 | Locked | Authentication limit can only be modified after mutual auth with a key that has the Crypto Config privilege (bit 5 of NFC_KPx) | + +### NFC Key Headers (NFC_KHx) + +**Addresses:** Key headers live in blocks 0x10/0x12/0x14/0x16, specifically in **Byte 1** of each block (other bytes are RFU). + +| Key | Block (NFC) | Block (I2C) | Byte Position | +| ------- | ----------- | ----------- | ------------- | +| NFC_KH0 | 0x10 | 0x1010 | Byte 1 | +| NFC_KH1 | 0x12 | 0x1012 | Byte 1 | +| NFC_KH2 | 0x14 | 0x1014 | Byte 1 | +| NFC_KH3 | 0x16 | 0x1016 | Byte 1 | + +**Key Header Values:** + +| Value | Status | Description | +| ----- | -------------------- | ------------------------------------------------------------- | +| 0x81 | Not active (default) | Key can be read/written but CANNOT be used for authentication | +| 0xE7 | Active and locked | Key is active and locked - can be used for authentication | +| 0xFF | Disabled | Key slot is disabled (NXP recommends disabling unused keys) | + +**IMPORTANT:** +- Keys with header 0x81 cannot be used for authentication +- The header must be set to 0xE7 to activate the key +- Once 0xE7 is set, the key value cannot be modified +- Key headers are **one-way** (lower→higher values only) and **irreversible** + +### NFC Key Privileges (NFC_KPx) + +**Addresses:** Key privileges live in blocks 0x11/0x13/0x15/0x17, specifically in **Byte 0** of each block (other bytes are RFU). + +| Key | Block (NFC) | Block (I2C) | Byte Position | +| ------- | ----------- | ----------- | ------------- | +| NFC_KP0 | 0x11 | 0x1011 | Byte 0 | +| NFC_KP1 | 0x13 | 0x1013 | Byte 0 | +| NFC_KP2 | 0x15 | 0x1015 | Byte 0 | +| NFC_KP3 | 0x17 | 0x1017 | Byte 0 | + +**NFC_KPx Bit Meanings (AES mode):** + +Each NFC_KPx is a 1-byte bitmask. Bit = 1 grants that capability after successful mutual authentication using KeyID = x. + +| Bit | Name | Privilege Granted | +| --- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------ | +| 7 | Restricted AREA_1 Write | Write access to restricted user memory AREA_1 | +| 6 | Restricted AREA_1 Read | Read access to restricted user memory AREA_1 | +| 5 | Crypto Config | Preset of Authentication Limit | +| 4 | EAS/AFI | Access to write-alike EAS/AFI commands: PROTECT EAS/AFI, SET EAS, RESET EAS, LOCK EAS, WRITE EAS ID, WRITE AFI, LOCK AFI | +| 3 | Destroy | Access to DESTROY functionality | +| 2 | Privacy | Enable/disable PRIVACY mode | +| 1 | Write | Write access to the protected user memory area | +| 0 | Read | Read access to the protected user memory area | + +**Common Privilege Values:** +- `0x03` = Read + Write access to protected memory +- `0x07` = Read + Write + Privacy control +- `0xFF` = All privileges enabled + +### AES Key Storage + +**Key Addresses (per NTP53x2 datasheet page 16):** +| Key | Blocks (NFC) | Blocks (I2C) | Size | +| ---- | ------------ | ------------- | ------------------- | +| KEY0 | 0x20-0x23 | 0x1020-0x1023 | 16 bytes (4 blocks) | +| KEY1 | 0x24-0x27 | 0x1024-0x1027 | 16 bytes | +| KEY2 | 0x28-0x2B | 0x1028-0x102B | 16 bytes | +| KEY3 | 0x2C-0x2F | 0x102C-0x102F | 16 bytes | + +**NOTE:** Addresses 0x1E-0x1F are RFU (reserved), NOT key storage! + +**Default Key Value:** Unknown - likely not all zeros. Factory default may vary. + +**Key Write Requirements:** +- Must enable AES mode (DEV_SEC_CONFIG = 0xA2) before writing keys +- Keys can be written and read back while header is 0x81 (not active) +- Once a key header is set to 0xE7, the key storage becomes unreadable +- Always verify key values by reading back before activating headers + +## Safe Programming Order + +This order prevents locking yourself out when provisioning AES mode on an NTP5332. + +### Step 1: Keep NFC_GCH at 0x81 (Deactivated) + +While NFC_GCH = 0x81: +- Protection settings aren't enforced yet +- Protection pointer/conditions can be modified via PROTECT PAGE without prior mutual auth +- Keys and privileges are freely readable/writable + +**WARNING:** NFC_GCH only programs one-way (lower→higher) and is irreversible. Don't jump to C1/C7/E7 until you've tested. + +### Step 2: Set Up Memory Protection Layout + +Configure these while GCH is still 0x81: +- Protection Pointer Address +- Protection Pointer Condition +- Any EAS/AFI protection conditions you plan to use + +### Step 3: Write Key Privileges and Keys + +While each NFC_KHx is still 0x81 (Not active): +- Write NFC_KPx (Key Privileges) to define what each key can authorize +- Write the 128-bit key values to storage blocks (0x20-0x2F) +- Key and privileges are readable/writable via READ/WRITE CONFIG +- Key is not active for authentication yet + +**Key Storage Addresses (per datasheet page 16):** +- KEY0: blocks 0x20-0x23 +- KEY1: blocks 0x24-0x27 +- KEY2: blocks 0x28-0x2B +- KEY3: blocks 0x2C-0x2F + +### Step 4: Activate/Lock Only the Keys You Need + +For each key you intend to use: +- Set NFC_KHx → 0xE7 (Active and locked) +- Key becomes usable for authentication +- Key material can no longer be read/written via READ/WRITE CONFIG + +For unused keys (NXP recommendation): +- Set NFC_KHx → 0xFF (Disabled) +- Prevents unused key slots from being exploited + +**WARNING:** Key headers are one-way (lower→higher) and irreversible. + +### Step 5: Enable Enforcement + +Set NFC_GCH → 0xC1 (Access right activated): +- Read/write protection becomes enforced according to your settings +- EAS/AFI protection becomes enforced +- Keys can still be modified if their headers are still 0x81 + +### Step 6: Test Everything + +Verify your protection configuration: +- Try protected reads/writes WITHOUT mutual auth (should be denied) +- Mutual-auth with each KeyID and confirm only the privileges in that key's KPx work +- Test all access patterns you need + +### Step 7: (Optional) Freeze Configuration + +Only after thorough testing: +- NFC_GCH → 0xC7: Locks privileges, keeps enforcement, keys still modifiable per their KH status +- NFC_GCH → 0xE7: "Finalize" state - enforcement on, all key headers/privileges/keys locked forever + +### Step 8: (Optional) Lock Authentication Limit + +Only when you're sure about auth-limit behavior: +- Set NFC_CCH → 0xE7 to lock the authentication-limit setting +- After this, auth limit can only be modified with a key that has Crypto Config privilege (bit 5) + +### Footgun to Avoid + +**Authentication Limit:** If you enable the Authentication Limit feature (block 0x0E), hitting the terminal failure count causes **permanent, irreversible lockout** of that key slot. Set a reasonable limit or leave at 0 (unlimited) during development. + +## Legacy Key Provisioning (Simple Version) + +For simple use cases (single key, no protection pointer): + +1. **Ensure AES Mode is Enabled** + - Read DEV_SEC_CONFIG at 0x3F + - If bits 3-0 != 0010b, write new value with AES mode enabled + - Power cycle the chip (remove and replace on reader) + +2. **Write Key Value** + - Write 16 bytes to KEY0 blocks 0x20-0x23 + - Key can only be written when NFC_KH0 is 0x81 + +3. **Activate Key Header** + - Write 0xE7 to block 0x10 byte 1 + - This is IRREVERSIBLE - key is locked after this + +4. **(Optional) Activate NFC_GCH** + - Write 0xC1 or 0xE7 to enable protection pointer enforcement + - 0xE7 locks all crypto configuration permanently + +## Authentication Protocol + +### COMPLETE MAM AUTHENTICATION (VERIFIED WORKING 2026-02-13) + +**NTAG5 Link uses UNDOCUMENTED AuthMethod values for MAM:** +| Value | Method | Result | +| ----- | --------------- | -------------------------------------------------- | +| 0x00 | TAM1 (Tag Auth) | SUCCESS - 18 byte response | +| 0x02 | **MAM1** | SUCCESS - 24 byte response | +| 0x06 | **MAM2** | SUCCESS - empty response (authentication complete) | +| 0x80 | MAM1 (per ISO) | Error 0x0F (NOT SUPPORTED) | +| 0x90 | MAM2 (per ISO) | Error 0x0F (NOT SUPPORTED) | + +**CRITICAL BYTE ORDERING RULES:** +All data to/from the chip must be byte-reversed: +- IChallenge sent to chip: **REVERSED** +- TChallenge_high from response: **REVERSED** +- IResponse sent to chip: **REVERSED** + +### MAM1 (AuthMethod = 0x02) + +**Command Format:** +``` +FLAGS(0x22) | CMD(0x35) | UID(8) | CSI(0x00) | AuthMethod(0x02) | KeyID | IChallenge[::-1] +``` +- IChallenge: 10 bytes, must be **REVERSED** before sending + +**Response Structure (24 bytes):** +| Offset | Length | Content | +| ------ | ------ | ---------------------------------------------- | +| 0 | 1 | Flags: 0x04 | +| 1 | 1 | Header: 0xA7 | +| 2-7 | 6 | TChallenge[79:32] (plaintext, needs reversal!) | +| 8-23 | 16 | Encrypted block | + +**Decryption Procedure:** +1. Extract TChallenge_high_raw = bytes 2-7 (6 bytes) +2. Extract encrypted block = bytes 8-23 (16 bytes) +3. **REVERSE** encrypted block, then AES-ECB decrypt +4. Parse decrypted: `C_MAM1(2) || TChallenge[31:0](4) || IChallenge(10)` +5. **REVERSE** TChallenge_high_raw to get actual TChallenge_high +6. TChallenge[79:0] = TChallenge_high + TChallenge_low + +**Verified Constants:** +- C_MAM1 = 0xDA83 +- IChallenge in decrypted = original IChallenge (chip de-reverses what it receives) + +### MAM2 (AuthMethod = 0x06) + +**Command Format (NO KeyID!):** +``` +FLAGS(0x22) | CMD(0x35) | UID(8) | CSI(0x00) | AuthMethod(0x06) | IResponse[::-1] +``` +- IResponse: 16 bytes, must be **REVERSED** before sending + +**IResponse Computation:** +```python +# Build plaintext (16 bytes) +C_MAM2_PURPOSE = bytes([0xDA, 0x80]) # C_MAM2[11:0] || Purpose[3:0] +IChallenge_31_0 = original_ichallenge[6:10] # Last 4 bytes of ORIGINAL +TChallenge_79_0 = tc_high_reversed + tc_low # From MAM1 response + +plaintext = C_MAM2_PURPOSE + IChallenge_31_0 + TChallenge_79_0 + +# Compute IResponse +iresponse_to_send = AES_DEC(key, plaintext)[::-1] # Decrypt then REVERSE +``` + +**Expected Response:** Empty (just flags/header, no data) = SUCCESS + +### Complete MAM Example (Python) + +```python +from Crypto.Cipher import AES + +def perform_mam(hcard, proto, uid_lsb, key, key_id, ichallenge): + cipher = AES.new(key, AES.MODE_ECB) + + # MAM1: Send IChallenge REVERSED + mam1_cmd = bytes([0x22, 0x35]) + uid_lsb + bytes([0x00, 0x02, key_id]) + ichallenge[::-1] + mam1_resp = send_command(hcard, proto, mam1_cmd) + + # Parse MAM1 response + tc_high_raw = mam1_resp[2:8] # Needs reversal! + encrypted = mam1_resp[8:24] + + decrypted = cipher.decrypt(encrypted[::-1]) + assert decrypted[0:2] == bytes([0xDA, 0x83]) # C_MAM1 + + tc_high = tc_high_raw[::-1] # REVERSE! + tc_low = decrypted[2:6] + tc_full = tc_high + tc_low + + # MAM2: Build IResponse + C_MAM2_PURPOSE = bytes([0xDA, 0x80]) + ich_31_0 = ichallenge[6:10] # Original, not reversed + plaintext = C_MAM2_PURPOSE + ich_31_0 + tc_full + iresponse = cipher.decrypt(plaintext)[::-1] # Decrypt then REVERSE + + # MAM2: No KeyID in command! + mam2_cmd = bytes([0x22, 0x35]) + uid_lsb + bytes([0x00, 0x06]) + iresponse + mam2_resp = send_command(hcard, proto, mam2_cmd) + + return mam2_resp is empty or success +``` + +**Verified Working (2026-02-13):** All 4 key slots (KEY0-KEY3) authenticate successfully using this protocol. + +**Standard MAM1/MAM2 (0x80/0x90) fail** with Error 0x0F despite the chip's GET NXP SYSTEM INFORMATION reporting MUTUAL AUTH = 1. Use AuthMethod 0x02/0x06 instead. + +--- + +### ISO 29167-10 Standard MAM (AuthMethod 0x80/0x90) - NOT SUPPORTED + +**NOTE:** The standard ISO 29167-10 AuthMethod values (0x80 for MAM1, 0x90 for MAM2) are **NOT supported** by NTAG5 Link hardware, despite documentation suggesting otherwise. + +Use the undocumented AuthMethod values instead: +- **AuthMethod 0x02** for MAM1 +- **AuthMethod 0x06** for MAM2 (no KeyID in command) + +See "Complete MAM Authentication" section above for the working protocol. + +## TAM1 Authentication (Tag Authentication Method 1) + +TAM1 is a one-way authentication where the reader verifies the tag's identity. It uses the CHALLENGE (0x39) and READBUFFER (0x3A) commands instead of the full mutual authentication. + +### TAM1 Protocol Flow + +1. **Reader sends CHALLENGE** - Contains KeyID and random IChallenge +2. **Tag computes AES encryption** - No response sent +3. **Reader sends READBUFFER** - Requests the crypto result +4. **Tag returns TResponse** - 16-byte encrypted block +5. **Reader decrypts and verifies** - Checks C_TAM1 constant and IChallenge + +### TAM1 Message Format + +**CHALLENGE Command (0x39):** +``` +Flags | 0x39 | CSI | AuthMethod/RFU | KeyID | IChallenge (10 bytes) +``` +- Flags: 0x02 (high data rate, NOT addressed) +- CSI: 0x00 (AES-128 Crypto Suite Identifier) +- AuthMethod/RFU: 0x00 (TAM1 = 00b, CustomData = 0, RFU = 00000b) +- KeyID: 0x00-0x03 (KEY0-KEY3) +- IChallenge: 10 bytes random challenge + +**READBUFFER Command (0x3A):** +``` +Flags | 0x3A +``` +- Flags: 0x02 (high data rate, NOT addressed) + +**TResponse Format (16 bytes):** +``` +AES-ECB-ENC(Key, C_TAM1 || TRnd || IChallenge) +``` +- C_TAM1: 0x96C5 (2 bytes) - constant per ISO 29167-10 +- TRnd: 4 bytes random from tag +- IChallenge: 10 bytes echoed from CHALLENGE command + +### TAM1 Verification + +To verify TAM1: +1. **Reverse the TResponse byte order** (ACR1552 returns data in LSB-first order) +2. Decrypt TResponse with the expected key using AES-128-ECB +3. Check bytes 0-1 equal 0x96C5 (C_TAM1 constant) +4. **Reverse the echoed IChallenge** (bytes 6-15) before comparing +5. Check reversed bytes 6-15 equal the IChallenge you sent +6. If both match, the tag is authentic + +**CRITICAL: ACR1552 Byte Ordering** + +The ACR1552 returns TResponse in reversed byte order due to ISO 15693's LSB-first transmission. +Both the entire TResponse and the echoed IChallenge within the decrypted data must be reversed. + +```python +# Example verification (Python) +tresponse_raw = resp[1:17] # Skip flags byte +tresponse = tresponse_raw[::-1] # Reverse entire TResponse + +cipher = AES.new(key, AES.MODE_ECB) +decrypted = cipher.decrypt(tresponse) + +c_tam1 = decrypted[0:2] # Should be 96C5 +trnd = decrypted[2:6] # 4 bytes random from tag +echoed_raw = decrypted[6:16] # Echoed IChallenge (reversed) +echoed = echoed_raw[::-1] # Reverse to get original order + +if c_tam1 == bytes([0x96, 0xC5]) and echoed == ichallenge: + # Authentication successful +``` + +### ACR1552 Complete APDU Format for TAM1 + +The ACR1552 uses pseudo-APDUs with TLV wrapping for transparent mode communication. + +**PCSC APDU Structure:** +``` +FF C2 00 01 Lc [TLV Data] 00 +``` +- CLA: 0xFF (pseudo-APDU) +- INS: 0xC2 (transparent exchange) +- P1: 0x00 +- P2: 0x01 (exchange function) +- Lc: Length of TLV data +- Le: 0x00 + +**TLV Data Structure:** +``` +5F46 04 [timeout_4bytes] Timeout in microseconds (big-endian) +FF6E 03 03 01 0F FWTI (Frame Waiting Time Integer) +95 xx [iso15693_command] ISO 15693 command payload +``` + +**Response TLV Structure:** +``` +C0 03 [status_3bytes] Error/status TLV (SW1 SW2 at bytes 1-2) +92 01 xx Framing TLV +96 02 xx xx Status TLV +97 xx [iso15693_response] Response data TLV (includes flags byte) +``` +- SW = 0x9000: Success +- SW = 0x6401: Execution error (no response from ICC) + +### Complete APDU Examples (Working TAM1 Flow) + +**Example 1: GET SYSTEM INFO (required before CHALLENGE)** +``` +TX: ffc20001115f4604000f4240ff6e0303010f9502022b00 + ├─ FF C2 00 01 11 PCSC header (Lc=17) + ├─ 5F46 04 000F4240 Timeout = 1,000,000 µs (1 second) + ├─ FF6E 03 030100F FWTI + ├─ 95 02 022B ISO15693: Flags=02, Cmd=2B (GET SYSTEM INFO) + └─ 00 Le + +RX: c00300900092010096020000970f000f02e88c59580104e00000ff0301 + ├─ C0 03 009000 Status OK (SW=9000) + ├─ 92 01 00 Framing OK + ├─ 96 02 0000 Status OK + └─ 97 0F 00... Response: Flags=00 + System Info (14 bytes) +``` + +**Example 2: CHALLENGE (KeyID=3, IChallenge=33ccc451ad4d9a6e9cd6)** +``` +TX: ffc200011e5f4604000f4240ff6e0303010f950f023900000333ccc451ad4d9a6e9cd600 + ├─ FF C2 00 01 1E PCSC header (Lc=30) + ├─ 5F46 04 000F4240 Timeout = 1 second + ├─ FF6E 03 03010F FWTI + ├─ 95 0F 02 39 00 00 03 33ccc451ad4d9a6e9cd6 + │ ├─ 02 Flags (high data rate) + │ ├─ 39 CHALLENGE command + │ ├─ 00 CSI (AES-128) + │ ├─ 00 AuthMethod=TAM1 + │ ├─ 03 KeyID=3 + │ └─ 33ccc451ad4d9a6e9cd6 IChallenge (10 bytes) + └─ 00 Le + +RX: c003036401 + └─ C0 03 036401 Status: SW=6401 (no response from ICC) + This is EXPECTED - CHALLENGE has no response +``` + +**Example 3: READBUFFER** +``` +TX: ffc20001115f4604000f4240ff6e0303010f9502023a00 + ├─ FF C2 00 01 11 PCSC header (Lc=17) + ├─ 5F46 04 000F4240 Timeout = 1 second + ├─ FF6E 03 03010F FWTI + ├─ 95 02 02 3A ISO15693: Flags=02, Cmd=3A (READBUFFER) + └─ 00 Le + +RX: c00300900092010096020000971102238333363a3142b26328d1bf618362e4 + ├─ C0 03 009000 Status OK + ├─ 92 01 00 Framing OK + ├─ 96 02 0000 Status OK + └─ 97 11 02 238333363a3142b26328d1bf618362e4 + ├─ 02 Flags (success, no error) + └─ 238333...e4 TResponse (16 bytes) +``` + +### Critical Timing Requirement + +**IMPORTANT:** The chip requires time to compute the AES encryption after receiving CHALLENGE. + +| Scenario | READBUFFER Result | +| ----------------------------------------------------- | ------------------------- | +| READBUFFER sent immediately after CHALLENGE | Often fails (no response) | +| GET SYSTEM INFO sent between CHALLENGE and READBUFFER | Works (implicit delay) | +| 1 second delay between CHALLENGE and READBUFFER | Works reliably | + +**Recommended approach:** Either: +1. Send any command (e.g., GET SYSTEM INFO) after CHALLENGE before READBUFFER, OR +2. Add a minimum 50-100ms delay after CHALLENGE before sending READBUFFER + +The crypto calculation typically takes ~4-8ms, but reader/driver overhead can cause timing issues if READBUFFER is sent too quickly. + +### READBUFFER Failure Example (Insufficient Delay) + +When CHALLENGE is sent immediately after connection with no prior commands and READBUFFER follows too quickly: + +``` +TX: ffc200011e5f4604000f4240ff6e0303010f950f0239000003[IChallenge]00 +RX: c003036401 (SW=6401, expected - no response for CHALLENGE) + +TX: ffc20001115f4604000f4240ff6e0303010f9502023a00 +RX: c003036401 (SW=6401, FAILURE - chip not ready) +``` + +The second `c003036401` response indicates "no response from ICC" - the chip hasn't finished computing the crypto result yet. Adding a delay or sending an intermediate command resolves this. + +### TAM1 vs State Machine + +Per the datasheet, CHALLENGE can only be executed in READY state with non-addressed mode. However, testing shows: + +- CHALLENGE works even after GET SYSTEM INFO (which puts chip in SELECTED state) +- The critical factor is the **timing delay**, not the chip state +- Sending GET SYSTEM INFO first provides the necessary delay for crypto completion + +## Reader-Specific Notes + +### ACS ACR1552 + +- Uses transparent mode with TLV-wrapped commands +- Supports ISO15693 natively +- Switch protocol command: `8F 02 02 03` (ISO15693 Layer 3) +- Timeout TLV: `5F46` with 4-byte timeout in microseconds +- Commands work correctly for basic operations +- CHALLENGE returns SW=6401 (no response from ICC) - this is expected +- READBUFFER returns SW=9000 with 17-byte response (flags + 16-byte TResponse) +- Authentication returns error 0x0F when key is not properly configured + +**CRITICAL: Byte Order Reversal Required** +- TResponse data is returned in LSB-first (reversed) byte order +- Must reverse entire TResponse before AES decryption +- Echoed IChallenge in decrypted data is also reversed +- See TAM1 Verification section for correct processing order +- AUTHENTICATE (0x35) command returns "no ICC response" - use CHALLENGE/READBUFFER instead + +### HID Omnikey 5022 CL + +- Uses transceive APDU: `FF 68 0E 03` +- 10-byte header: TxRxFlags | ValidBits | Timeout(4) | RFU(4) +- TxRxFlags: 0x05 for ISO15693 (CRC TX + CRC RX enabled) +- Response format: STS(1) | RxB(1) | Card_Response(n) | SW(2) +- Direct transceive works WITHOUT entering transparent session first +- ISO15693 transparent session may not work reliably - use direct transceive + +**AUTHENTICATE (0x35) for TAM1 - CONFIRMED WORKING EXAMPLE:** + +TAM1 command format (AuthMethod = 0x00): +``` +Flags(22) | Cmd(35) | UID(8) | CSI(00) | AuthMethod(00) | KeyID | IChallenge(10) +``` + +**Verified Example (2026-02-13):** + +Input parameters: +- Chip UID: E0040158F08BE802 (MSB) / 02E88BF0580104E0 (LSB) +- KEY0: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA +- IChallenge: 112233445566778899AA + +Command sent to reader: +``` +FF680E03210500004C4B4000000000223502E88BF0580104E0000000112233445566778899AA00 +``` + +| Bytes | Value | Meaning | +| -------------------- | ---------- | -------------------- | +| FF680E03 | Header | Omnikey transceive | +| 21 | Lc=33 | Data length | +| 05 | TxRxFlags | ISO15693 CRC | +| 00 | ValidBits | Whole bytes | +| 004C4B40 | Timeout | 5 seconds | +| 00000000 | RFU | Reserved | +| 22 | Flags | Addressed, high rate | +| 35 | Cmd | AUTHENTICATE | +| 02E88BF0580104E0 | UID | LSB order | +| 00 | CSI | AES-128 | +| 00 | AuthMethod | TAM1 | +| 00 | KeyID | KEY0 | +| 112233445566778899AA | IChallenge | 10 bytes | +| 00 | Le | Response length | + +Response received: +``` +000006A7F2E114AFC63A0F6A7170A4F1692DD29F9000 +``` + +| Bytes | Value | Meaning | +| -------------------------------- | --------- | ------------------ | +| 0000 | STS+RxB | Transceive OK | +| 06 | Flags | Success | +| A7 | Header | Barker code (skip) | +| F2E114AFC63A0F6A7170A4F1692DD29F | TResponse | 16 bytes encrypted | +| 9000 | SW | Success | + +**Decryption process:** + +1. Extract TResponse: `F2E114AFC63A0F6A7170A4F1692DD29F` +2. REVERSE bytes: `9FD22D69F1A470716A0F3AC6AF14E1F2` +3. AES-ECB decrypt with KEY0: `96C51B6F50BBAA998877665544332211` +4. Parse result: + - C_TAM1: `96C5` (correct constant) + - TRnd: `1B6F50BB` (tag random) + - Echoed: `AA998877665544332211` (IChallenge reversed) + +**VERIFICATION: SUCCESS** - C_TAM1=96C5, IChallenge matches (reversed) + +```python +# Omnikey 5022 TAM1 verification (AuthMethod=0x00) +card_resp = response[2:-2] # Strip STS, RxB, and SW +data_after_flags = card_resp[1:] # Skip flags byte (0x06) +tresponse = data_after_flags[1:17] # Skip header byte (0xA7), take 16 bytes +tresponse = tresponse[::-1] # Reverse byte order + +cipher = AES.new(key, AES.MODE_ECB) +decrypted = cipher.decrypt(tresponse) + +c_tam1 = decrypted[0:2] # Should be 96C5 +trnd = decrypted[2:6] # 4-byte tag random +echoed = decrypted[6:16] # REVERSED - compare against ichallenge[::-1] + +if c_tam1 == bytes([0x96, 0xC5]) and echoed == ichallenge[::-1]: + # TAM1 authentication successful +``` + +## Error Codes + +### ISO15693 Error Codes (from flag byte with error bit set) + +| Code | Description | +| ---- | ---------------------- | +| 0x01 | Command not supported | +| 0x02 | Command not recognized | +| 0x03 | Option not supported | +| 0x0F | Unknown error | +| 0x10 | Block not available | +| 0x11 | Block already locked | +| 0x12 | Block locked | +| 0x13 | Programming failed | +| 0x14 | Locking failed | + +**Error 0x0F during authentication typically means:** +- Key header not activated (still 0x81 instead of 0xE7) +- Wrong key slot used +- Chip not in AES mode +- Malformed MAM1 message + +## READ CONFIG Command Requirements + +**IMPORTANT:** READ CONFIG (0xC0) must use **unaddressed mode** (flags=0x02). + +**Correct Format:** +``` +Flags | Cmd | MfgCode | BlockAddr | NumBlocks +0x02 | 0xC0 | 0x04 | addr | 0x00 +``` + +Example: `bytes([0x02, 0xC0, 0x04, 0x3F, 0x00])` to read DEV_SEC_CONFIG + +**Wrong Format (does not work):** +``` +Flags | Cmd | MfgCode | UID (8 bytes) | BlockAddr +0x22 | 0xC0 | 0x04 | uid_lsb | addr +``` + +Using addressed mode (flags=0x22 with UID) returns no response from the chip. + +## Common Issues + +1. **"Unknown error" (0x0F) on AUTHENTICATE** + - Check DEV_SEC_CONFIG is in AES mode (bits 3-0 = 0010b) + - Check Key Header is 0xE7 (active and locked) + - Power cycle chip after mode changes + +2. **Key Header reads as 0x00** + - Incorrect address used - Key headers are at 0x10, 0x12, 0x14, 0x16 + - Block 0x0E is NFC_AUTH_LIMIT, not KEY0 header + +3. **Cannot write key after activation** + - Once key header is 0xE7, key is permanently locked + - Must set key value BEFORE activating header + +4. **Mode change doesn't take effect** + - Configuration changes require power cycle (POR) + - Remove card from reader field and place again + +5. **Keys must be written in AES mode** + - Enable AES mode (DEV_SEC_CONFIG = 0xA2) before writing key values + - Power cycle after enabling AES mode, then write keys + - Keys written while in password mode may not work correctly for authentication + +6. **TAM1/MAM1 authentication fails despite correct key** + - Key values verified by read-back but authentication still fails + - This is a known issue under investigation with NXP (open support ticket) + - Ensure chip is in AES mode before writing keys + +## Troubleshooting Checklist + +Before attempting authentication: + +1. **Verify AES Mode:** + ``` + Read config block 0x3F + DEV_SEC_CONFIG byte 0, bits 3-0 should be 0010b + Example: 0xA2 = AES mode enabled, writable + ``` + +2. **Verify Key Header is Active:** + ``` + Read config block 0x10 (KEY0), 0x12 (KEY1), 0x14 (KEY2), or 0x16 (KEY3) + Byte 1 should be 0xE7 for the key you want to use + 0x81 = not active (cannot authenticate) + ``` + +3. **Verify Key Value is Known:** + ``` + If key header was 0x81 before you configured it: + - Was AES mode enabled BEFORE writing key values? + - Did read-back verification match what you wrote? + - After activating header to 0xE7, key blocks should return errors (protected) + ``` + +4. **Check Auth Failure Counter:** + ``` + Read config block 0x0E + NFC_AUTH_LIMIT (bits 9-0) sets max failures before lockout + If counter reaches limit, key slot is permanently blocked + ``` + +5. **Test with Correct Command Format:** + ``` + AUTHENTICATE: 0x22 0x35 0x00 0x80 + - FLAGS = 0x22 (high data rate, addressed mode) - REQUIRED + - CSI = 0x00 (AES-128) + - MAM1_Header = 0x80 + ``` diff --git a/docs/plans/2026-03-17-trace-formatter-design.md b/docs/plans/2026-03-17-trace-formatter-design.md new file mode 100644 index 0000000..e0cef38 --- /dev/null +++ b/docs/plans/2026-03-17-trace-formatter-design.md @@ -0,0 +1,134 @@ +# Trace Formatter Design + +## Problem + +The sim session trace output is a raw `print()` with no color, no command decoding, no wrapping, and no leading newline (so it sometimes renders on the REPL prompt). The PM3 C client has rich colored, decoded trace output — we want that for our Python sim traces. + +## Design + +### New module: `pm3py/sim/trace_fmt.py` + +A `TraceFormatter` class that owns all rendering logic. `_trace_reader` in `sim_session.py` delegates to it instead of calling `print()` directly. + +### Modes + +| Mode | Tag | Color | +|------|-----|-------| +| Sim (we are the tag) | `[Sim]` | Magenta (`\033[35m`) | +| Reader (we are the reader) | `[Rdr]` | Cyan (`\033[36m`) | +| Sniff (passive observer) | `[Snf]` | Default/white | + +### Direction colors + +| Direction | Color | +|-----------|-------| +| Reader -> Tag (command) | Cyan (`\033[36m`) | +| Tag -> Reader (response) | Yellow (`\033[33m`) | + +### Additional colors + +| Element | Color | +|---------|-------| +| Command annotation | Green (`\033[32m`) | +| Hex bytes | Dim (`\033[2m`) | + +Colors suppressed when `sys.stdout.isatty()` is False. + +### Layout + +``` +\n[Sim] Reader -> Tag: 26 01 00 INVENTORY + ^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^ + mode prefix(colored) hex(dim) annotation(green) +``` + +Prefix column = fixed width (e.g., 24 chars). This is the indent for continuation lines. + +Hex bytes are space-separated (`de ad be ef`, not `deadbeef`). + +### Wrapping + +When hex + annotation fit on one line: annotation appears inline after hex. + +When hex overflows terminal width: hex wraps at prefix column, annotation goes on its own line below (also at prefix column). + +``` +\n[Sim] Tag -> Reader: 00 de ad be ef 00 00 00 00 00 00 00 00 00 00 + 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fe a1 + READ MULTIPLE BLOCK [0x00, 13 blocks] OK +``` + +### Terminal width handling + +- Cache `os.get_terminal_size()` on init +- Register `signal.SIGWINCH` handler (Unix) to update cached width +- Fallback: re-read every 10 lines if signal registration fails +- Default 80 columns if no TTY + +### ISO 15693 command decode table + +Extracted from command byte (byte index 1 of frame): + +| Cmd | Name | Extra annotation | +|-----|------|-----------------| +| 0x01 | INVENTORY | mask_len if present | +| 0x02 | STAY QUIET | -- | +| 0x20 | READ SINGLE BLOCK | block number | +| 0x21 | WRITE SINGLE BLOCK | block number + data length | +| 0x23 | READ MULTIPLE BLOCK | start block, count | +| 0x26 | RESET TO READY | -- | +| 0x2B | GET SYSTEM INFO | -- | +| 0x2C | GET MULTIPLE BLOCK SECURITY | -- | +| 0xA1 | NXP READ CONFIG | reg index | +| 0xA2 | NXP WRITE CONFIG | reg index, value | +| 0xB2 | NXP GET RANDOM | -- | +| 0xB3 | NXP SET PASSWORD | pwd_id | + +Response decoding: flags byte 0x00 = OK, 0x01 = ERROR + error code. Inventory responses show UID. Read responses show data. + +### ISO 14443-A command decode table + +| Byte(s) | Name | +|---------|------| +| 0x26 | REQA | +| 0x52 | WUPA | +| 0x50 00 | HLTA | +| 0x93 20 | ANTICOL CL1 | +| 0x93 70 | SELECT CL1 | +| 0x95 20/70 | ANTICOL/SELECT CL2 | +| 0x97 20/70 | ANTICOL/SELECT CL3 | +| 0xE0 | RATS | +| 0x02/0x03 | I-BLOCK | +| 0xA2/0xA3 | R-ACK | +| 0xB2/0xB3 | R-NAK | +| 0xC2/0xF2 | S(DESELECT)/S(WTX) | + +Response decoding: ATQA as `ATQA xx xx`, SAK as `SAK xx`, ATS as `ATS [len]`. + +### API + +```python +class TraceFormatter: + def __init__(self, mode: str = "sim"): + """mode: "sim", "reader", or "sniff" """ + + def format(self, direction: int, payload: bytes) -> str: + """Returns fully formatted, colored, wrapped string with leading \\n.""" + + def print(self, direction: int, payload: bytes) -> None: + """format() + print to stdout.""" +``` + +Decoder functions: `decode_15693(direction, payload) -> str | None` and `decode_14443a(direction, payload) -> str | None`. + +### Integration + +- `sim_session.py`: instantiate `TraceFormatter("sim")` in `start_15693()`, call `self._formatter.print()` in `_trace_reader()` +- 14443-A trace path: decode table built now, wired up when firmware trace path lands +- No changes to `replay.py` or `InteractiveReader.dump_trace()` + +### Out of scope + +- Replay/recorder trace formatting +- InteractiveReader dump formatting +- Protocol decode beyond 15693 + 14443-A diff --git a/docs/plans/2026-03-17-trace-formatter-plan.md b/docs/plans/2026-03-17-trace-formatter-plan.md new file mode 100644 index 0000000..53154d7 --- /dev/null +++ b/docs/plans/2026-03-17-trace-formatter-plan.md @@ -0,0 +1,998 @@ +# Trace Formatter Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Replace raw `print()` trace output with colored, decoded, column-wrapped trace rendering for sim sessions. + +**Architecture:** New `trace_fmt.py` module with `TraceFormatter` class + protocol decode functions. `sim_session.py` delegates to it. Terminal width detected dynamically with SIGWINCH support. + +**Tech Stack:** Python stdlib only (os, signal, sys, struct, shutil). No external dependencies. + +--- + +### Task 1: ISO 15693 request decoder + +**Files:** +- Create: `pm3py/sim/trace_fmt.py` +- Test: `tests/test_sim_trace_fmt.py` + +**Step 1: Write failing tests for 15693 request decoding** + +```python +"""Tests for pm3py.sim.trace_fmt — trace formatting and protocol decoding.""" +import os +import pytest + +from pm3py.sim.trace_fmt import decode_15693 + + +class TestDecode15693Request: + """Decode reader->tag (direction=0) 15693 commands.""" + + def test_inventory(self): + # flags=0x26, cmd=0x01, mask_len=0 + payload = bytes([0x26, 0x01, 0x00]) + result = decode_15693(0, payload) + assert result == "INVENTORY" + + def test_inventory_with_mask(self): + payload = bytes([0x26, 0x01, 0x08, 0xAB]) + result = decode_15693(0, payload) + assert result == "INVENTORY mask=8" + + def test_stay_quiet(self): + payload = bytes([0x22, 0x02]) + b"\x01" * 8 + result = decode_15693(0, payload) + assert result == "STAY QUIET" + + def test_read_single_block_addressed(self): + # flags=0x22 (addressed), cmd=0x20, uid(8), block=0x03 + payload = bytes([0x22, 0x20]) + b"\x01" * 8 + bytes([0x03]) + result = decode_15693(0, payload) + assert result == "READ SINGLE BLOCK #3" + + def test_read_single_block_unaddressed(self): + # flags=0x02 (no address flag), cmd=0x20, block=0x00 + payload = bytes([0x02, 0x20, 0x00]) + result = decode_15693(0, payload) + assert result == "READ SINGLE BLOCK #0" + + def test_write_single_block(self): + payload = bytes([0x22, 0x21]) + b"\x01" * 8 + bytes([0x05]) + b"\xDE\xAD\xBE\xEF" + result = decode_15693(0, payload) + assert result == "WRITE SINGLE BLOCK #5 [4B]" + + def test_read_multiple_block(self): + # flags=0x22, cmd=0x23, uid(8), start=0x00, count=0x0D + payload = bytes([0x22, 0x23]) + b"\x01" * 8 + bytes([0x00, 0x0D]) + result = decode_15693(0, payload) + assert result == "READ MULTIPLE BLOCK #0+13" + + def test_reset_to_ready(self): + payload = bytes([0x22, 0x26]) + b"\x01" * 8 + result = decode_15693(0, payload) + assert result == "RESET TO READY" + + def test_get_system_info(self): + payload = bytes([0x22, 0x2B]) + b"\x01" * 8 + result = decode_15693(0, payload) + assert result == "GET SYSTEM INFO" + + def test_get_multiple_block_security(self): + payload = bytes([0x22, 0x2C]) + b"\x01" * 8 + bytes([0x00, 0x3F]) + result = decode_15693(0, payload) + assert result == "GET MULTIPLE BLOCK SECURITY #0+63" + + def test_nxp_read_config(self): + # flags=0x22, cmd=0xA1, mfg=0x04, reg=0x02 + payload = bytes([0x22, 0xA1, 0x04, 0x02]) + result = decode_15693(0, payload) + assert result == "NXP READ CONFIG reg=2" + + def test_nxp_write_config(self): + payload = bytes([0x22, 0xA2, 0x04, 0x03, 0xFF]) + result = decode_15693(0, payload) + assert result == "NXP WRITE CONFIG reg=3 val=0xFF" + + def test_nxp_get_random(self): + payload = bytes([0x22, 0xB2, 0x04]) + result = decode_15693(0, payload) + assert result == "NXP GET RANDOM" + + def test_nxp_set_password(self): + payload = bytes([0x22, 0xB3, 0x04, 0x01]) + b"\x00" * 4 + result = decode_15693(0, payload) + assert result == "NXP SET PASSWORD id=1" + + def test_unknown_command(self): + payload = bytes([0x22, 0xFF]) + result = decode_15693(0, payload) + assert result is None + + def test_too_short(self): + result = decode_15693(0, bytes([0x26])) + assert result is None +``` + +**Step 2: Run tests to verify they fail** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestDecode15693Request -v` +Expected: FAIL — `ModuleNotFoundError: No module named 'pm3py.sim.trace_fmt'` + +**Step 3: Implement 15693 request decoder** + +Create `pm3py/sim/trace_fmt.py`: + +```python +"""Trace formatter — colored, decoded, column-wrapped trace output.""" +from __future__ import annotations + +# ---- ISO 15693 flags ---- +_15693_FLAG_INVENTORY = 0x04 +_15693_FLAG_ADDRESS = 0x20 + +# ---- ISO 15693 command names ---- +_15693_CMDS = { + 0x01: "INVENTORY", + 0x02: "STAY QUIET", + 0x20: "READ SINGLE BLOCK", + 0x21: "WRITE SINGLE BLOCK", + 0x23: "READ MULTIPLE BLOCK", + 0x26: "RESET TO READY", + 0x2B: "GET SYSTEM INFO", + 0x2C: "GET MULTIPLE BLOCK SECURITY", +} + +# NXP custom commands (manufacturer code 0x04) +_15693_NXP_CMDS = { + 0xA1: "NXP READ CONFIG", + 0xA2: "NXP WRITE CONFIG", + 0xB2: "NXP GET RANDOM", + 0xB3: "NXP SET PASSWORD", +} + + +def _15693_block_offset(flags: int) -> int: + """Return the byte offset of the block number field after flags+cmd.""" + is_inventory = bool(flags & _15693_FLAG_INVENTORY) + if not is_inventory and (flags & _15693_FLAG_ADDRESS): + return 10 # flags(1) + cmd(1) + uid(8) + return 2 # flags(1) + cmd(1) + + +def decode_15693(direction: int, payload: bytes) -> str | None: + """Decode an ISO 15693 frame into a human-readable annotation. + + Args: + direction: 0 = reader->tag, 1 = tag->reader + payload: raw frame bytes + + Returns: + Annotation string or None if unrecognized. + """ + if direction == 0: + return _decode_15693_request(payload) + else: + return _decode_15693_response(payload) + + +def _decode_15693_request(payload: bytes) -> str | None: + if len(payload) < 2: + return None + + flags = payload[0] + cmd = payload[1] + + # Standard commands + name = _15693_CMDS.get(cmd) + if name is not None: + return _annotate_15693_request(name, cmd, flags, payload) + + # NXP custom commands (check manufacturer code) + name = _15693_NXP_CMDS.get(cmd) + if name is not None: + return _annotate_nxp_request(name, cmd, payload) + + return None + + +def _annotate_15693_request(name: str, cmd: int, flags: int, payload: bytes) -> str: + blk_off = _15693_block_offset(flags) + + if cmd == 0x01: # INVENTORY + if len(payload) > 2 and payload[2] > 0: + return f"{name} mask={payload[2]}" + return name + + if cmd in (0x20, 0x21): # READ/WRITE SINGLE BLOCK + if len(payload) > blk_off: + block = payload[blk_off] + if cmd == 0x21: + data_len = len(payload) - blk_off - 1 + return f"{name} #{block} [{data_len}B]" + return f"{name} #{block}" + return name + + if cmd in (0x23, 0x2C): # READ MULTIPLE / GET MULTIPLE BLOCK SECURITY + if len(payload) > blk_off + 1: + start = payload[blk_off] + count = payload[blk_off + 1] + return f"{name} #{start}+{count}" + return name + + return name + + +def _annotate_nxp_request(name: str, cmd: int, payload: bytes) -> str: + if cmd == 0xA1 and len(payload) >= 4: # READ CONFIG + return f"{name} reg={payload[3]}" + if cmd == 0xA2 and len(payload) >= 5: # WRITE CONFIG + return f"{name} reg={payload[3]} val=0x{payload[4]:02X}" + if cmd == 0xB3 and len(payload) >= 4: # SET PASSWORD + return f"{name} id={payload[3]}" + return name + + +def _decode_15693_response(payload: bytes) -> str | None: + # Placeholder — implemented in Task 2 + return None +``` + +**Step 4: Run tests to verify they pass** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestDecode15693Request -v` +Expected: all PASS + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/sim/trace_fmt.py tests/test_sim_trace_fmt.py +git commit --no-gpg-sign -m "feat(trace): add ISO 15693 request decoder" +``` + +--- + +### Task 2: ISO 15693 response decoder + +**Files:** +- Modify: `pm3py/sim/trace_fmt.py` — replace `_decode_15693_response` placeholder +- Test: `tests/test_sim_trace_fmt.py` + +**Step 1: Write failing tests** + +Append to `tests/test_sim_trace_fmt.py`: + +```python +class TestDecode15693Response: + """Decode tag->reader (direction=1) 15693 responses.""" + + def test_ok_empty(self): + # Just flags=0x00, no data (e.g., write ack) + result = decode_15693(1, bytes([0x00])) + assert result == "OK" + + def test_inventory_response(self): + # flags=0x00, dsfid=0x00, uid (8 bytes LSB-first) + uid_lsb = bytes([0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x04, 0xE0]) + payload = bytes([0x00, 0x00]) + uid_lsb + result = decode_15693(1, payload) + assert result == "OK INVENTORY UID=E004010101010101" + + def test_error_response(self): + payload = bytes([0x01, 0x0F]) + result = decode_15693(1, payload) + assert result == "ERROR 0x0F" + + def test_error_block_not_available(self): + payload = bytes([0x01, 0x10]) + result = decode_15693(1, payload) + assert result == "ERROR 0x10 block not available" + + def test_data_response(self): + # flags=0x00 + 4 data bytes (read single block response) + payload = bytes([0x00, 0xDE, 0xAD, 0xBE, 0xEF]) + result = decode_15693(1, payload) + assert result == "OK [4B]" + + def test_too_short(self): + result = decode_15693(1, b"") + assert result is None +``` + +**Step 2: Run to verify failure** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestDecode15693Response -v` +Expected: FAIL — responses decode as `None` + +**Step 3: Implement response decoder** + +Replace `_decode_15693_response` in `trace_fmt.py`: + +```python +_15693_ERRORS = { + 0x01: "not supported", + 0x02: "not recognized", + 0x0F: "unknown error", + 0x10: "block not available", + 0x11: "block already locked", + 0x12: "block locked", + 0x13: "block not written", + 0x14: "block not locked", +} + + +def _decode_15693_response(payload: bytes) -> str | None: + if len(payload) < 1: + return None + + flags = payload[0] + + if flags & 0x01: # Error + if len(payload) >= 2: + code = payload[1] + desc = _15693_ERRORS.get(code, "") + if desc: + return f"ERROR 0x{code:02X} {desc}" + return f"ERROR 0x{code:02X}" + return "ERROR" + + # Success — try to identify the response type + data = payload[1:] + if len(data) == 0: + return "OK" + + # Inventory response: dsfid(1) + uid(8) = 9 bytes + if len(data) == 9: + uid_msb = bytes(reversed(data[1:9])) + return f"OK INVENTORY UID={uid_msb.hex().upper()}" + + # Generic data response + return f"OK [{len(data)}B]" +``` + +**Step 4: Run tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestDecode15693Response -v` +Expected: all PASS + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/sim/trace_fmt.py tests/test_sim_trace_fmt.py +git commit --no-gpg-sign -m "feat(trace): add ISO 15693 response decoder" +``` + +--- + +### Task 3: ISO 14443-A decoder + +**Files:** +- Modify: `pm3py/sim/trace_fmt.py` +- Test: `tests/test_sim_trace_fmt.py` + +**Step 1: Write failing tests** + +Append to test file: + +```python +from pm3py.sim.trace_fmt import decode_14443a + + +class TestDecode14443aRequest: + """Decode reader->tag 14443-A commands.""" + + def test_reqa(self): + assert decode_14443a(0, bytes([0x26])) == "REQA" + + def test_wupa(self): + assert decode_14443a(0, bytes([0x52])) == "WUPA" + + def test_hlta(self): + assert decode_14443a(0, bytes([0x50, 0x00])) == "HLTA" + + def test_anticol_cl1(self): + assert decode_14443a(0, bytes([0x93, 0x20])) == "ANTICOL CL1" + + def test_select_cl1(self): + payload = bytes([0x93, 0x70]) + b"\x01\x02\x03\x04\x04" + assert decode_14443a(0, payload) == "SELECT CL1" + + def test_anticol_cl2(self): + assert decode_14443a(0, bytes([0x95, 0x20])) == "ANTICOL CL2" + + def test_select_cl2(self): + payload = bytes([0x95, 0x70]) + b"\x01\x02\x03\x04\x04" + assert decode_14443a(0, payload) == "SELECT CL2" + + def test_anticol_cl3(self): + assert decode_14443a(0, bytes([0x97, 0x20])) == "ANTICOL CL3" + + def test_select_cl3(self): + payload = bytes([0x97, 0x70]) + b"\x01\x02\x03\x04\x04" + assert decode_14443a(0, payload) == "SELECT CL3" + + def test_rats(self): + assert decode_14443a(0, bytes([0xE0, 0x50])) == "RATS" + + def test_iblock_even(self): + assert decode_14443a(0, bytes([0x02, 0x00, 0xA4])) == "I-BLOCK(0)" + + def test_iblock_odd(self): + assert decode_14443a(0, bytes([0x03, 0x00, 0xA4])) == "I-BLOCK(1)" + + def test_rack(self): + assert decode_14443a(0, bytes([0xA2])) == "R-ACK(0)" + assert decode_14443a(0, bytes([0xA3])) == "R-ACK(1)" + + def test_rnak(self): + assert decode_14443a(0, bytes([0xB2])) == "R-NAK(0)" + assert decode_14443a(0, bytes([0xB3])) == "R-NAK(1)" + + def test_deselect(self): + assert decode_14443a(0, bytes([0xC2])) == "S(DESELECT)" + + def test_wtx(self): + assert decode_14443a(0, bytes([0xF2, 0x01])) == "S(WTX)" + + def test_unknown(self): + assert decode_14443a(0, bytes([0xFF])) is None + + def test_empty(self): + assert decode_14443a(0, b"") is None + + +class TestDecode14443aResponse: + """Decode tag->reader 14443-A responses.""" + + def test_atqa(self): + # 2-byte response to REQA/WUPA + assert decode_14443a(1, bytes([0x04, 0x00])) == "ATQA 04 00" + + def test_sak(self): + # 1-byte response to SELECT + assert decode_14443a(1, bytes([0x20])) == "SAK 20" + + def test_ats(self): + # ATS: first byte is length + ats = bytes([0x05, 0x78, 0x80, 0x70, 0x02]) + assert decode_14443a(1, ats) == "ATS [5]" + + def test_iblock_response(self): + assert decode_14443a(1, bytes([0x02, 0x90, 0x00])) == "I-BLOCK(0)" + + def test_empty(self): + assert decode_14443a(1, b"") is None +``` + +**Step 2: Run to verify failure** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestDecode14443aRequest tests/test_sim_trace_fmt.py::TestDecode14443aResponse -v` +Expected: FAIL — `ImportError: cannot import name 'decode_14443a'` + +**Step 3: Implement 14443-A decoder** + +Add to `trace_fmt.py`: + +```python +# ---- ISO 14443-A constants ---- +_14A_CL_MAP = {0x93: "CL1", 0x95: "CL2", 0x97: "CL3"} + + +def decode_14443a(direction: int, payload: bytes) -> str | None: + """Decode an ISO 14443-A frame into a human-readable annotation.""" + if not payload: + return None + if direction == 0: + return _decode_14443a_request(payload) + else: + return _decode_14443a_response(payload) + + +def _decode_14443a_request(payload: bytes) -> str | None: + b0 = payload[0] + + # Short frames (single byte) + if b0 == 0x26: + return "REQA" + if b0 == 0x52: + return "WUPA" + + # HLTA + if b0 == 0x50 and len(payload) >= 2: + return "HLTA" + + # Anticollision / Select + if b0 in _14A_CL_MAP and len(payload) >= 2: + cl = _14A_CL_MAP[b0] + nvb = payload[1] + if nvb == 0x20: + return f"ANTICOL {cl}" + if nvb == 0x70: + return f"SELECT {cl}" + return f"ANTICOL {cl} nvb={nvb:02X}" + + # RATS + if b0 == 0xE0: + return "RATS" + + # ISO-DEP I-block + if b0 & 0xE2 == 0x02: + bn = b0 & 0x01 + return f"I-BLOCK({bn})" + + # R-ACK + if b0 & 0xF6 == 0xA2: + bn = b0 & 0x01 + return f"R-ACK({bn})" + + # R-NAK + if b0 & 0xF6 == 0xB2: + bn = b0 & 0x01 + return f"R-NAK({bn})" + + # S(DESELECT) + if b0 == 0xC2: + return "S(DESELECT)" + + # S(WTX) + if b0 == 0xF2: + return "S(WTX)" + + return None + + +def _decode_14443a_response(payload: bytes) -> str | None: + if not payload: + return None + + b0 = payload[0] + + # I-block response + if b0 & 0xE2 == 0x02: + bn = b0 & 0x01 + return f"I-BLOCK({bn})" + + # ATQA (2 bytes) + if len(payload) == 2 and b0 & 0xF0 == 0x00: + return f"ATQA {payload[0]:02X} {payload[1]:02X}" + + # SAK (1 byte) + if len(payload) == 1: + return f"SAK {b0:02X}" + + # ATS (first byte = length, length >= 2) + if len(payload) >= 2 and payload[0] == len(payload): + return f"ATS [{len(payload)}]" + + return None +``` + +**Step 4: Run tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py -v` +Expected: all PASS (15693 + 14443-A) + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/sim/trace_fmt.py tests/test_sim_trace_fmt.py +git commit --no-gpg-sign -m "feat(trace): add ISO 14443-A decoder" +``` + +--- + +### Task 4: TraceFormatter — color, wrapping, terminal width + +**Files:** +- Modify: `pm3py/sim/trace_fmt.py` +- Test: `tests/test_sim_trace_fmt.py` + +**Step 1: Write failing tests** + +Append to test file: + +```python +from pm3py.sim.trace_fmt import TraceFormatter + + +class TestTraceFormatterBasic: + """Test formatting output (colors stripped for assertion).""" + + def _strip_ansi(self, s: str) -> str: + import re + return re.sub(r'\033\[[0-9;]*m', '', s) + + def test_starts_with_newline(self): + fmt = TraceFormatter(mode="sim") + result = fmt.format(0, bytes([0x26, 0x01, 0x00])) + assert result.startswith("\n") + + def test_sim_mode_tag(self): + fmt = TraceFormatter(mode="sim") + result = self._strip_ansi(fmt.format(0, bytes([0x26, 0x01, 0x00]))) + assert "[Sim]" in result + + def test_reader_mode_tag(self): + fmt = TraceFormatter(mode="reader") + result = self._strip_ansi(fmt.format(0, bytes([0x26, 0x01, 0x00]))) + assert "[Rdr]" in result + + def test_sniff_mode_tag(self): + fmt = TraceFormatter(mode="sniff") + result = self._strip_ansi(fmt.format(0, bytes([0x26, 0x01, 0x00]))) + assert "[Snf]" in result + + def test_reader_to_tag_arrow(self): + fmt = TraceFormatter(mode="sim") + result = self._strip_ansi(fmt.format(0, bytes([0x26, 0x01, 0x00]))) + assert "Reader \u2192 Tag:" in result + + def test_tag_to_reader_arrow(self): + fmt = TraceFormatter(mode="sim") + result = self._strip_ansi(fmt.format(1, bytes([0x00]))) + assert "Tag \u2192 Reader:" in result + + def test_hex_space_separated(self): + fmt = TraceFormatter(mode="sim") + result = self._strip_ansi(fmt.format(0, bytes([0x22, 0x20, 0x03]))) + assert "22 20 03" in result + + def test_annotation_present(self): + fmt = TraceFormatter(mode="sim") + result = self._strip_ansi(fmt.format(0, bytes([0x26, 0x01, 0x00]))) + assert "INVENTORY" in result + + def test_no_annotation_for_unknown(self): + fmt = TraceFormatter(mode="sim") + result = self._strip_ansi(fmt.format(0, bytes([0x22, 0xFF]))) + assert "22 FF" in result or "22 ff" in result.lower() + + +class TestTraceFormatterWrapping: + """Test column-aligned wrapping for long payloads.""" + + def _strip_ansi(self, s: str) -> str: + import re + return re.sub(r'\033\[[0-9;]*m', '', s) + + def test_short_payload_inline_annotation(self): + """Short payload: annotation on same line as hex.""" + fmt = TraceFormatter(mode="sim", width=80) + result = self._strip_ansi(fmt.format(0, bytes([0x26, 0x01, 0x00]))) + lines = result.strip().split("\n") + assert len(lines) == 1 + assert "INVENTORY" in lines[0] + assert "26 01 00" in lines[0] + + def test_long_payload_wraps(self): + """Long payload wraps at prefix column.""" + fmt = TraceFormatter(mode="sim", width=60) + long_payload = bytes([0x00]) + bytes(40) + result = self._strip_ansi(fmt.format(1, long_payload)) + lines = result.strip().split("\n") + assert len(lines) > 1 + # Continuation lines should be indented to prefix column + first_hex_col = lines[0].index("00") + for line in lines[1:]: + if line.strip(): + leading = len(line) - len(line.lstrip()) + assert leading >= first_hex_col - 1 + + def test_wrapped_annotation_on_own_line(self): + """When hex wraps, annotation goes on its own line.""" + fmt = TraceFormatter(mode="sim", width=50) + # Inventory response with UID — long enough to wrap at width=50 + uid_lsb = bytes([0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x04, 0xE0]) + payload = bytes([0x00, 0x00]) + uid_lsb + result = self._strip_ansi(fmt.format(1, payload)) + lines = result.strip().split("\n") + # Annotation should be on a separate line + annotation_lines = [l for l in lines if "INVENTORY" in l] + hex_lines = [l for l in lines if "00 01" in l.lower() or "04 e0" in l.lower()] + if len(hex_lines) > 1: + # Wrapped — annotation must be on its own line + assert len(annotation_lines) == 1 + assert annotation_lines[0] not in hex_lines + + +class TestTraceFormatterNoColor: + """Colors suppressed when is_tty=False.""" + + def test_no_ansi_when_not_tty(self): + fmt = TraceFormatter(mode="sim", is_tty=False) + result = fmt.format(0, bytes([0x26, 0x01, 0x00])) + assert "\033[" not in result +``` + +**Step 2: Run to verify failure** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestTraceFormatterBasic tests/test_sim_trace_fmt.py::TestTraceFormatterWrapping tests/test_sim_trace_fmt.py::TestTraceFormatterNoColor -v` +Expected: FAIL — `ImportError: cannot import name 'TraceFormatter'` + +**Step 3: Implement TraceFormatter** + +Add to `trace_fmt.py`: + +```python +import os +import signal +import sys + + +# ---- ANSI colors ---- +_C_CYAN = "\033[36m" +_C_YELLOW = "\033[33m" +_C_MAGENTA = "\033[35m" +_C_GREEN = "\033[32m" +_C_DIM = "\033[2m" +_C_RESET = "\033[0m" + +_MODE_TAGS = { + "sim": ("[Sim]", _C_MAGENTA), + "reader": ("[Rdr]", _C_CYAN), + "sniff": ("[Snf]", ""), +} + + +class TraceFormatter: + """Colored, decoded, column-wrapped trace output. + + Args: + mode: "sim", "reader", or "sniff" + protocol: "15693" or "14443a" (selects decoder) + width: override terminal width (None = auto-detect) + is_tty: override TTY detection (None = auto-detect) + """ + + def __init__(self, mode: str = "sim", protocol: str = "15693", + width: int | None = None, is_tty: bool | None = None): + self._mode = mode + self._protocol = protocol + self._width = width or self._detect_width() + self._is_tty = is_tty if is_tty is not None else sys.stdout.isatty() + self._line_count = 0 + + # Try to register SIGWINCH for dynamic resize + if width is None: + try: + signal.signal(signal.SIGWINCH, self._on_resize) + except (OSError, ValueError): + pass # not main thread or not Unix + + def _detect_width(self) -> int: + try: + return os.get_terminal_size().columns + except (OSError, ValueError): + return 80 + + def _on_resize(self, signum, frame): + self._width = self._detect_width() + + def _color(self, code: str, text: str) -> str: + if not self._is_tty or not code: + return text + return f"{code}{text}{_C_RESET}" + + def format(self, direction: int, payload: bytes) -> str: + """Format a trace line with colors, decoding, and wrapping.""" + # Re-check width periodically if no SIGWINCH + self._line_count += 1 + if self._line_count % 10 == 0: + try: + self._width = self._detect_width() + except Exception: + pass + + # Mode tag + mode_tag, mode_color = _MODE_TAGS.get(self._mode, ("[???]", "")) + + # Direction + if direction == 0: + arrow = "Reader \u2192 Tag:" + dir_color = _C_CYAN + else: + arrow = "Tag \u2192 Reader:" + dir_color = _C_YELLOW + + # Build prefix (uncolored for width calc) + prefix_plain = f"{mode_tag} {arrow} " + prefix_len = len(prefix_plain) + + # Colored prefix + prefix_colored = self._color(mode_color, mode_tag) + " " + self._color(dir_color, arrow) + " " + + # Hex bytes (space-separated, uppercase) + hex_str = " ".join(f"{b:02X}" for b in payload) + + # Decode annotation + if self._protocol == "15693": + annotation = decode_15693(direction, payload) + elif self._protocol == "14443a": + annotation = decode_14443a(direction, payload) + else: + annotation = None + + # Layout: determine if everything fits on one line + avail = self._width - prefix_len + if annotation: + one_line = f"{hex_str} {annotation}" + else: + one_line = hex_str + + if len(one_line) <= avail: + # Single line + hex_colored = self._color(_C_DIM, hex_str) + if annotation: + ann_colored = self._color(_C_GREEN, annotation) + line = f"{prefix_colored}{hex_colored} {ann_colored}" + else: + line = f"{prefix_colored}{hex_colored}" + return f"\n{line}" + + # Multi-line: wrap hex, annotation on its own line + pad = " " * prefix_len + hex_lines = self._wrap_hex(hex_str, avail) + parts = [f"{prefix_colored}{self._color(_C_DIM, hex_lines[0])}"] + for hl in hex_lines[1:]: + parts.append(f"{pad}{self._color(_C_DIM, hl)}") + if annotation: + parts.append(f"{pad}{self._color(_C_GREEN, annotation)}") + + return "\n" + "\n".join(parts) + + def _wrap_hex(self, hex_str: str, avail: int) -> list[str]: + """Wrap space-separated hex string into lines of at most `avail` chars.""" + tokens = hex_str.split(" ") + lines = [] + current = "" + for tok in tokens: + candidate = f"{current} {tok}" if current else tok + if len(candidate) <= avail: + current = candidate + else: + if current: + lines.append(current) + current = tok + if current: + lines.append(current) + return lines or [""] + + def print(self, direction: int, payload: bytes) -> None: + """Format and print a trace line to stdout.""" + sys.stdout.write(self.format(direction, payload)) + sys.stdout.flush() +``` + +**Step 4: Run tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py -v` +Expected: all PASS + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/sim/trace_fmt.py tests/test_sim_trace_fmt.py +git commit --no-gpg-sign -m "feat(trace): add TraceFormatter with color, wrapping, terminal resize" +``` + +--- + +### Task 5: Wire TraceFormatter into SimSession + +**Files:** +- Modify: `pm3py/sim/sim_session.py:1-2,126-175,177-224` + +**Step 1: Write failing test** + +Append to `tests/test_sim_trace_fmt.py`: + +```python +from unittest.mock import MagicMock, patch +import struct + +from pm3py.sim.sim_session import SimSession, CMD_HF_ISO15693_SIM_TRACE +from pm3py.transport import encode_ng_frame, RESP_PREAMBLE_MAGIC, RESP_PREAMBLE_SIZE, RESP_POSTAMBLE_SIZE + + +class TestSimSessionTrace: + """Verify SimSession uses TraceFormatter instead of raw print.""" + + def _make_trace_frame(self, direction: int, payload: bytes) -> bytes: + """Build a fake firmware trace response frame.""" + from pm3py.transport import RESP_POSTAMBLE_MAGIC + data = bytes([direction]) + payload + length = len(data) | 0x8000 # NG bit set + header = struct.pack("tag inventory + frame = self._make_trace_frame(0, bytes([0x26, 0x01, 0x00])) + + # Mock serial that returns frame then stops + mock_serial = MagicMock() + mock_serial.in_waiting = len(frame) + call_count = 0 + + def read_side_effect(n): + nonlocal call_count + call_count += 1 + if call_count == 1: + return frame + session._active = False + return b"" + + mock_serial.read.side_effect = read_side_effect + + session._trace_reader(mock_serial) + mock_fmt.print.assert_called_once_with(0, bytes([0x26, 0x01, 0x00])) +``` + +**Step 2: Run to verify failure** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py::TestSimSessionTrace -v` +Expected: FAIL — `ImportError` or `AttributeError` (no `TraceFormatter` import in sim_session) + +**Step 3: Modify sim_session.py** + +Add import at top of `sim_session.py`: + +```python +from .trace_fmt import TraceFormatter +``` + +In `start_15693()`, replace the trace thread creation (around line 167-171) to instantiate the formatter: + +```python + if trace: + self._formatter = TraceFormatter(mode="sim", protocol="15693") + self._trace_thread = threading.Thread( + target=self._trace_reader, args=(ser,), daemon=True + ) + self._trace_thread.start() +``` + +In `_trace_reader()`, replace line 216-217: + +```python + arrow = "Reader → Tag" if direction == 0 else "Tag → Reader" + print(f"[Trace] {arrow}: {payload.hex()}") +``` + +With: + +```python + self._formatter.print(direction, payload) +``` + +**Step 4: Run all tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_sim_trace_fmt.py -v` +Expected: all PASS + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/ -v` +Expected: all PASS (no regressions) + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/sim/sim_session.py pm3py/sim/trace_fmt.py tests/test_sim_trace_fmt.py +git commit --no-gpg-sign -m "feat(trace): wire TraceFormatter into SimSession" +``` diff --git a/docs/plans/2026-03-18-ndef-trace-decode.md b/docs/plans/2026-03-18-ndef-trace-decode.md new file mode 100644 index 0000000..d65949e --- /dev/null +++ b/docs/plans/2026-03-18-ndef-trace-decode.md @@ -0,0 +1,677 @@ +# NDEF Trace Decode Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Detect and decode NDEF content in ISO 15693 trace output — both sim trace and sniff — showing human-readable NDEF records in the annotation column, with overflow on the next line. + +**Architecture:** Add an `decode_ndef()` function that parses NDEF TLV + records from block data. Wire it into `decode_15693_response()` so READ SINGLE BLOCK and READ MULTIPLE BLOCKS responses that contain NDEF data show decoded content. CC (block 0) gets its own annotation. Overflow annotations wrap to a continuation line at the annotation column. + +**Tech Stack:** Python, pytest, existing hf_15.py decoder + sim trace_fmt.py + +**Working directory:** `/home/work/pm3py/.worktrees/sim-framework/` + +**Test command:** `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py tests/test_sim_trace_fmt.py -v` + +--- + +### Context for implementer + +The trace formatter produces lines like: + +``` +[Sim] Tag → Reader: 00 E1 40 0D 01 C1 C0 OK [4B] +[Sim] Tag → Reader: 00 03 0B D1 01 07 54 02 65 6E 74 65 73 74 FE 00 ... FD 02 OK [60B] +``` + +The response decoder (`decode_15693_response`) currently sees all success responses as either inventory (9B data), empty, or generic `OK [NB]`. We want to detect and decode: + +1. **CC block** (block 0): `E1 40 XX YY` → `CC v1.0 MLEN=XX MBREAD=Y` +2. **NDEF TLV**: `03 LL FE` → decode the NDEF records inside +3. **NDEF records**: Text → `NDEF Text "en" "hello"`, URI → `NDEF URI https://example.com`, Media → `NDEF MIME text/plain [5B]` + +When the annotation is longer than the available space, it wraps to a continuation line right-justified at the annotation column (same pattern the hex wrapping already uses). + +**Tricky parts:** + +- The decoder only sees the response payload (after flags byte), with no context about WHICH command produced it. It must detect CC/NDEF purely from the data pattern. +- READ MULTIPLE BLOCKS responses can contain block 0 (CC) followed by NDEF data. The CC is in the first 4 bytes, NDEF TLV starts at byte 4. +- READ SINGLE BLOCK #0 response is just 4 bytes of CC data. +- We need both `hf_15.py` (sniff decoder) and `sim/trace_fmt.py` (sim decoder) to share the NDEF decode logic. Put the shared code in `hf_15.py` since that's where the 15693 decoders live; the sim trace_fmt calls into it via `decode_15693`. + +**NFC Forum Type 5 CC format (4 bytes):** +- Byte 0: 0xE1 = NDEF magic +- Byte 1: version (upper nibble = major, lower = minor) + access bits +- Byte 2: MLEN (data area size / 8) +- Byte 3: feature flags (bit 0 = MBREAD) + +**NDEF TLV format:** +- Type 0x03 = NDEF message, 0xFE = terminator, 0x00 = NULL (skip) +- Length: 1 byte if < 0xFF, else 0xFF + 2 bytes big-endian +- Value: raw NDEF message bytes + +**NDEF record format:** +- Byte 0: flags (MB=0x80, ME=0x40, CF=0x20, SR=0x10, IL=0x08, TNF=0x07) +- Byte 1: TYPE_LENGTH +- If SR: byte 2 = PAYLOAD_LENGTH (1 byte), else bytes 2-5 = PAYLOAD_LENGTH (4 bytes BE) +- If IL: next byte = ID_LENGTH +- TYPE (TYPE_LENGTH bytes) +- ID (ID_LENGTH bytes, if IL set) +- PAYLOAD (PAYLOAD_LENGTH bytes) + +**Well-known RTD types:** +- "T" (0x54): Text record — payload[0] = lang_len, payload[1:1+lang_len] = lang, rest = text +- "U" (0x55): URI record — payload[0] = prefix code, rest = URI suffix + +--- + +### Task 1: NDEF TLV + record decoder + +**Files:** +- Modify: `pm3py/hf_15.py` +- Modify: `tests/test_hf_15.py` + +**Step 1: Write failing tests** + +Append to `tests/test_hf_15.py`: + +```python +from pm3py.hf_15 import decode_ndef_annotation + +# ---- NDEF decode ---- + +class TestDecodeNdefAnnotation: + def test_cc_block(self): + """4-byte CC recognized and decoded.""" + data = bytes([0xE1, 0x40, 0x0D, 0x01]) + ann = decode_ndef_annotation(data) + assert ann is not None + assert "CC" in ann + assert "v1.0" in ann + assert "MBREAD" in ann + + def test_cc_no_mbread(self): + data = bytes([0xE1, 0x40, 0x10, 0x00]) + ann = decode_ndef_annotation(data) + assert "CC" in ann + assert "MBREAD" not in ann + + def test_ndef_text_record(self): + """NDEF TLV with Text record.""" + # TLV: 03 0B FE + # Record: D1 01 07 54 02 65 6E 74 65 73 74 + # MB+ME+SR, TNF=well-known, type="T", lang="en", text="test" + msg = bytes([0xD1, 0x01, 0x07, 0x54, 0x02, 0x65, 0x6E, + 0x74, 0x65, 0x73, 0x74]) + tlv = bytes([0x03, len(msg)]) + msg + bytes([0xFE]) + ann = decode_ndef_annotation(tlv) + assert ann is not None + assert '"test"' in ann + + def test_ndef_uri_record(self): + """NDEF TLV with URI record.""" + # URI: https://example.com → prefix 0x04 + "example.com" + uri_payload = bytes([0x04]) + b"example.com" + rec = bytes([0xD1, 0x01, len(uri_payload), 0x55]) + uri_payload + tlv = bytes([0x03, len(rec)]) + rec + bytes([0xFE]) + ann = decode_ndef_annotation(tlv) + assert "https://example.com" in ann + + def test_ndef_mime_record(self): + """NDEF TLV with MIME record.""" + mime_type = b"text/plain" + payload = b"hello" + rec = bytes([0xD2, len(mime_type), len(payload)]) + mime_type + payload + tlv = bytes([0x03, len(rec)]) + rec + bytes([0xFE]) + ann = decode_ndef_annotation(tlv) + assert "text/plain" in ann + + def test_no_ndef_magic(self): + """Random data without NDEF TLV returns None.""" + ann = decode_ndef_annotation(bytes([0x00, 0x00, 0x00, 0x00])) + assert ann is None + + def test_cc_plus_ndef(self): + """Data starting with CC followed by NDEF TLV (READ MULTIPLE from block 0).""" + cc = bytes([0xE1, 0x40, 0x0D, 0x01]) + msg = bytes([0xD1, 0x01, 0x07, 0x54, 0x02, 0x65, 0x6E, + 0x74, 0x65, 0x73, 0x74]) + tlv = bytes([0x03, len(msg)]) + msg + bytes([0xFE]) + data = cc + tlv + bytes(50) # pad like real response + ann = decode_ndef_annotation(data) + assert "CC" in ann + assert '"test"' in ann + + def test_empty_ndef(self): + """NDEF TLV with zero length.""" + tlv = bytes([0x03, 0x00, 0xFE]) + ann = decode_ndef_annotation(tlv) + assert ann is not None + assert "empty" in ann.lower() or "0B" in ann or "NDEF" in ann + + def test_long_text_truncated(self): + """Long text is truncated in annotation.""" + long_text = "A" * 200 + msg = ndef_text(long_text) + tlv = bytes([0x03, 0xFF, (len(msg) >> 8) & 0xFF, len(msg) & 0xFF]) + msg + bytes([0xFE]) + ann = decode_ndef_annotation(tlv) + assert ann is not None + assert "..." in ann # truncated +``` + +**Step 2: Run tests to see them fail** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py::TestDecodeNdefAnnotation -v` +Expected: FAIL — `ImportError: cannot import name 'decode_ndef_annotation'` + +**Step 3: Implement decode_ndef_annotation** + +Add to `pm3py/hf_15.py`, after the existing `URI_PREFIXES` dict (copy it from type5.py or import it), before the `_15693_CMDS` dict: + +```python +# ---- NDEF decode (for trace annotation) ---- + +# NFC Type 5 CC magic +_NDEF_MAGIC = 0xE1 + +# URI prefix codes (NFC Forum RTD URI) +_URI_PREFIXES = { + 0x00: "", + 0x01: "http://www.", + 0x02: "https://www.", + 0x03: "http://", + 0x04: "https://", + 0x05: "tel:", + 0x06: "mailto:", +} + +# TNF values +_TNF_EMPTY = 0x00 +_TNF_WELL_KNOWN = 0x01 +_TNF_MEDIA = 0x02 +_TNF_URI = 0x03 +_TNF_EXTERNAL = 0x04 + +_MAX_ANN_TEXT = 40 # truncate decoded text in annotations + + +def decode_ndef_annotation(data: bytes) -> str | None: + """Decode NDEF content from block data for trace annotation. + + Handles: + - CC block (4 bytes starting with 0xE1) + - NDEF TLV (starting with 0x03) + - CC + NDEF TLV (READ MULTIPLE from block 0) + + Returns annotation string or None if not NDEF. + """ + if len(data) < 4: + return None + + parts = [] + offset = 0 + + # Check for CC at start + if data[0] == _NDEF_MAGIC: + ver_major = (data[1] >> 6) & 0x03 + ver_minor = (data[1] >> 4) & 0x03 + mlen = data[2] + features = data[3] + cc_str = f"CC v{ver_major}.{ver_minor} MLEN={mlen}" + if features & 0x01: + cc_str += " MBREAD" + parts.append(cc_str) + offset = 4 + if offset >= len(data): + return cc_str + + # Scan for NDEF TLV + while offset < len(data): + tlv_type = data[offset] + offset += 1 + + if tlv_type == 0x00: # NULL TLV — skip + continue + if tlv_type == 0xFE: # Terminator + break + if tlv_type != 0x03: # Not NDEF — stop + break + + # Parse TLV length + if offset >= len(data): + break + tlv_len = data[offset] + offset += 1 + if tlv_len == 0xFF: + if offset + 2 > len(data): + break + tlv_len = (data[offset] << 8) | data[offset + 1] + offset += 2 + + if tlv_len == 0: + parts.append("NDEF empty") + continue + + # Parse NDEF message (one or more records) + msg_end = min(offset + tlv_len, len(data)) + records = _parse_ndef_records(data[offset:msg_end]) + for rec in records: + parts.append(rec) + offset = msg_end + + return " | ".join(parts) if parts else None + + +def _parse_ndef_records(msg: bytes) -> list[str]: + """Parse NDEF records from raw message bytes. Returns list of annotation strings.""" + records = [] + offset = 0 + + while offset < len(msg): + if offset + 3 > len(msg): + break + + flags = msg[offset] + tnf = flags & 0x07 + sr = bool(flags & 0x10) + il = bool(flags & 0x08) + type_len = msg[offset + 1] + offset += 2 + + # Payload length + if sr: + if offset >= len(msg): + break + payload_len = msg[offset] + offset += 1 + else: + if offset + 4 > len(msg): + break + payload_len = int.from_bytes(msg[offset:offset + 4], "big") + offset += 4 + + # ID length + id_len = 0 + if il: + if offset >= len(msg): + break + id_len = msg[offset] + offset += 1 + + # Type + if offset + type_len > len(msg): + break + rec_type = msg[offset:offset + type_len] + offset += type_len + + # ID (skip) + offset += id_len + + # Payload + if offset + payload_len > len(msg): + payload = msg[offset:] + else: + payload = msg[offset:offset + payload_len] + offset += payload_len + + # Decode based on TNF + type + ann = _decode_ndef_record(tnf, rec_type, payload) + records.append(ann) + + return records + + +def _decode_ndef_record(tnf: int, rec_type: bytes, payload: bytes) -> str: + """Decode a single NDEF record into an annotation string.""" + if tnf == _TNF_WELL_KNOWN: + if rec_type == b"T" and len(payload) >= 1: + lang_len = payload[0] & 0x3F + lang = payload[1:1 + lang_len].decode("ascii", errors="replace") + text = payload[1 + lang_len:].decode("utf-8", errors="replace") + if len(text) > _MAX_ANN_TEXT: + text = text[:_MAX_ANN_TEXT] + "..." + return f'NDEF Text "{lang}" "{text}"' + if rec_type == b"U" and len(payload) >= 1: + prefix = _URI_PREFIXES.get(payload[0], "") + suffix = payload[1:].decode("utf-8", errors="replace") + uri = prefix + suffix + if len(uri) > _MAX_ANN_TEXT: + uri = uri[:_MAX_ANN_TEXT] + "..." + return f"NDEF URI {uri}" + return f"NDEF WK type={rec_type!r} [{len(payload)}B]" + + if tnf == _TNF_MEDIA: + mime = rec_type.decode("ascii", errors="replace") + if len(mime) > 30: + mime = mime[:30] + "..." + return f"NDEF MIME {mime} [{len(payload)}B]" + + if tnf == _TNF_EXTERNAL: + ext_type = rec_type.decode("ascii", errors="replace") + return f"NDEF EXT {ext_type} [{len(payload)}B]" + + if tnf == _TNF_EMPTY: + return "NDEF empty" + + return f"NDEF TNF={tnf} [{len(payload)}B]" +``` + +**Step 4: Run tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py::TestDecodeNdefAnnotation -v` +Expected: All pass + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/hf_15.py tests/test_hf_15.py +git commit --no-gpg-sign -m "feat: NDEF TLV + record decoder for trace annotations" +``` + +--- + +### Task 2: Wire NDEF decode into response annotations + +**Files:** +- Modify: `pm3py/hf_15.py` +- Modify: `tests/test_hf_15.py` + +**Step 1: Write failing tests** + +Append to `tests/test_hf_15.py`: + +```python +class TestResponseNdefDecode: + def test_read_single_block_0_cc(self): + """READ SINGLE BLOCK #0 response shows CC annotation.""" + # Response: flags(1) + block_data(4) + data = bytes([0x00, 0xE1, 0x40, 0x0D, 0x01]) + ann = decode_15693_response(data) + assert "CC" in ann + assert "MBREAD" in ann + + def test_read_multiple_blocks_ndef(self): + """READ MULTIPLE BLOCKS response with NDEF shows decoded text.""" + # Response: flags(1) + blocks data + msg = bytes([0xD1, 0x01, 0x07, 0x54, 0x02, 0x65, 0x6E, + 0x74, 0x65, 0x73, 0x74]) + tlv = bytes([0x03, len(msg)]) + msg + bytes([0xFE]) + data = bytes([0x00]) + tlv + bytes(50) + ann = decode_15693_response(data) + assert '"test"' in ann + + def test_read_multiple_with_cc_and_ndef(self): + """READ MULTIPLE from block 0: CC + NDEF both decoded.""" + cc = bytes([0xE1, 0x40, 0x0D, 0x01]) + msg = bytes([0xD1, 0x01, 0x07, 0x54, 0x02, 0x65, 0x6E, + 0x74, 0x65, 0x73, 0x74]) + tlv = bytes([0x03, len(msg)]) + msg + bytes([0xFE]) + data = bytes([0x00]) + cc + tlv + bytes(40) + ann = decode_15693_response(data) + assert "CC" in ann + assert '"test"' in ann + + def test_generic_data_no_ndef(self): + """Non-NDEF data still shows OK [NB].""" + data = bytes([0x00, 0xDE, 0xAD, 0xBE, 0xEF]) + ann = decode_15693_response(data) + assert "OK [4B]" in ann + + def test_inventory_not_affected(self): + """Inventory response still decoded as inventory, not NDEF.""" + data = bytes([0x00, 0x00]) + bytes(8) + ann = decode_15693_response(data) + assert "INVENTORY" in ann +``` + +**Step 2: Run tests to see them fail** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py::TestResponseNdefDecode -v` +Expected: FAIL — `"CC"` not in `"OK [4B]"` + +**Step 3: Update decode_15693_response** + +Replace the generic response handler section in `decode_15693_response` (the part after error check, starting at `data = payload[1:]`): + +```python +def decode_15693_response(payload: bytes) -> str | None: + """Decode an ISO 15693 response frame into annotation.""" + if len(payload) < 1: + return None + + flags = payload[0] + + if flags & 0x01: # Error + if len(payload) >= 2: + code = payload[1] + desc = _15693_ERRORS.get(code, "") + return f"ERROR 0x{code:02X} {desc}" if desc else f"ERROR 0x{code:02X}" + return "ERROR" + + data = payload[1:] + if len(data) == 0: + return "OK" + + # Inventory response: dsfid(1) + uid(8) = 9 bytes + if len(data) == 9: + uid_msb = bytes(reversed(data[1:9])) + return f"OK INVENTORY UID={uid_msb.hex().upper()}" + + # Try NDEF decode (CC block, NDEF TLV, or both) + ndef_ann = decode_ndef_annotation(data) + if ndef_ann: + return f"OK {ndef_ann}" + + return f"OK [{len(data)}B]" +``` + +**Step 4: Run tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py -v` +Expected: All pass (including existing tests — verify `test_decode_response_ok_data` still passes since it uses non-NDEF data) + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/hf_15.py tests/test_hf_15.py +git commit --no-gpg-sign -m "feat: wire NDEF decode into ISO 15693 response annotations" +``` + +--- + +### Task 3: Annotation overflow wrapping + +**Files:** +- Modify: `pm3py/hf_15.py` (sniff formatter) +- Modify: `pm3py/sim/trace_fmt.py` (sim formatter) +- Modify: `tests/test_hf_15.py` +- Modify: `tests/test_sim_trace_fmt.py` + +The current formatters already wrap hex to multiple lines when it's too wide. But annotations are either right-justified on the last hex line (short) or on their own line (wrapped hex). NDEF annotations can be long (e.g. `OK CC v1.0 MLEN=13 MBREAD | NDEF Text "en" "test"`). When the annotation overflows the available width, it should wrap to the next line at the annotation column. + +**Step 1: Write failing tests** + +In `tests/test_hf_15.py`, append: + +```python +class TestSniffLineAnnotationOverflow: + def test_long_annotation_wraps(self): + """Long NDEF annotation wraps to continuation line.""" + # Build a response with CC + NDEF text that produces a long annotation + cc = bytes([0xE1, 0x40, 0x0D, 0x01]) + long_text = "A" * 30 + from pm3py.sim.type5 import ndef_text + msg = ndef_text(long_text) + tlv = bytes([0x03, len(msg)]) + msg + bytes([0xFE]) + payload = bytes([0x00]) + cc + tlv + # Add CRC + payload += bytes([0xAA, 0xBB]) + + entry = {"is_response": True, "data": payload, + "timestamp": 0, "duration": 0, "parity": b""} + line = format_sniff_line(entry, width=80, is_tty=False) + lines = line.split("\n") + # Annotation should be on a separate continuation line + ann_lines = [l for l in lines if "NDEF" in l] + assert len(ann_lines) >= 1 + + def test_short_annotation_no_wrap(self): + """Short annotation stays on same line as hex.""" + data = bytes([0x00, 0xE1, 0x40, 0x0D, 0x01, 0xAA, 0xBB]) + entry = {"is_response": True, "data": data, + "timestamp": 0, "duration": 0, "parity": b""} + line = format_sniff_line(entry, width=120, is_tty=False) + assert "CC" in line + # Should be single-line (plus possible hex wrap) + non_empty = [l for l in line.split("\n") if l.strip()] + assert len(non_empty) <= 2 # hex + annotation on same or next line +``` + +In `tests/test_sim_trace_fmt.py`, append to `TestTraceFormatterWrapping`: + +```python + def test_long_annotation_wraps_to_next_line(self): + """Long NDEF annotation wraps to continuation line.""" + # Build response with CC + NDEF that gives a long annotation + cc = bytes([0xE1, 0x40, 0x0D, 0x01]) + from pm3py.sim.type5 import ndef_text + msg = ndef_text("A" * 30) + tlv = bytes([0x03, len(msg)]) + msg + bytes([0xFE]) + payload = bytes([0x00]) + cc + tlv + fmt = TraceFormatter(mode="sim", decoder=decode_15693, width=80) + result = self._strip_ansi(fmt.format(1, payload)) + lines = result.strip().split("\n") + ann_lines = [l for l in lines if "NDEF" in l] + assert len(ann_lines) >= 1 +``` + +**Step 2: Run tests to see them fail** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py::TestSniffLineAnnotationOverflow tests/test_sim_trace_fmt.py::TestTraceFormatterWrapping::test_long_annotation_wraps_to_next_line -v` +Expected: Might pass or fail depending on current wrapping behavior — the annotation might already go on its own line if hex is wrapped. Run to check. + +**Step 3: Update annotation wrapping in both formatters** + +In `format_sniff_line()` in `hf_15.py`, replace the annotation handling in the multi-line section (the `if annotation:` block near the end): + +```python + if annotation: + # Wrap annotation if it exceeds available width + ann_lines = _wrap_annotation(annotation, avail) + for al in ann_lines: + ann_pad = max(avail - len(al), 0) + parts.append(f"{pad}{' ' * ann_pad}{_color(ann_color, al, is_tty)}") +``` + +And add the helper (shared between both formatters): + +```python +def _wrap_annotation(annotation: str, avail: int) -> list[str]: + """Wrap annotation text to fit within avail columns. + + Splits on ' | ' boundaries first, then by word if still too long. + """ + if len(annotation) <= avail: + return [annotation] + + # Try splitting on pipe separator (NDEF uses " | " between parts) + if " | " in annotation: + parts = annotation.split(" | ") + lines = [] + current = parts[0] + for part in parts[1:]: + candidate = f"{current} | {part}" + if len(candidate) <= avail: + current = candidate + else: + lines.append(current) + current = part + lines.append(current) + return lines + + # Fall back to truncation + return [annotation[:avail - 3] + "..."] +``` + +In `trace_fmt.py`, apply the same pattern to the `TraceFormatter.format()` method's multi-line annotation section. Import `_wrap_annotation` from `hf_15` or duplicate the logic inline: + +In the multi-line section of `TraceFormatter.format()`, replace: + +```python + if annotation: + ann_pad = avail - len(annotation) + ann_pad = max(ann_pad, 0) + parts.append(f"{pad}{' ' * ann_pad}{self._color(ann_color, annotation)}") +``` + +With: + +```python + if annotation: + ann_lines = self._wrap_annotation(annotation, avail) + for al in ann_lines: + ann_pad = max(avail - len(al), 0) + parts.append(f"{pad}{' ' * ann_pad}{self._color(ann_color, al)}") +``` + +And add to `TraceFormatter`: + +```python + def _wrap_annotation(self, annotation: str, avail: int) -> list[str]: + """Wrap annotation to fit within avail columns.""" + if len(annotation) <= avail: + return [annotation] + if " | " in annotation: + parts = annotation.split(" | ") + lines = [] + current = parts[0] + for part in parts[1:]: + candidate = f"{current} | {part}" + if len(candidate) <= avail: + current = candidate + else: + lines.append(current) + current = part + lines.append(current) + return lines + return [annotation[:avail - 3] + "..."] +``` + +**Step 4: Run all tests** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/test_hf_15.py tests/test_sim_trace_fmt.py -v` +Expected: All pass + +**Step 5: Commit** + +```bash +cd /home/work/pm3py/.worktrees/sim-framework +git add pm3py/hf_15.py pm3py/sim/trace_fmt.py tests/test_hf_15.py tests/test_sim_trace_fmt.py +git commit --no-gpg-sign -m "feat: annotation overflow wrapping for NDEF decode in trace output" +``` + +--- + +### Task 4: Copy hf_15.py to main branch + final test + +**Files:** +- Copy: `pm3py/hf_15.py` → `/home/work/pm3py/pm3py/hf_15.py` +- Copy: `tests/test_hf_15.py` → `/home/work/pm3py/tests/test_hf_15.py` + +**Step 1: Copy files** + +```bash +cp /home/work/pm3py/.worktrees/sim-framework/pm3py/hf_15.py /home/work/pm3py/pm3py/hf_15.py +cp /home/work/pm3py/.worktrees/sim-framework/tests/test_hf_15.py /home/work/pm3py/tests/test_hf_15.py +``` + +**Step 2: Run main branch tests** + +Run: `cd /home/work/pm3py && python -m pytest tests/ -v` +Expected: All pass + +**Step 3: Run worktree full test suite** + +Run: `cd /home/work/pm3py/.worktrees/sim-framework && python -m pytest tests/ -v` +Expected: All pass diff --git a/docs/plans/2026-03-19-live-sniff-design.md b/docs/plans/2026-03-19-live-sniff-design.md new file mode 100644 index 0000000..4aa88d8 --- /dev/null +++ b/docs/plans/2026-03-19-live-sniff-design.md @@ -0,0 +1,333 @@ +# Live Sniff Streaming — Design + +## Overview + +Rewrite sniff workflow from batch (sniff → button stop → download → decode) to live +streaming with button-toggle pause/resume. User types `session.start_15693()` in the +REPL, sees trace output in real-time, toggles sniffing with the PM3 button, and +accumulates entries across pause/resume cycles. + +Three changes: +1. **Auto-load** — `.pythonstartup.py` creates a `session` object in sniff mode +2. **Live streaming** — firmware flushes trace entries over USB on RF idle +3. **Button-toggle** — firmware pause/resume within a persistent sniff session + +## Protocol + +### New command IDs (`pm3_cmd.h`) + +| Command | Value | Direction | Purpose | +|---------|-------|-----------|---------| +| `CMD_HF_SNIFF_STREAM` | 0x0337 | FW→Host | Fire-and-forget trace entry (like `SIM_TRACE`) | +| `CMD_HF_SNIFF_STATUS` | 0x0338 | FW→Host | State change: started / paused / resumed / stopped | + +### Sniff command payload + +Sent with existing `CMD_HF_ISO15693_SNIFF` (and future `CMD_HF_ISO14443A_SNIFF`). +Legacy behavior preserved when `flags=0`. + +```c +struct sniff_params_t { + uint16_t idle_timeout_ms; // flush threshold (default 400ms for 15693) + uint8_t flags; // bit 0: live streaming (1=stream, 0=legacy BigBuf) + // bit 1: button-toggle (1=pause/resume, 0=exit on press) +} PACKED; +``` + +### Stream trace payload (`CMD_HF_SNIFF_STREAM`) + +```c +struct sniff_trace_t { + uint8_t protocol; // 0=15693, 1=14443A + uint8_t direction; // 0=reader->tag, 1=tag->reader + uint16_t duration; // frame duration ticks + uint32_t timestamp; // us since sniff start + uint8_t data[]; // frame bytes (variable length) +} PACKED; +``` + +### Status payload (`CMD_HF_SNIFF_STATUS`) + +```c +struct sniff_status_t { + uint8_t state; // 0=started, 1=paused, 2=resumed, 3=stopped + uint16_t count; // total frames captured so far +} PACKED; +``` + +## Firmware changes + +### `iso15693.c` — `SniffIso15693()` + +Current function signature gains parameters from the new payload. When +`flags & SNIFF_FLAG_STREAMING` is unset, existing behavior is unchanged. + +#### Streaming + button-toggle flow + +``` +SniffIso15693(idle_timeout_ms, flags): + if (!(flags & SNIFF_FLAG_STREAMING)): + // existing behavior, unchanged + return + + // --- streaming mode --- + ring_buf[TRACE_RING_SIZE] // 8-entry ring (same pattern as sim) + trace_rd = trace_wr = 0 + state = SNIFF_ACTIVE + frame_count = 0 + reply_ng(CMD_HF_SNIFF_STATUS, {state=0, count=0}) // "started" + + while (state != SNIFF_STOPPED): + + // --- button toggle --- + if (BUTTON_PRESS()): + debounce(200ms) + if (state == SNIFF_ACTIVE): + TRACE_FLUSH() + state = SNIFF_PAUSED + reply_ng(CMD_HF_SNIFF_STATUS, {state=1, count=frame_count}) + LED_C_OFF() + else: + state = SNIFF_ACTIVE + reply_ng(CMD_HF_SNIFF_STATUS, {state=2, count=frame_count}) + LED_C_ON() + + // --- paused: spin --- + if (state == SNIFF_PAUSED): + WDT_HIT() + // check for BREAK_LOOP (below) + goto check_break + + // --- existing DMA decode logic --- + // on decoded frame: + TRACE_PUSH(direction, data, len, timestamp, duration) + frame_count++ + last_activity = GetTickCount() + + // --- idle flush --- + if (trace_wr != trace_rd && + (GetTickCount() - last_activity) > idle_timeout_ms): + TRACE_FLUSH() + + check_break: + // --- Python stop --- + if (data_available()): + receive_ng(&rx) + if (rx.cmd == CMD_BREAK_LOOP): + TRACE_FLUSH() + state = SNIFF_STOPPED + + reply_ng(CMD_HF_SNIFF_STATUS, {state=3, count=frame_count}) +``` + +#### TRACE_PUSH / TRACE_FLUSH macros + +Reuse the sim pattern. Each `TRACE_FLUSH` entry sent as: + +```c +reply_ng(CMD_HF_SNIFF_STREAM, PM3_SUCCESS, &sniff_trace_t, sizeof(hdr) + data_len); +``` + +Payload is `sniff_trace_t` with protocol=0 (15693), direction, duration, timestamp, +and the raw frame bytes. + +#### LED feedback + +| State | LED | +|-------|-----| +| Sniffing active | Green ON (Easy=A, RDV4=B) | +| Frame captured | Orange blink (Easy=C, RDV4=A) | +| Paused | Green OFF | + +#### Ring buffer sizing + +8 entries. At 15693 speeds (~10-20 frames/sec during active reader polling), +8 entries buffer ~400-800ms of traffic. Flush on idle empties before overflow. + +### `pm3_cmd.h` — New definitions + +```c +#define CMD_HF_SNIFF_STREAM 0x0337 +#define CMD_HF_SNIFF_STATUS 0x0338 + +#define SNIFF_FLAG_STREAMING 0x01 +#define SNIFF_FLAG_BUTTON_TOGGLE 0x02 + +#define SNIFF_STATE_STARTED 0 +#define SNIFF_STATE_PAUSED 1 +#define SNIFF_STATE_RESUMED 2 +#define SNIFF_STATE_STOPPED 3 +``` + +## Python changes + +### `sniff/session.py` — `SniffSession` rewrite + +Persistent session with background reader thread, mirroring `SimSession`. + +```python +class SniffSession: + def __init__(self, ser): + self._ser = ser # raw pyserial (like SimSession) + self._active = False + self._entries = [] # accumulated parsed trace entries + self._reader_thread = None + self._formatter = TraceFormatter() + self._state = "idle" # idle / sniffing / paused / stopped + + @classmethod + def open(cls, port=None, baudrate=115200): + """Auto-detect PM3 port and open serial connection.""" + if port is None: + port = find_pm3_port() + ser = serial.Serial(port, baudrate, timeout=0.1) + return cls(ser) + + def start_15693(self, live=True, idle_timeout_ms=400): + """Start 15693 sniff session. + + live=True: streaming mode with button-toggle pause/resume (default) + live=False: legacy BigBuf mode, button exits + """ + self._protocol = 0 + flags = 0 + if live: + flags |= SNIFF_FLAG_STREAMING | SNIFF_FLAG_BUTTON_TOGGLE + payload = struct.pack("> Started +[Snf] Reader -> Tag: 26 01 00 INVENTORY +[Snf] Tag -> Reader: 00 E0 04 ... INVENTORY RESPONSE +[Snf] || Paused (12 frames) +[Snf] >> Resumed (12 frames) +[Snf] [] Stopped (28 frames) +``` + +### `.pythonstartup.py` — Auto-load session + +In `sniff` mode (and `all` mode), auto-create session: + +```python +if _MODE in ("sniff", "all"): + from pm3py.sniff.session import SniffSession + # ... other sniff imports ... + + try: + session = SniffSession.open() + print("Sniff session ready -- session.start_15693()") + except Exception as e: + print(f"Sniff session: {e}") +``` + +No `pm3` client created in sniff mode — `session` owns the serial port exclusively, +same pattern as sim mode. + +## Protocol-agnostic design + +The `sniff_trace_t.protocol` field and `sniff_params_t` payload structure are shared +across protocols. Only 15693 is implemented now. To add 14443-A later: + +1. Parse `sniff_params_t` in `SniffIso14443a()` +2. Use same `CMD_HF_SNIFF_STREAM` / `CMD_HF_SNIFF_STATUS` commands +3. Set `protocol=1` in trace entries +4. Python `_stream_reader` dispatches to `decode_14a()` based on protocol field +5. Add `session.start_14a(live=True, idle_timeout_ms=100)` — 14443-A is faster, + shorter idle timeout appropriate + +## Idle timeout rationale + +15693 default: **400ms**. A typical inventory+read cycle completes in ~20-30ms. Reader +poll interval is 50-100ms. 400ms = 4 missed polls = reader is definitely gone. Short +enough that trace appears almost instantly after removing the device. + +Configurable via `idle_timeout_ms` parameter for different reader behaviors. + +## Legacy compatibility + +`live=False` preserves existing behavior exactly: +- `flags=0` in payload → firmware takes legacy path (unchanged code) +- Python blocks, waits for button exit, downloads BigBuf, decodes batch +- No background thread, no streaming + +## REPL workflow + +``` +$ source activate sniff +$ python +Sniff session ready -- session.start_15693() + +>>> session.start_15693() +[Snf] >> Started + # hold phone to antenna... +[Snf] Reader -> Tag: 26 01 00 INVENTORY +[Snf] Tag -> Reader: 00 E0 04 ... INVENTORY RESPONSE +[Snf] Reader -> Tag: 22 20 04 ... READ SINGLE BLOCK blk=0 +[Snf] Tag -> Reader: 00 E1 10 06 01 ... READ SINGLE BLOCK RESPONSE + # pull phone away, 400ms later trace flushes + # press button to pause +[Snf] || Paused (12 frames) + # examine data in REPL +>>> session.entries[-1] +{'timestamp': 42318, 'direction': 1, 'data': b'\x00\xe1\x10...', ...} + # press button to resume for more samples +[Snf] >> Resumed (12 frames) + # done +>>> session.stop() +[Snf] [] Stopped (28 frames) +>>> len(session.entries) +28 +``` diff --git a/docs/plans/2026-03-19-live-sniff-plan.md b/docs/plans/2026-03-19-live-sniff-plan.md new file mode 100644 index 0000000..3c546a9 --- /dev/null +++ b/docs/plans/2026-03-19-live-sniff-plan.md @@ -0,0 +1,1108 @@ +# Live Sniff Streaming Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Rewrite sniff from batch download to live-streaming with button-toggle pause/resume, auto-loaded in the REPL. + +**Architecture:** Firmware sends trace entries as fire-and-forget `reply_ng()` packets on RF idle (reusing sim's ring buffer pattern). Python `SniffSession` receives in a background thread (mirroring `SimSession._trace_reader`). `.pythonstartup.py` auto-creates `session` in sniff mode. + +**Tech Stack:** C (firmware ARM), Python 3.12 (pyserial, threading), pytest with mocking + +**Design doc:** `docs/plans/2026-03-19-live-sniff-design.md` + +--- + +### Task 1: Firmware — Add command IDs and structs + +**Files:** +- Modify: `firmware/include/pm3_cmd.h:734` (after CMD_HF_ISO15693_SIM_TRACE) +- Modify: `pm3py/core/protocol.py:203` (after HF_ISO15693_EML_SETMEM) + +**Step 1: Add firmware command definitions** + +In `firmware/include/pm3_cmd.h`, after line 734 (`CMD_HF_ISO15693_SIM_TRACE`), add: + +```c +#define CMD_HF_SNIFF_STREAM 0x0337 // FW->Host: live sniff trace entry +#define CMD_HF_SNIFF_STATUS 0x0338 // FW->Host: sniff state change + +// Sniff streaming flags (sent in sniff command payload) +#define SNIFF_FLAG_STREAMING 0x01 +#define SNIFF_FLAG_BUTTON_TOGGLE 0x02 + +// Sniff state values (in CMD_HF_SNIFF_STATUS payload) +#define SNIFF_STATE_STARTED 0 +#define SNIFF_STATE_PAUSED 1 +#define SNIFF_STATE_RESUMED 2 +#define SNIFF_STATE_STOPPED 3 +``` + +**Step 2: Add Python Cmd enum entries** + +In `pm3py/core/protocol.py`, after `HF_ISO15693_EML_SETMEM = 0x0331` (line 203), add: + +```python + HF_SNIFF_STREAM = 0x0337 + HF_SNIFF_STATUS = 0x0338 +``` + +Also add constants after the `Cmd` class: + +```python +# Sniff streaming flags +SNIFF_FLAG_STREAMING = 0x01 +SNIFF_FLAG_BUTTON_TOGGLE = 0x02 + +# Sniff states +SNIFF_STATE_STARTED = 0 +SNIFF_STATE_PAUSED = 1 +SNIFF_STATE_RESUMED = 2 +SNIFF_STATE_STOPPED = 3 +``` + +**Step 3: Commit** + +```bash +git add firmware/include/pm3_cmd.h pm3py/core/protocol.py +git commit --no-gpg-sign -m "feat: add CMD_HF_SNIFF_STREAM/STATUS command IDs and constants" +``` + +--- + +### Task 2: Firmware — Streaming sniff loop + +**Files:** +- Modify: `firmware/armsrc/iso15693.c:1654-1875` (SniffIso15693) +- Modify: `firmware/armsrc/appmain.c:1500-1503` (CMD_HF_ISO15693_SNIFF handler) + +**Step 1: Modify appmain.c to pass payload to SniffIso15693** + +Currently (line 1500-1503): +```c +case CMD_HF_ISO15693_SNIFF: { + SniffIso15693(0, NULL, false); + reply_ng(CMD_HF_ISO15693_SNIFF, PM3_SUCCESS, NULL, 0); + break; +} +``` + +Change to: +```c +case CMD_HF_ISO15693_SNIFF: { + uint16_t idle_timeout_ms = 0; + uint8_t sniff_flags = 0; + if (packet->length >= 3) { + idle_timeout_ms = packet->data.asBytes[0] | (packet->data.asBytes[1] << 8); + sniff_flags = packet->data.asBytes[2]; + } + SniffIso15693(0, NULL, false, idle_timeout_ms, sniff_flags); + if (!(sniff_flags & SNIFF_FLAG_STREAMING)) { + // Legacy mode: send completion response (streaming mode sends STATUS) + reply_ng(CMD_HF_ISO15693_SNIFF, PM3_SUCCESS, NULL, 0); + } + break; +} +``` + +**Step 2: Update SniffIso15693 signature and add streaming mode** + +In `firmware/armsrc/iso15693.c`, change the function signature (line 1654): + +```c +void SniffIso15693(uint8_t jam_search_len, uint8_t *jam_search_string, bool iclass, + uint16_t idle_timeout_ms, uint8_t sniff_flags) { +``` + +After the existing init code (after line 1698, the DMA setup), add the streaming mode branch. The key change: when `sniff_flags & SNIFF_FLAG_STREAMING`, use a ring buffer + idle flush loop instead of LogTrace to BigBuf. + +Insert after the DMA setup succeeds (after line 1697), before the state variables (line 1699): + +```c + // --- Streaming mode --- + if (sniff_flags & SNIFF_FLAG_STREAMING) { + + // Ring buffer (same pattern as SimTagIso15693) + #define SNIFF_TRACE_ENTRIES 8 + #define SNIFF_TRACE_ENTRY_MAX 72 // 8-byte header + 64 data + static uint8_t sniff_ring[SNIFF_TRACE_ENTRIES][SNIFF_TRACE_ENTRY_MAX]; + static uint16_t sniff_ring_len[SNIFF_TRACE_ENTRIES]; + uint8_t sniff_wr = 0, sniff_rd = 0; + uint16_t frame_count = 0; + uint32_t last_activity = 0; + bool sniff_paused = false; + + // Macro: push trace entry with full metadata + // Layout: [protocol(1) | direction(1) | duration(2) | timestamp(4) | data(*)] + #define SNIFF_PUSH(dir, data_ptr, data_len, ts, dur) do { \ + if (((sniff_wr + 1) % SNIFF_TRACE_ENTRIES) != sniff_rd) { \ + uint8_t *_e = sniff_ring[sniff_wr]; \ + _e[0] = 0; /* protocol: 0=15693 */ \ + _e[1] = (dir); \ + _e[2] = (dur) & 0xFF; _e[3] = ((dur) >> 8) & 0xFF; \ + _e[4] = (ts) & 0xFF; _e[5] = ((ts) >> 8) & 0xFF; \ + _e[6] = ((ts) >> 16) & 0xFF; _e[7] = ((ts) >> 24) & 0xFF; \ + uint16_t _dl = (data_len); \ + if (_dl > SNIFF_TRACE_ENTRY_MAX - 8) _dl = SNIFF_TRACE_ENTRY_MAX - 8; \ + memcpy(_e + 8, (data_ptr), _dl); \ + sniff_ring_len[sniff_wr] = 8 + _dl; \ + sniff_wr = (sniff_wr + 1) % SNIFF_TRACE_ENTRIES; \ + } \ + } while(0) + + #define SNIFF_FLUSH() do { \ + while (sniff_rd != sniff_wr) { \ + reply_ng(CMD_HF_SNIFF_STREAM, PM3_SUCCESS, \ + sniff_ring[sniff_rd], sniff_ring_len[sniff_rd]); \ + sniff_rd = (sniff_rd + 1) % SNIFF_TRACE_ENTRIES; \ + } \ + } while(0) + + // Send started status + uint8_t status_buf[3] = { SNIFF_STATE_STARTED, 0, 0 }; + reply_ng(CMD_HF_SNIFF_STATUS, PM3_SUCCESS, status_buf, 3); + + LED_A_ON(); // green = sniffing active + + // --- Streaming sniff main loop --- + // (This duplicates the DMA decode logic from the legacy path below, + // but replaces LogTrace calls with SNIFF_PUSH and adds idle flush + + // button toggle + BREAK_LOOP check.) + + bool tag_is_active = false; + bool reader_is_active = false; + bool expect_tag_answer = false; + bool expect_fsk_answer = false; + bool expect_fast_answer = true; + int dma_start_time = 0; + int samples = 0; + const uint16_t *upTo = dma->buf; + bool exit_sniff = false; + + while (!exit_sniff) { + + volatile int behind_by = ((uint16_t *)AT91C_BASE_PDC_SSC->PDC_RPR - upTo) & (DMA_BUFFER_SIZE - 1); + if (behind_by < 1) { + // No new DMA data — check idle flush, button, break + if (!sniff_paused && sniff_wr != sniff_rd && last_activity > 0) { + uint32_t now = GetTickCount(); + if ((now - last_activity) > idle_timeout_ms) { + SNIFF_FLUSH(); + } + } + + // Check for Python BREAK_LOOP + if (data_available()) { + PacketCommandNG rx; + if (receive_ng(&rx) == PM3_SUCCESS && rx.cmd == CMD_BREAK_LOOP) { + SNIFF_FLUSH(); + exit_sniff = true; + } + } + continue; + } + + samples++; + if (samples == 1) { + dma_start_time = GetCountSspClk() & 0xfffffff0; + } + + volatile uint16_t sniffdata = 0; + volatile uint16_t sniffdata_prev = sniffdata; + sniffdata = *upTo++; + + if (upTo >= dma->buf + DMA_BUFFER_SIZE) { + upTo = dma->buf; + + if (AT91C_BASE_SSC->SSC_SR & (AT91C_SSC_ENDRX)) { + if (AT91C_BASE_PDC_SSC->PDC_RCR == 0) { + AT91C_BASE_PDC_SSC->PDC_RPR = (uint32_t) dma->buf; + AT91C_BASE_PDC_SSC->PDC_RCR = DMA_BUFFER_SIZE; + } + if (AT91C_BASE_PDC_SSC->PDC_RNCR == 0) { + AT91C_BASE_PDC_SSC->PDC_RNPR = (uint32_t) dma->buf; + AT91C_BASE_PDC_SSC->PDC_RNCR = DMA_BUFFER_SIZE; + } + + WDT_HIT(); + + // Button toggle + if (BUTTON_PRESS()) { + SpinDelay(200); // debounce + while (BUTTON_PRESS()) { WDT_HIT(); } // wait for release + SpinDelay(200); // debounce release + + if (sniff_paused) { + sniff_paused = false; + LED_A_ON(); + status_buf[0] = SNIFF_STATE_RESUMED; + status_buf[1] = frame_count & 0xFF; + status_buf[2] = (frame_count >> 8) & 0xFF; + reply_ng(CMD_HF_SNIFF_STATUS, PM3_SUCCESS, status_buf, 3); + } else { + SNIFF_FLUSH(); + sniff_paused = true; + LED_A_OFF(); + status_buf[0] = SNIFF_STATE_PAUSED; + status_buf[1] = frame_count & 0xFF; + status_buf[2] = (frame_count >> 8) & 0xFF; + reply_ng(CMD_HF_SNIFF_STATUS, PM3_SUCCESS, status_buf, 3); + } + } + + // Check for Python BREAK_LOOP + if (data_available()) { + PacketCommandNG rx; + if (receive_ng(&rx) == PM3_SUCCESS && rx.cmd == CMD_BREAK_LOOP) { + SNIFF_FLUSH(); + exit_sniff = true; + } + } + } + } + + if (sniff_paused) continue; + + // --- Reader decode (same as legacy) --- + if (tag_is_active == false) { + int extra_8s = 1; + if (Handle15693SampleFromReader((sniffdata & 0x02) >> 1, &dreader) || + (++extra_8s && Handle15693SampleFromReader(sniffdata & 0x01, &dreader))) { + + if (dreader.byteCount > 0) { + uint32_t eof_time = dma_start_time + (samples * 16) + (extra_8s * 8) - DELAY_READER_TO_ARM_SNIFF; + uint32_t sof_time = eof_time + - dreader.byteCount * (dreader.Coding == CODING_1_OUT_OF_4 ? 1024 : 16384) + - 256 - 128; + + uint32_t ts_us = (sof_time * 4) / 14; // ssp_clk*4 -> ~us + uint16_t dur = (uint16_t)(((eof_time - sof_time) * 4) / 14); + SNIFF_PUSH(0, dreader.output, dreader.byteCount, ts_us, dur); + frame_count++; + last_activity = GetTickCount(); + + if (iclass == false) { + expect_fsk_answer = dreader.output[0] & ISO15_REQ_SUBCARRIER_TWO; + expect_fast_answer = dreader.output[0] & ISO15_REQ_DATARATE_HIGH; + } + } + + DecodeTagReset(&dtag); + DecodeTagFSKReset(&dtagfsk); + reader_is_active = false; + expect_tag_answer = true; + } else { + reader_is_active = (dreader.state >= STATE_READER_RECEIVE_DATA_1_OUT_OF_4); + } + } + + // --- Tag decode: single subcarrier (same as legacy) --- + if ((reader_is_active == false) && expect_tag_answer) { + if (expect_fsk_answer == false) { + if (Handle15693SamplesFromTag((sniffdata >> 4) << 2, &dtag, expect_fast_answer)) { + uint32_t eof_time = dma_start_time + (samples * 16) - DELAY_TAG_TO_ARM_SNIFF; + if (dtag.lastBit == SOF_PART2) { + eof_time -= (8 * 16); + } + uint32_t sof_time = eof_time + - dtag.len * 1024 - 512 + - (dtag.lastBit != SOF_PART2 ? 512 : 0); + + uint32_t ts_us = (sof_time * 4) / 14; + uint16_t dur = (uint16_t)(((eof_time - sof_time) * 4) / 14); + SNIFF_PUSH(1, dtag.output, dtag.len, ts_us, dur); + frame_count++; + last_activity = GetTickCount(); + + DecodeTagReset(&dtag); + DecodeTagFSKReset(&dtagfsk); + DecodeReaderReset(&dreader); + expect_tag_answer = false; + tag_is_active = false; + } else { + tag_is_active = (dtag.state >= STATE_TAG_RECEIVING_DATA); + } + + } else { + // --- Tag decode: dual subcarrier FSK (same as legacy) --- + if (FREQ_IS_0((sniffdata >> 2) & 0x3)) { + sniffdata = sniffdata_prev; + } + if (Handle15693FSKSamplesFromTag((sniffdata >> 2) & 0x3, &dtagfsk, expect_fast_answer)) { + if (dtagfsk.len > 0) { + uint32_t eof_time = dma_start_time + (samples * 16) - DELAY_TAG_TO_ARM_SNIFF; + if (dtagfsk.lastBit == SOF) { + eof_time -= (8 * 16); + } + uint32_t sof_time = eof_time + - dtagfsk.len * 1016 - 512 + - (dtagfsk.lastBit != SOF ? 512 : 0); + + uint32_t ts_us = (sof_time * 4) / 14; + uint16_t dur = (uint16_t)(((eof_time - sof_time) * 4) / 14); + SNIFF_PUSH(1, dtagfsk.output, dtagfsk.len, ts_us, dur); + frame_count++; + last_activity = GetTickCount(); + } + + DecodeTagFSKReset(&dtagfsk); + DecodeReaderReset(&dreader); + expect_tag_answer = false; + tag_is_active = false; + expect_fsk_answer = false; + } else { + tag_is_active = (dtagfsk.state >= STATE_FSK_RECEIVING_DATA_484); + } + } + } + } + + // Cleanup and send stopped status + FpgaDisableTracing(); + switch_off(); + status_buf[0] = SNIFF_STATE_STOPPED; + status_buf[1] = frame_count & 0xFF; + status_buf[2] = (frame_count >> 8) & 0xFF; + reply_ng(CMD_HF_SNIFF_STATUS, PM3_SUCCESS, status_buf, 3); + return; + } + + // --- Legacy mode (existing code, unchanged from here) --- +``` + +Also update the function declaration in `iso15693.h` (if it exists) or any forward declarations. + +**Step 3: Update any forward declarations** + +Search for `SniffIso15693` declarations and update the signature to include the two new parameters. + +**Step 4: Commit** + +```bash +git add firmware/armsrc/iso15693.c firmware/armsrc/appmain.c +# Also add any header files modified +git commit --no-gpg-sign -m "feat(fw): streaming sniff with ring buffer, button-toggle, idle flush" +``` + +--- + +### Task 3: Python — SniffSession rewrite with streaming + +**Files:** +- Modify: `pm3py/sniff/session.py` (full rewrite) +- Reference: `pm3py/sim/sim_session.py:54-83` (open pattern), `pm3py/sim/sim_session.py:237-290` (trace reader pattern) + +**Step 1: Write tests for the new SniffSession** + +Create `tests/test_sniff_session.py`: + +```python +"""Tests for SniffSession — live streaming sniff with button-toggle.""" +import struct +import threading +import time +from unittest.mock import MagicMock, patch, PropertyMock + +import pytest + +from pm3py.core.protocol import ( + Cmd, SNIFF_FLAG_STREAMING, SNIFF_FLAG_BUTTON_TOGGLE, + SNIFF_STATE_STARTED, SNIFF_STATE_PAUSED, + SNIFF_STATE_RESUMED, SNIFF_STATE_STOPPED, +) +from pm3py.core.transport import ( + encode_ng_frame, RESP_PREAMBLE_MAGIC, RESP_POSTAMBLE_NOCRC, +) +from pm3py.sniff.session import SniffSession + + +def _make_resp_frame(cmd: int, status: int, data: bytes) -> bytes: + """Build a raw NG response frame (USB, no CRC).""" + payload_len = len(data) + preamble = struct.pack(" bytes: + """Build a CMD_HF_SNIFF_STREAM response frame.""" + payload = struct.pack(" bytes: + """Build a CMD_HF_SNIFF_STATUS response frame.""" + payload = struct.pack("= len(self._buf): + time.sleep(0.01) # avoid busy spin + return b"" + end = min(self._pos + size, len(self._buf)) + data = bytes(self._buf[self._pos:end]) + self._pos = end + return data + + def write(self, data): + return len(data) + + def reset_input_buffer(self): + pass + + def reset_output_buffer(self): + pass + + def close(self): + self._closed = True + + +class TestSniffSessionStreaming: + """Tests for live streaming mode.""" + + def test_start_15693_sends_streaming_flags(self): + """start_15693(live=True) sends correct payload.""" + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_status_frame(SNIFF_STATE_STOPPED, 0), + ]) + sent = bytearray() + orig_write = ser.write + def capture_write(data): + sent.extend(data) + return orig_write(data) + ser.write = capture_write + + session = SniffSession(ser) + session.start_15693(live=True, idle_timeout_ms=400) + time.sleep(0.1) + session.stop() + + # Verify payload contains flags + assert len(sent) > 0 + # The payload should contain idle_timeout_ms (400=0x0190) and flags (0x03) + assert b"\x90\x01\x03" in bytes(sent) + + def test_start_15693_legacy_mode(self): + """start_15693(live=False) sends zero flags.""" + ser = FakeSerial([ + _make_resp_frame(Cmd.HF_ISO15693_SNIFF.value, 0, b""), + ]) + sent = bytearray() + orig_write = ser.write + def capture_write(data): + sent.extend(data) + return orig_write(data) + ser.write = capture_write + + session = SniffSession(ser) + session.start_15693(live=False) + # Legacy should block and return + + def test_receives_trace_entries(self): + """Background thread decodes CMD_HF_SNIFF_STREAM frames.""" + reader_cmd = bytes([0x26, 0x01, 0x00]) # INVENTORY + tag_resp = bytes([0x00, 0xE0, 0x04]) + + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_sniff_stream_frame(0, 0, 100, 1000, reader_cmd), + _make_sniff_stream_frame(0, 1, 200, 2000, tag_resp), + _make_status_frame(SNIFF_STATE_STOPPED, 2), + ]) + + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.3) # let background thread process + + assert len(session.entries) == 2 + assert session.entries[0]["direction"] == 0 + assert session.entries[0]["data"] == reader_cmd + assert session.entries[0]["timestamp"] == 1000 + assert session.entries[1]["direction"] == 1 + assert session.entries[1]["data"] == tag_resp + + def test_status_updates(self): + """Status frames update session state.""" + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_sniff_stream_frame(0, 0, 100, 1000, b"\x26\x01\x00"), + _make_status_frame(SNIFF_STATE_PAUSED, 1), + _make_status_frame(SNIFF_STATE_RESUMED, 1), + _make_status_frame(SNIFF_STATE_STOPPED, 1), + ]) + + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.3) + + assert session.state == "stopped" + + def test_stop_sends_break_loop(self): + """stop() sends CMD_BREAK_LOOP to firmware.""" + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_status_frame(SNIFF_STATE_STOPPED, 0), + ]) + sent = bytearray() + orig_write = ser.write + def capture_write(data): + sent.extend(data) + return orig_write(data) + ser.write = capture_write + + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.1) + session.stop() + + # BREAK_LOOP = 0x0118 + assert b"\x18\x01" in bytes(sent) + + def test_entries_accumulate_across_cycles(self): + """Entries persist across pause/resume cycles.""" + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_sniff_stream_frame(0, 0, 100, 1000, b"\x26\x01\x00"), + _make_status_frame(SNIFF_STATE_PAUSED, 1), + _make_status_frame(SNIFF_STATE_RESUMED, 1), + _make_sniff_stream_frame(0, 0, 100, 3000, b"\x26\x01\x00"), + _make_status_frame(SNIFF_STATE_STOPPED, 2), + ]) + + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.3) + + assert len(session.entries) == 2 + + def test_clear_entries(self): + """clear() resets accumulated entries.""" + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_sniff_stream_frame(0, 0, 100, 1000, b"\x26\x01\x00"), + _make_status_frame(SNIFF_STATE_STOPPED, 1), + ]) + + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.3) + + assert len(session.entries) == 1 + session.clear() + assert len(session.entries) == 0 +``` + +**Step 2: Run tests to verify they fail** + +```bash +cd /home/work/pm3py && python -m pytest tests/test_sniff_session.py -v +``` + +Expected: ImportError / AttributeError — new constants and SniffSession API don't exist yet. + +**Step 3: Implement SniffSession** + +Rewrite `pm3py/sniff/session.py`: + +```python +"""Sniff sessions — live streaming with button-toggle, or legacy batch.""" +import struct +import sys +import threading +import time + +import serial + +from ..core.protocol import ( + Cmd, PM3_CMD_DATA_SIZE, + SNIFF_FLAG_STREAMING, SNIFF_FLAG_BUTTON_TOGGLE, + SNIFF_STATE_STARTED, SNIFF_STATE_PAUSED, + SNIFF_STATE_RESUMED, SNIFF_STATE_STOPPED, +) +from ..core.transport import ( + encode_ng_frame, decode_response_frame, + RESP_PREAMBLE_MAGIC, RESP_PREAMBLE_SIZE, RESP_POSTAMBLE_SIZE, +) +from ..trace import parse_tracelog +from ..trace.decode_iso15 import decode_15693 +from ..trace.format import TraceFormatter, format_sniff_line + +# Default serial port (same as SimSession) +DEFAULT_PORT = "/dev/ttyACM0" + +_STATE_NAMES = { + SNIFF_STATE_STARTED: "sniffing", + SNIFF_STATE_PAUSED: "paused", + SNIFF_STATE_RESUMED: "sniffing", + SNIFF_STATE_STOPPED: "stopped", +} + +_STATE_ICONS = { + SNIFF_STATE_STARTED: ">>", + SNIFF_STATE_PAUSED: "||", + SNIFF_STATE_RESUMED: ">>", + SNIFF_STATE_STOPPED: "[]", +} + +_STATE_LABELS = { + SNIFF_STATE_STARTED: "Started", + SNIFF_STATE_PAUSED: "Paused", + SNIFF_STATE_RESUMED: "Resumed", + SNIFF_STATE_STOPPED: "Stopped", +} + + +class SniffSession: + """Persistent sniff session with live streaming and button-toggle. + + For live mode (default): + session = SniffSession.open() + session.start_15693() # streams trace live, button toggles pause + session.stop() # or let button cycles accumulate + session.entries # all captured frames + + For legacy mode: + session.start_15693(live=False) # blocks until button, batch download + """ + + def __init__(self, ser): + self._ser = ser + self._active = False + self._entries: list[dict] = [] + self._reader_thread: threading.Thread | None = None + self._formatter: TraceFormatter | None = None + self._state = "idle" + + @classmethod + def open(cls, port: str = DEFAULT_PORT, baudrate: int = 115200) -> "SniffSession": + """Open serial connection to PM3. Auto-detects port if default.""" + ser = serial.Serial() + ser.port = port + ser.baudrate = baudrate + ser.bytesize = 8 + ser.parity = "N" + ser.stopbits = 1 + ser.timeout = 0.1 + ser.xonxoff = False + ser.rtscts = False + ser.dsrdtr = False + ser.open() + + ser.reset_input_buffer() + ser.reset_output_buffer() + time.sleep(0.5) + ser.reset_input_buffer() + + return cls(ser) + + def start_15693(self, live: bool = True, idle_timeout_ms: int = 400) -> None: + """Start 15693 sniff session. + + live=True (default): streaming with button-toggle pause/resume. + live=False: legacy BigBuf batch mode, blocks until button exits. + """ + self._protocol = 0 # 15693 + flags = 0 + if live: + flags = SNIFF_FLAG_STREAMING | SNIFF_FLAG_BUTTON_TOGGLE + + payload = struct.pack(" None: + """Background thread: read and print trace frames live.""" + buf = bytearray() + while self._active: + try: + chunk = self._ser.read(self._ser.in_waiting or 1) + if not chunk: + continue + buf.extend(chunk) + + while self._try_decode_frame(buf): + pass + + except serial.SerialException: + break + except Exception: + continue + + def _try_decode_frame(self, buf: bytearray) -> bool: + """Decode one response frame from buf. Returns True if consumed.""" + if len(buf) < RESP_PREAMBLE_SIZE + RESP_POSTAMBLE_SIZE: + return False + + idx = buf.find(struct.pack(" 0: + del buf[:idx] + + if len(buf) < RESP_PREAMBLE_SIZE: + return False + + length_ng = struct.unpack_from(" None: + """Parse and print a trace entry from CMD_HF_SNIFF_STREAM.""" + if len(data) < 8: + return + + protocol, direction, duration, timestamp = struct.unpack_from(" None: + """Handle a CMD_HF_SNIFF_STATUS state change.""" + if len(data) < 3: + return + + state, count = struct.unpack_from(" 0 else "" + + msg = f"\n[Snf] {icon} {label}{count_str}\n" + sys.stdout.write(msg) + sys.stdout.flush() + + if state == SNIFF_STATE_STOPPED: + self._active = False + + def _legacy_sniff(self) -> None: + """Legacy batch mode: block until button, download BigBuf, decode.""" + from ..core.transport import PM3Transport + + print("[Snf] Sniffing ISO 15693... press PM3 button to stop.") + + # Read frames until we get the sniff completion response + buf = bytearray() + while True: + chunk = self._ser.read(self._ser.in_waiting or 1) + if chunk: + buf.extend(chunk) + + idx = buf.find(struct.pack(" 0: + buf = buf[idx:] + + if len(buf) < RESP_PREAMBLE_SIZE: + continue + + length_ng = struct.unpack_from(" None: + """End sniff session from Python (sends BREAK_LOOP).""" + if self._active: + frame = encode_ng_frame(Cmd.BREAK_LOOP, b"") + self._ser.write(frame) + if self._reader_thread: + self._reader_thread.join(timeout=2.0) + self._active = False + self._state = "stopped" + + def close(self) -> None: + """Stop and close serial port.""" + if self._active: + self.stop() + self._ser.close() + + @property + def state(self) -> str: + """Current session state: idle, sniffing, paused, stopped.""" + return self._state + + @property + def entries(self) -> list[dict]: + """All captured trace entries across pause/resume cycles.""" + return list(self._entries) + + def clear(self) -> None: + """Clear accumulated entries.""" + self._entries.clear() +``` + +**Step 4: Run tests** + +```bash +cd /home/work/pm3py && python -m pytest tests/test_sniff_session.py -v +``` + +Expected: All pass. + +**Step 5: Commit** + +```bash +git add pm3py/sniff/session.py tests/test_sniff_session.py pm3py/core/protocol.py +git commit --no-gpg-sign -m "feat: rewrite SniffSession with live streaming and button-toggle" +``` + +--- + +### Task 4: Python — Update .pythonstartup.py and sniff/__init__.py + +**Files:** +- Modify: `.pythonstartup.py:57-63` (SNIFF imports), `.pythonstartup.py:98-105` (auto-connect) +- Modify: `pm3py/sniff/__init__.py` + +**Step 1: Update .pythonstartup.py** + +In the SNIFF import group (lines 57-63), the SniffSession import stays. The key change is in the auto-connect section. + +After the existing `if _MODE == "core":` block (lines 98-105), add sniff mode auto-session: + +```python + if _MODE == "sniff": + try: + session = SniffSession.open() + print("Sniff session ready -- session.start_15693()") + except Exception as e: + print(f" (Sniff session failed: {e})") +``` + +**Step 2: Update sniff/__init__.py exports** + +No changes needed — `SniffSession` is already exported. The new constants are in `core/protocol.py`. + +**Step 3: Run existing tests to verify nothing broke** + +```bash +cd /home/work/pm3py && python -m pytest tests/ -v +``` + +Expected: All existing tests pass. + +**Step 4: Commit** + +```bash +git add .pythonstartup.py +git commit --no-gpg-sign -m "feat: auto-create sniff session in REPL sniff mode" +``` + +--- + +### Task 5: Verify end-to-end with mocked serial + +**Files:** +- Modify: `tests/test_sniff_session.py` (add integration-style test) + +**Step 1: Add end-to-end test simulating full REPL workflow** + +Add to `tests/test_sniff_session.py`: + +```python +class TestSniffSessionEndToEnd: + """Integration-style test: full start → stream → pause → resume → stop.""" + + def test_full_workflow(self): + """Simulate complete sniff session lifecycle.""" + inv_cmd = bytes([0x26, 0x01, 0x00]) + inv_resp = bytes([0x00, 0xE0, 0x04, 0x12, 0x34, 0x56, 0x78]) + rdbl_cmd = bytes([0x22, 0x20, 0x04]) + rdbl_resp = bytes([0x00, 0xE1, 0x10, 0x06, 0x01]) + + frames = [ + _make_status_frame(SNIFF_STATE_STARTED, 0), + # First burst + _make_sniff_stream_frame(0, 0, 100, 1000, inv_cmd), + _make_sniff_stream_frame(0, 1, 200, 1500, inv_resp), + _make_sniff_stream_frame(0, 0, 100, 2000, rdbl_cmd), + _make_sniff_stream_frame(0, 1, 200, 2500, rdbl_resp), + # Pause + _make_status_frame(SNIFF_STATE_PAUSED, 4), + # Resume + _make_status_frame(SNIFF_STATE_RESUMED, 4), + # Second burst + _make_sniff_stream_frame(0, 0, 100, 5000, inv_cmd), + _make_sniff_stream_frame(0, 1, 200, 5500, inv_resp), + # Stop + _make_status_frame(SNIFF_STATE_STOPPED, 6), + ] + + ser = FakeSerial(frames) + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.5) + + assert session.state == "stopped" + assert len(session.entries) == 6 + + # Verify timestamps are monotonically increasing + timestamps = [e["timestamp"] for e in session.entries] + assert timestamps == sorted(timestamps) + + # Verify directions alternate (reader/tag pairs) + directions = [e["direction"] for e in session.entries] + assert directions == [0, 1, 0, 1, 0, 1] + + def test_close_cleans_up(self): + """close() stops session and closes serial port.""" + ser = FakeSerial([ + _make_status_frame(SNIFF_STATE_STARTED, 0), + _make_status_frame(SNIFF_STATE_STOPPED, 0), + ]) + + session = SniffSession(ser) + session.start_15693(live=True) + time.sleep(0.1) + session.close() + + assert session.state == "stopped" + assert ser._closed +``` + +**Step 2: Run all tests** + +```bash +cd /home/work/pm3py && python -m pytest tests/test_sniff_session.py -v +``` + +Expected: All pass. + +**Step 3: Run full test suite** + +```bash +cd /home/work/pm3py && python -m pytest tests/ -v +``` + +Expected: All pass, no regressions. + +**Step 4: Commit** + +```bash +git add tests/test_sniff_session.py +git commit --no-gpg-sign -m "test: add end-to-end sniff session workflow tests" +``` + +--- + +### Task 6: Firmware build verification + +**Files:** +- No new files — verify firmware compiles + +**Step 1: Build firmware** + +```bash +cd /home/work/pm3py/firmware && make armsrc/obj/iso15693.o +``` + +Expected: Compiles without errors or warnings. If there are compile errors, fix them (likely missing includes or function signature mismatches). + +**Step 2: Check for any forward declaration mismatches** + +```bash +grep -rn "SniffIso15693" /home/work/pm3py/firmware/armsrc/*.h /home/work/pm3py/firmware/include/ +``` + +Update any declarations to match the new 5-parameter signature. + +**Step 3: Commit any fixes** + +```bash +git add firmware/ +git commit --no-gpg-sign -m "fix(fw): update SniffIso15693 forward declarations" +``` diff --git a/docs/plans/2026-07-03-firmware-upstream-rebase-plan.md b/docs/plans/2026-07-03-firmware-upstream-rebase-plan.md new file mode 100644 index 0000000..2a73fad --- /dev/null +++ b/docs/plans/2026-07-03-firmware-upstream-rebase-plan.md @@ -0,0 +1,131 @@ +# Firmware Upstream Rebase Plan (2026-07-03) + +Rebase the `firmware/` submodule (fork `dangerous-tac0s/proxmark3-pm3py`) +onto current RRG `upstream/master` **without losing our sim / sniff / +inventory patch**. Backed by a non-destructive dry-run (worktree + merge-tree). + +## Current state (measured) + +- **merge-base:** `c7086c227` (2026-02-20) — where we branched. +- **our patch:** 33 commits on top of merge-base (main == `35db99fa5`). +- **upstream ahead:** 1049 commits (tip `87388b389`, 2026-07-03) — ~4.5 months. +- Firmware is on `main`; ALM branch already scrapped. + +## Dry-run findings — the conflict surface is small and concentrated + +A **squash-then-rebase** (all 33 commits applied as one combined diff, per +`git merge-tree`) conflicts in **exactly 3 files**; everything else +auto-merges or is untouched: + +| File | Verdict | ours vs upstream (lines) | +|------|---------|--------------------------| +| `armsrc/iso15693.c` | ⚠️ real work — 13 hunks | +599/−146 vs +523/−214 | +| `client/src/cmdhf15.c` | ⚠️ moderate | +372/−7 vs +544/−45 | +| `include/pm3_cmd.h` | ✅ trivial (textual re-append) | +21/−0 vs +94/−113 | +| `appmain.c`, `iclass.c`, `iso15693.h`, `iso15.h`, `Makefile` | ✅ auto-merge | — | +| `sim_table.c/h`, `sim_crypto.c/h`, `Standalone/hf_15sniff.c`, `Standalone/hf_unisniff.c` | ✅ upstream never touched | — | + +A **sequential** rebase (replay 33 commits) is worse: it conflicts on +commit #1 and, because **24 of the 33 commits touch `iso15693.c`**, it +re-conflicts that file many times. → **Squash first, resolve once.** + +### CMD-ID collision check — CLEAR ✅ + +All six IDs we added are free in upstream (upstream's `0x033x` block only +reaches `CSETUID_V2 0x0333`; `0x095x` is empty): + +``` +0x0336 CMD_HF_ISO15693_SIM_TRACE 0x0950 CMD_SIM_TABLE_UPLOAD +0x0337 CMD_HF_SNIFF_STREAM 0x0951 CMD_SIM_TABLE_CLEAR +0x0338 CMD_HF_SNIFF_STATUS 0x0952 CMD_SIM_TABLE_UPDATE +``` + +So `pm3_cmd.h` is a pure textual re-append (drop our `#define`s at a clean +spot in the churned region), **not** a semantic collision. + +### `iso15693.c` hunk map (13 hunks, 2 functions) + +- **15693 sniff loop (~L1598–1853):** our streaming-sniff feature (ring + buffer, button-toggle, idle flush) vs upstream sniff edits. Biggest hunk + ~53 lines (our idle/flush block) — mostly *keep-ours-add-both*. +- **SimTag / sim loop (~L2475–3372):** our sim handler (table lookup, + access control, random UID, periodic field report) + the `CheckCrc15` + 4-byte-frame fix vs upstream. ~10 hunks: three large (span 36–43, our + inserted blocks → take-ours), the rest small (≤16) needing genuine + line-level reconciliation — notably **our `CheckCrc15` vs upstream's + `CalculateCrc15`** at ~L2591 (keep ours; it fixes 4-byte frames). + +Effort estimate: **~half a day** of focused merge work, dominated by +`iso15693.c`; `cmdhf15.c` moderate; `pm3_cmd.h` ~10 min. + +## Strategy: squash-to-one → rebase (single 3-file resolution) → validate → re-split + +Resolve conflicts exactly once (matching the merge-tree preview), validate +the build, then restore clean per-file commits for cheap future rebases +(the design-doc "atomic single-file commit" model). + +### Procedure (all in an isolated worktree — real checkout never moves) + +```bash +FW=/home/work/pm3py/firmware +WT=/fw-rebase +git -C $FW fetch upstream master +git -C $FW worktree add -b rebase-wip "$WT" 35db99fa5 + +cd "$WT" +# 1. collapse 33 commits into one diff on our base (no upstream yet → no conflicts) +MB=$(git merge-base HEAD upstream/master) +git reset --soft "$MB" +git commit -m "pm3py sim+sniff+inventory patch (squashed for rebase)" + +# 2. rebase the single commit → conflicts in the 3 known files, resolved once +git rebase upstream/master # resolve iso15693.c, cmdhf15.c, pm3_cmd.h +# ... resolve, git add, git rebase --continue + +# 3. re-split into logical per-file commits (soft reset + staged adds) +git reset --soft upstream/master +# commit A: sim_table.c/h + sim_crypto.c/h (new modules) +# commit B: pm3_cmd.h + iso15.h + iso15693.h (IDs + struct fields) +# commit C: appmain.c (dispatch) +# commit D: iso15693.c (sim + sniff + inventory) +# commit E: cmdhf15.c (client reader/inventory) +# commit F: Makefile + Standalone/* (build glue) +``` + +### Validation (all local per build-workflow — ARM builds here, not the Mac) + +1. `make -C "$WT" clean && make -C "$WT" PLATFORM=PM3GENERIC` — ARM compiles. +2. `make -C "$WT" client` — validates `cmdhf15.c`. +3. `python -m pytest tests/` in pm3py — sim tests are transport-mocked, so + they stay green; confirms no Python-side regression. +4. **Hardware smoke test (user, has PM3):** flash, then `hf 15` scan + + 15693 sim + a phone read (the known-good sim path). This is the only + check the dry-run can't cover — upstream may have altered 15693 wire + behavior around our insertions. + +### Landing + +- Show resolved `iso15693.c` diff for review **before** anything touches + `main`. +- Fast-forward `firmware` main to `rebase-wip`, bump the submodule pointer + in a pm3py commit, force-push the fork (`--force-with-lease`). +- Remove the worktree (`git worktree remove`, no `--force`). + +## Risks & rollback + +- **Semantic drift in `iso15693.c`:** upstream may have changed shared + helpers our sim path calls. Compile catches signature breaks; behavior + needs the hardware smoke test. +- **Rollback:** old main is preserved. Recovery ref for pre-rebase tip: + `35db99fa5`. Nothing is force-pushed until build + review pass. +- **Safety Net:** `branch -D`, `rm -rf`, `git clean -f`, `worktree remove + --force` are blocked — use `-d`/plain `rm`/`worktree remove` (no force), + or hand force-ops to the user. + +## Go-forward (close the gap that let us drift 1049 commits) + +From the earlier maintenance discussion, still unbuilt: +- `pm3py/firmware_compat.py` — pin `UPSTREAM_TESTED_COMMIT` = `87388b389`. +- A **real** rebase-check CI (the existing `firmware/.github/workflows/ + rebase.yml` is upstream's Changelog Reminder, not ours). +``` diff --git a/docs/plans/2026-07-05-14a-custom-commands-design.md b/docs/plans/2026-07-05-14a-custom-commands-design.md new file mode 100644 index 0000000..98389e6 --- /dev/null +++ b/docs/plans/2026-07-05-14a-custom-commands-design.md @@ -0,0 +1,117 @@ +# Custom ISO14443-A command handling for the card sim — Design + +**Status:** Design approved, unimplemented · **Date:** 2026-07-05 · **Plan:** [2026-07-05-14a-custom-commands-plan.md](2026-07-05-14a-custom-commands-plan.md) + +## Problem + +The 14a card sim serves the *standard* command set from firmware-native handlers +backed by emulator RAM (EML). It has **no mechanism for custom/vendored commands**: + +- **14a-3 (Layer 3):** proprietary commands common in the parts we model — most + importantly **NTAG I2C** `SECTOR_SELECT` (0xC2) + cross-sector `READ`/`FAST_READ`, + plus vendor reads (ST25TN system block / product ID). Today an unknown L3 command + falls through `SimulateIso14443aTagEx`'s final `else` and gets **no response**. +- **14a-4 (Layer 4 / ISO-DEP):** vendor APDUs and crypto (DESFire, JCOP, EMV, custom + applets). Today RATS→ATS is answered, but any I-block APDU gets a **blanket fake + `90 00`** (`iso14443a.c:2529-2545`) with no parsing — wrong for everything real. + +This contrasts with the **15693 sim**, which already serves custom commands from a +Python-compiled response table in BigBuf (`sim_table`) consulted in its `default:` +case (`iso15693.c:~3081`). That mechanism is proven; 14a simply never got it wired in. + +## Constraints + +1. **86µs FDT at Layer 3.** A tag answer must start ~86µs after the reader command + ends. This rules out a live USB round-trip for L3 — no reader-command → host → + response is possible inside the window. +2. **Reuse over reinvention.** The firmware is a fork maintained with atomic + single-file commits for easy rebase against upstream PM3. The 15693 `sim_table` + server, the 120-byte `TableEntry` wire format, the `TableCompiler`, and the + `0x0950-0x0952` upload command IDs already exist and are fork-local. +3. **Additive, non-invasive.** The native-EML sim is correct and battle-tested for + standard commands. Custom-command support must *layer on top* of it, not replace it. + +## Decision + +**A table-first L3 + WTX-L4 hybrid, layered on the native-EML sim.** Five mechanisms, +each the cheapest *correct* one for its command class, all tracing through the existing +`EmLogTrace` chokepoint: + +| # | Layer | Mechanism | Handles | Host round-trip? | +|---|-------|-----------|---------|------------------| +| A | L3 | **Native-EML** (unchanged) | Standard: anticoll, READ, FAST_READ, WRITE, GET_VERSION, READ_SIG, counters, RATS | No | +| B | L3 | **Native-EML + sector offset** (graft) | NTAG I2C cross-sector READ/FAST_READ | No | +| C | L3 | **Static `sim_table`** (new hook) | Custom-command *state machine* (SECTOR_SELECT), static vendor reads | No (pre-uploaded) | +| D | L3 | **Native firmware auth** (existing branches) | UL-C 3DES, UL-AES, MFC Crypto1 | No | +| E | L4 | **Static `sim_table`** (same hook) | Deterministic ISO-DEP APDUs: Type-4 NDEF SELECT/ReadBinary, canned GET/PUT DATA | No (pre-uploaded) | +| F | L4 | **WTX relay** (new) | Dynamic ISO-DEP crypto: DESFire/JCOP/EMV, unpredictable reader nonce | Yes (S(WTX) buys seconds) | + +The whole thing is **three additive firmware hooks** in `SimulateIso14443aTagEx`, +copied in spirit from the 15693 sim: + +1. **Host-poll** at loop top — `data_available()`/`receive_ng()` handling + `CMD_SIM_TABLE_UPLOAD/UPDATE/CLEAR`, EML load, relay responses (mirrors + `iso15693.c:2472-2508`). Prerequisite: without it the table is empty forever. +2. **Table-check** at the top of the final `else` (`:2493`) — normalize, `sim_table_lookup`, + `sim_table_execute`; placed *before* the ST25TA block and the fake-`90 00` switch so + the table overrides those fakes. On an L4 I-block miss with ISO-DEP active, falls into + the WTX relay. +3. **Sector-offset graft** on native `READ`/`FAST_READ` — add `sim_active_sector()*0x400` + to the `emlGet` source offset. + +## Why not the alternatives + +- **Relay-first at L3 (rejected).** Serving custom L3 commands by relaying to Python on + a reader *retry* was scored `correctness 2 / fdt_safety 2` and rejected: the 86µs FDT + is unreachable over USB, and a reader that gives up drops to REQA/WUPA + anticollision, + losing the very state (selected sector) the relay was trying to serve. Retry-relay + survives only as a debug affordance. +- **Per-page table for cross-sector reads (rejected).** Representing NTAG I2C sectors as + hundreds of per-page `sim_table` entries blows the linear-scan lookup toward the FDT and + the entry/response-size ceilings. Instead, **reads stay native** and become + sector-aware via a one-line offset — O(1), wire-speed, no entry explosion. + +## The SECTOR_SELECT problem → group state + a firmware projection + +NTAG I2C `SECTOR_SELECT` is the hard case that shaped the design. It is: + +- **Two-phase:** `C2 FF` (packet 1) → 4-bit ACK, then a *bare* sector byte `s 00 00 00` + (packet 2, no opcode) → **silent success** (the tag transmits nothing for >1ms). +- **Stateful and persistent:** it changes the addressing of *all subsequent* READs until + the next SECTOR_SELECT or a re-selection. + +This is handled by the `sim_table` **group state machine** plus a read-only firmware +projection: + +- Phase 1 (`C2 FF`) activates a transient `GRP_SEL_PENDING` group and 4-bit-ACKs. +- Phase 2 (`s 00 00 00`) matches **only inside `GRP_SEL_PENDING`** (so a stray `00`/`01` + frame can't false-flip the sector), sets `GRP_SECTOR_s`, clears the pending group, and + responds with `response_len == 0` = the passive-ACK silence. +- An invalid-sector catch-all in `GRP_SEL_PENDING` returns a 4-bit NAK. +- `sim_active_sector()` derives the active sector **read-only** from the group register, + so the group register stays the single source of truth (no drift between routing and + read data), and native READ/FAST_READ offset into the right sector. + +This is the "graft" that resolves both judges' top risks: it keeps the table tiny +(~6 entries, not 256) so FDT is trivially safe, and it makes native-firing-first +*correct* (reads are sector-aware) instead of a bug that serves flat sector-0 data. + +## Crypto taxonomy correction + +The judges flagged a miscategorization worth stating plainly: **Ultralight-C 3DES +(0x1A/0xAF) and UL-AES are Layer-3 framed** (no ISO-DEP). They are servable by *neither* +the table (unpredictable reader `RndB`) nor the L4 WTX relay (86µs FDT). They use the +**existing native firmware auth branches** (`iso14443a.c:2242/2273/2324/2355`) with the +model key loaded into EML — a "native-auth" track (mechanism D), first cut with the +firmware's fixed nonce (a documented fidelity gap vs the model's random `RndB`). Only +genuinely dynamic *ISO-DEP* crypto (DESFire/JCOP AES) goes to the L4 relay. + +## Provenance + +This design was produced by a multi-agent workflow: 6 parallel readers mapped the 14a +sim dispatch, the proven 15693 `sim_table` + retry-relay template, the tag-model custom +commands, the `TableCompiler`, the WTX-relay design, and the L3 timing/retry reality; +3 competing designs (static-table-first, relay-first, hybrid) were adversarially judged; +and the result was synthesized. Judge outcome: static-table-first and hybrid both +"adopt-with-changes" (reuse 5/5); relay-first "reject" (FDT-unsafe). See the plan doc for +the sequenced, implementation-grade steps. diff --git a/docs/plans/2026-07-05-14a-custom-commands-plan.md b/docs/plans/2026-07-05-14a-custom-commands-plan.md new file mode 100644 index 0000000..1fb18f6 --- /dev/null +++ b/docs/plans/2026-07-05-14a-custom-commands-plan.md @@ -0,0 +1,248 @@ +# Custom ISO14443-A command handling — Implementation Plan + +**Status:** Planned, unimplemented · **Date:** 2026-07-05 · **Design:** [2026-07-05-14a-custom-commands-design.md](2026-07-05-14a-custom-commands-design.md) + +Serve custom/vendored ISO14443-A commands in the card sim, layered on the native-EML +14a sim. Two independently shippable tracks: **L3** (NTAG I2C SECTOR_SELECT + cross-sector +reads, static vendor reads, native auth) and **L4** (static ISO-DEP APDUs + WTX relay for +dynamic crypto). Reuses the proven 15693 `sim_table` server, the 120-byte `TableEntry` +format, and the `TableCompiler`. All firmware line numbers are anchors against the current +`firmware/armsrc/iso14443a.c`; each firmware change lands as an atomic single-file commit +anchored to **stable symbols** (`ISO14443A_CMD_RATS`, the final `else`, +`EmSendPrecompiledCmd`), not line numbers. + +--- + +## L3 track — mechanisms + +- **(A) Native-EML (unchanged):** REQA/WUPA/anticoll/SELECT/PPS/GET_VERSION/READ_SIG/ + READ/FAST_READ/WRITE/COMPAT_WRITE/READ_CNT/INCR_CNT/HALT/RATS. Byte-correct within + 86µs, zero new code. WRITE mutates the EM BigBuf so later reads reflect writes natively. +- **(B) Native-EML + sector offset (graft):** cross-sector READ/FAST_READ. + `sim_active_sector()` maps the `sim_table` active-group bits → `{0,1,3}`; native READ + (`:2033-2058`) and FAST_READ (`:2081-2093`) add `sim_active_sector()*0x400` to the + `emlGet` source offset. Python pre-zeroes invalid pages in the uploaded multi-sector EML + image so native 00-fill falls out for free. Reads stay O(1)-native, sector-correct. +- **(C) Static `sim_table` (new `:2493` hook):** the SECTOR_SELECT *state machine* + + genuinely-custom static reads only (see worked example). No per-page entries. +- **(D) Native firmware auth (existing branches):** UL-C 3DES (`:2242`), UL-AES + (`:2273/2324/2355`), MFC Crypto1 — model key in EML, firmware computes challenge/response, + first cut fixed-nonce. Not table, not relay. + +**L3 miss policy:** reader-retry relay is debug-only (86µs FDT is unreachable over USB). +Everything reachable at L3 is covered by A–D. + +## L4 track — mechanisms + +Gated by a new `iso_dep_active` flag (set at RATS→ATS `:2234`; cleared on HALT / S(DESELECT)). + +- **(E) Static `sim_table`** (same `:2493` hook): deterministic ISO-DEP APDUs — Type-4 + NDEF SELECT-App(AID) / SELECT-File(CC,NDEF) / ReadBinary, canned GET/PUT DATA, fixed + `90 00`. Firmware masks the PCB (`(pcb&0xE2)==0x02` = I-block), computes + `off = 1 + CID + NAD`, strips the trailing CRC, `MATCH_PREFIX` on the bare APDU; on hit + re-wraps (`response[0]=pcb`, CID re-inserted at `response[1]` when `pcb&0x08`, payload at + `response+off`, `AddCrc14A`). **Overrides the bogus blanket `90 00`** at `:2529-2545` — + which is exactly why the hook sits at `:2493` (before the switch), not `:2576`. +- **(F) WTX relay** (dynamic crypto): on an I-block table miss with `iso_dep_active`, + firmware sends a tag-side **S(WTX)** to buy time (FWT×59 ≈ 4.8s), relays the normalized + APDU to Python, spin-waits for the computed response, re-wraps and sends. The Python + transponder model owns per-session nonce/IV/auth state. + +**L4 chaining** (gating dependency for DESFire/NDEF coverage): tag block-number toggle +(PCB bit0 per I-block), reader→tag reassembly (PCB bit4 → R(ACK) `0xA2|bn`), tag→reader +FSD split into chained I-blocks. `DYNAMIC_RESPONSE_BUFFER_SIZE` bumped `64→256` first. +v1 gated to single-frame + `0xAF` continuation until chaining lands. + +--- + +## Firmware steps + +| ID | File | Change | +|----|------|--------| +| **FW-1** | `armsrc/sim_table.h` + `.c` | Generalize `sim_table_execute` from `(iso15_tag_t *tag)` to `(uint8_t *eml_data, uint16_t eml_size, uint32_t *auth_state)`; replace `tag->data`/`sizeof(tag->data)`/`tag->auth_state` at `sim_table.c:132-169`. Drop `#include iso15.h` from the header. Add `#define SIM_FLAG_RESP_4BIT 0x04`. Struct unchanged (120 bytes PACKED). Fork-local. | +| **FW-2** | `armsrc/iso15693.c` | Update the single `sim_table_execute` call site (`~:3081`) to the new signature: `(tag->data, sizeof(tag->data), &tag->auth_state)`. Keeps 15693 green. | +| **FW-3** | `armsrc/iso14443a.c` (commit A) | `#include sim_table.h`. Add file-scope `static uint32_t s_auth_state` and `sim_active_sector()` near the sim-trace statics (`~:1727`); reset `s_auth_state=0` in the `FLAG_SIM_TRACE` arm. Bump `DYNAMIC_RESPONSE_BUFFER_SIZE 64→256` and the modulation buffer. Insert **host-poll** block at loop top after `sim_trace_flush` (`~:1907`): `CMD_SIM_TABLE_UPLOAD` (4-byte `initial_groups` + 120B entries → `sim_table_init/add`), `UPDATE`, `CLEAR`, `CMD_HF_MIFARE_EML_MEMSET` (`emlSet`), `CMD_BREAK_LOOP`. Copied ~verbatim from `iso15693.c:2472-2508`. | +| **FW-4** | `armsrc/iso14443a.c` (commit B) | **L3 table-check** at `:2493` (after the `response_n`/`modulation_n` reset, before `tagType==10` and the fake-`90 00` switch): `sim_table_lookup` + `sim_table_execute`; honor `SIM_FLAG_RESP_4BIT` (`EmSend4bit`), `response_len==0` (passive-ACK silence), byte response (join the `:2591` build path), `SIM_FLAG_APPEND_CRC`. **Sector-offset graft** on native READ/FAST_READ (`sim_active_sector()*0x400`). Add labels `build_send:` (`:2591`) and `after_send:` (past `:2620`). | +| **FW-5** | `include/pm3_cmd.h` | Append `CMD_SIM_RELAY_EVENT 0x0953` and `CMD_SIM_RELAY_RESP 0x0954` in the existing `0x095x` block. Additive. | +| **FW-6** | `armsrc/iso14443a.c` (commit C) | Add file-scope `bool iso_dep_active` (near `order` `:1870`); set at RATS→ATS (`:2234`), clear on HALT (`:2214`) and S(DESELECT). Extend the `:2493` hook: detect I-block, `off=1+CID+NAD`, strip PCB/CID/NAD+CRC, `MATCH_PREFIX` on the bare APDU; on hit re-wrap and join `build_send`. Overrides fake `90 00`. | +| **FW-7** | `armsrc/iso14443a.c` (commit D, then E) | **D:** `EmSendWtx(uint8_t wtxm)` tag-side S(WTX) sender modeled on `EmSendCmd` (the `:3787-3808` `send_wtx` is reader-side, not reusable). On L4 miss: `EmSendWtx(59)` → await reader S(WTX) echo (`(b&0xF2)==0xF2`) → `reply_ng(CMD_SIM_RELAY_EVENT,{L4,apdu})` → spin `data_available()`/`receive_ng()` for `CMD_SIM_RELAY_RESP` (GetTickCount timeout) → re-wrap + `EmSendCmd`. **E:** block-number toggle + reader→tag chaining reassembly + tag→reader FSD split. | + +## pm3py steps + +| ID | File | Change | +|----|------|--------| +| **PY-1** | `pm3py/core/protocol.py` | **FIX id drift:** repoint `SIM_TABLE_UPLOAD/CLEAR/UPDATE` from stale `0x0900/0901/0902` → `0x0950/0951/0952`. Add `SIM_RELAY_EVENT=0x0953`, `SIM_RELAY_RESP=0x0954`. Reuse `HF_MIFARE_EML_MEMSET=0x0602` for EML load. | +| **PY-2** | `pm3py/sim/table_compiler.py` | Add `RESP_FLAG_4BIT=0x04` + group constants (`GRP_SEL_PENDING=1, GRP_SECTOR0=2, GRP_SECTOR1=3, GRP_SECTOR3=4`). Add `compile_ntag_i2c(tag)`: `base=compile_14a(tag)` + **only** the SECTOR_SELECT machine (no per-page reads). Add an `enumerate_table_entries()` hook so a model can yield full `TableEntry` objects (groups/flags) alongside the legacy `enumerate_responses` 3-tuple. | +| **PY-3** | `pm3py/transponders/hf/iso14443a/nxp/ntag_i2c.py` | Add `build_eml_image()` laying out `[mfu prefix][sector0, invalid pages zeroed][sector1][sector3 regs]` at `0x400` stride, reusing `_masked_page`/`_page_readable`. Add `enumerate_table_entries()` yielding the SECTOR_SELECT machine. Keep `enumerate_responses` (sector-0) for other consumers. | +| **PY-4** | `pm3py/sim/sim_session.py` (L3) | Fix constants. In `start_14a`: if the tag is table-backed, load the EML image (`CMD_HF_MIFARE_EML_MEMSET`) then `_compile_and_upload_table` with a 14a dispatch branch (`NtagI2C→compile_ntag_i2c`, `MifareClassic→compile_mifare`, else `compile_14a`). Factor the `initial_groups`+120B chunked uploader into a shared `_upload(table, initial_groups)`. | +| **PY-5** | `pm3py/sim/sim_session.py` (L4) | Async relay path for L4-crypto tags: send `HF_ISO14443A_SIMULATE` via async `PM3Transport`, `create_task(self._relay_loop(tag))`; **do not** spawn the sync `_trace_reader` thread (single async reader owns trace + relay on one fd). Flesh out `_relay_loop`: `CMD_SIM_RELAY_EVENT` → `RESELECT`→`tag.power_on()`; `L4`→`out=await tag.handle_frame(RFFrame.from_bytes(apdu))`, reply `CMD_SIM_RELAY_RESP` with `resp_flags` + bare bytes. | + +## New / reused command IDs & flags + +- `CMD_SIM_RELAY_EVENT = 0x0953` **(new)** fw→host: normalized L4 APDU on table miss. +- `CMD_SIM_RELAY_RESP = 0x0954` **(new)** host→fw: computed L4 response. +- `CMD_SIM_TABLE_UPLOAD/CLEAR/UPDATE = 0x0950/0951/0952` **(reused;** fix `protocol.Cmd` stale `0x0900` series). +- `CMD_HF_MIFARE_EML_MEMSET = 0x0602` **(reused)** to load the multi-sector 14a EML image (native sim already reads tag memory from the EM BigBuf via `emlGet`). +- `SIM_FLAG_RESP_4BIT = 0x04` **(new)** response-flag (fw `sim_table.h` + `RESP_FLAG_4BIT` in `table_compiler.py`) — not a wire command. +- `GRP_SEL_PENDING=1 / GRP_SECTOR0=2 / GRP_SECTOR1=3 / GRP_SECTOR3=4` — compiler-side group-bit convention (max 31 groups), not firmware constants. + +## Wire formats + +**`sim_table_entry_t`** (120-byte PACKED, reused byte-for-byte, LE +`<32s B B 64s B B B H B B B B B I I B B B B>`): `match[32]@0`, `match_len@32`, +`match_mode@33` (0=EXACT, 1=PREFIX), `response[64]@34`, `response_len@98`, +`response_flags@99` (bit0=`APPEND_CRC`, **bit2=`RESP_4BIT`** new), `eml_action@100`, +`eml_offset(u16)@101`, `eml_len@103`, `eml_resp_insert@104`, `cmd_data_offset@105`, +`cmd_data_len@106`, `group@107`, `activate_groups(u32)@108`, `deactivate_groups(u32)@112`, +`set_auth@116`, `clear_auth@117`, `flags@118` (bit1=`CONSUME`), `_pad@119`. +`TableEntry.serialize()` already emits exactly this. + +**Upload** (unchanged): `CMD_SIM_TABLE_UPLOAD` first frame = 4-byte `initial_groups` (LE) ++ N×120B entries filling `512-4` bytes; `CMD_SIM_TABLE_UPDATE` = `(512//120)*120`B chunks; +`CMD_SIM_TABLE_CLEAR` = empty. + +**Relay event** (`CMD_SIM_RELAY_EVENT` payload): `byte[0]=ev_type` (0=RESELECT, 3=L3-debug, +4=L4), `byte[1]=norm_len`, `byte[2..]=normalized frame` (L4: PCB/CID/NAD+CRC stripped). +**Relay resp** (`CMD_SIM_RELAY_RESP` payload): `byte[0]=ev_type echo`, `byte[1]=resp_flags` +(bit0=append-CRC, bit1=4BIT, bit2=passive-ACK/no-tx, bit4=chaining-more), `byte[2..]=bare +response`. `tag.handle_frame` returns **bare bytes** — firmware owns PCB/CID/NAD framing +and block-number. + +## NTAG I2C worked example (NT3H2211, 2k) — reading Sector 1 + +**Compile (`compile_ntag_i2c`):** `initial_groups` seeds `GRP_SECTOR0` active. Multi-sector +EML image loaded via `CMD_HF_MIFARE_EML_MEMSET`. ~6 table entries (far under any FDT/entry +limit): +- `E1` `[C2 FF]` EXACT → `[0A]` `RESP_4BIT`, activate `GRP_SEL_PENDING`. +- `E2` `[00 00 00 00]` EXACT, group=`GRP_SEL_PENDING`, `response_len=0`, activate + `GRP_SECTOR0`, deactivate `SECTOR1|SECTOR3|SEL_PENDING`. +- `E3` `[01 00 00 00]` → `GRP_SECTOR1`. `E4` `[03 00 00 00]` → `GRP_SECTOR3`. +- `E5` `[]` PREFIX (last) in `GRP_SEL_PENDING` → `[00]` 4-bit NAK, deactivate `SEL_PENDING`. + +**Runtime:** +1. Native anticoll/SELECT (unchanged). `sim_active_sector()==0`. +2. Reader → `C2 FF` → `:2493` hit `E1` → `EmSend4bit(0x0A)`, `active_groups |= SEL_PENDING`. +3. Reader → `01 00 00 00` (bare sector byte) → `:2493` in `SEL_PENDING` hit `E3` → + `response_len==0` ⇒ apply state (+`SECTOR1`, clear others + `SEL_PENDING`), **transmit + nothing** = the >1ms passive-ACK. `sim_active_sector()` now returns 1. +4. Reader → `FAST_READ 3A 00 0F` → **native** FAST_READ fires first (correct), reads + `emlGet` at `base + 1*0x400` = sector-1 memory, returns via `EmSendCmd`. Sector-correct, + O(1), wire-speed, no table scan, no USB. +5. Back to sector 0: `C2 FF` (E1), `00 00 00 00` (E2 → `SECTOR0`, silence). +6. Invalid sector: `C2 FF` then `07 00 00 00` → `E5` → 4-bit NAK, `SEL_PENDING` cleared. + +All frames trace via `EmLogTrace`/`sim_trace_push` automatically. + +## Stateful command handling — five representations + +1. **In-table group state** (no host, wire-speed): SECTOR_SELECT active sector, + SLIX2-style privacy, PWD_AUTH-gated read sets. `activate_groups`/`deactivate_groups` + masks; two-phase handshakes use a transient PENDING group so a stray frame can't + false-flip; silent success = `response_len==0`; 4-bit ACK/NAK = `RESP_4BIT`. +2. **Firmware sector projection** (the graft): `sim_active_sector()` derives the active + sector *read-only* from `active_groups` — single source of truth, native reads O(1). +3. **EML-backed mutable state** (no host): live WRITE persists into EM BigBuf; later + native reads reflect it. +4. **Native-auth state** (no host): UL-C/UL-AES/MFC via existing firmware branches with + key in EML; `auth_state` word + firmware nonce chain carry session state. First cut + fixed-nonce. +5. **Relay state** (host, L4 only): dynamic ISO-DEP crypto; Python owns per-session + nonce/IV/auth in `handle_frame`. A challenge-prefix entry may be `SIM_FLAG_CONSUME` + (one-shot) to hand the encrypted tail to the relay. + +**Session reset:** on REQA/WUPA/HALT firmware resets `active_groups` + `iso_dep_active` +and (for L4 relay tags) emits `CMD_SIM_RELAY_EVENT{RESELECT}` so Python calls +`tag.power_on()`, keeping the C and Python state machines in lockstep across a reader +give-up → re-anticollision. + +## Test plan + +- **UNIT (pm3py):** `TableEntry.serialize()` round-trips `RESP_4BIT`/group fields at exact + byte offsets (99, 107-118), `sizeof==120`. +- **UNIT:** `compile_ntag_i2c` emits exactly the E1-E5 machine (correct match/groups/masks/ + flags); assert **no** per-page sector reads. +- **UNIT:** `build_eml_image()` lays out `[prefix][sector0 zeroed-invalid][sector1] + [sector3]` at 0x400 stride with per-sector PWD/PACK masking (no secret leakage). +- **UNIT:** `protocol.Cmd` drift fixed (`SIM_TABLE_* == 0x0950/1/2`, relay ids); sim_session + constants match; FLAG_SIM_TRACE bit confirmed against firmware. +- **UNIT:** `_relay_loop` parses `CMD_SIM_RELAY_EVENT{L4}`, calls `handle_frame`, replies + with correct `resp_flags`; RESELECT triggers `power_on`. +- **FIRMWARE UNIT (host build of `sim_table.c`):** group gating + activate/deactivate + transitions E1→E3; `response_len==0` path; `RESP_4BIT` path; `sim_active_sector()` maps + bits → `{0,1,3}`. +- **FIRMWARE UNIT:** worst-case `sim_table_lookup` latency on the 48MHz ARM for the ~30-entry + table vs 86µs FDT — **measure, do not hand-wave** (must-pass gate). +- **HARDWARE L3:** PM3 Easy sims NT3H2211; reader issues `C2 FF` + `01 00 00 00` + + FAST_READ; live-trace confirms 4-bit ACK, >1ms silence, sector-1 data, invalid NAK; and + that a table hit fully suppresses the fake-`90 00` path. +- **HARDWARE L3:** verify passive-ACK phase-2 emits **nothing** (no spurious frame the + reader misreads as NAK). +- **HARDWARE L4 static:** Type-4 NDEF SELECT-App/File/ReadBinary from the table, overriding + fake `90 00`; TagInfo reads NDEF. +- **HARDWARE L4 relay:** DESFire GET_VERSION (`0xAF` chaining) and a >FSD NDEF read via WTX + relay; assert S(WTX) echo, block-number toggle, chained reassembly both directions. +- **REGRESSION:** 15693 sim still works after the `sim_table_execute` signature change; full + pytest suite green. + +## Sequencing + +**L3 track (ship first, independently testable):** +1. FW-1 (`sim_table` signature + `RESP_4BIT`) → FW-2 (`iso15693.c` call site, keeps 15693 green). +2. FW-3 (commit A: include, statics, buffer bump, host-poll upload). +3. FW-4 (commit B: L3 table-check + sector-offset graft). +4. PY-1 (id fix) → PY-2 (`compile_ntag_i2c`) → PY-3 (`build_eml_image` + entries) → PY-4 (`start_14a` upload + dispatch). +5. **Test L3 on hardware.** Commit + bump firmware gitlink. + +**L4 track (independent, builds on the L3 host-poll hook):** +6. FW-5 (`pm3_cmd.h` relay ids). +7. FW-6 (commit C: `iso_dep_active` + static-APDU serve overriding fake `90 00`). +8. FW-7 (commit D: `EmSendWtx` + WTX relay; commit E: block-number toggle + chaining). +9. PY-5 (async `start_14a` relay path + `_relay_loop`). +10. **Test L4 on hardware** (DESFire chaining, NDEF>FSD). + +## Risks & mitigations + +| Risk | Mitigation | +|------|------------| +| **FDT scan blowout** — linear `sim_table_lookup` approaches 86µs on a big table. | Sector-offset graft keeps the 14a table tiny (~6-30 entries); cross-sector reads are native (O(1)). Firmware-host unit test **measures** worst-case latency vs 86µs before shipping. Contingency: first-byte bucket index in `sim_table.c` (benefits 15693 too). | +| **Dispatch order** — native READ/FAST_READ fire before `:2493` and would serve flat sector-0 data. | Resolved by design: cross-sector reads are **not** routed through the table; native branches are made sector-aware via one `emlGet` offset. Native-first is now correct. | +| **Passive-ACK phase-2** emits a spurious frame the reader reads as NAK. | `sim_table_execute` sets `*resp_len=0`; hook sets `p_response=NULL` and jumps past send; the `if(response_n>0)` build guard yields silence. Hardware test asserts >1ms silence. | +| **Command-id drift** — `protocol.py` at stale `0x0900` series mismatches framing. | PY-1 fixes it as the first Python commit; unit test asserts the values. | +| **UL-C/UL-AES miscategorized** as L4. | Routed to existing native firmware auth branches with model key via EML; documented fixed-nonce fidelity gap. Not claimed under table/relay. | +| **L4 relay claims DESFire/NDEF** but chaining/block-number don't exist in FW. | Chaining is an explicit gating commit (FW-7 E) before DESFire is claimed; buffer bumped to 256 first; v1 gated to single-frame + `0xAF`. | +| **`iso14443a.c` is a hot upstream file** — hooks churn on rebase. | Atomic single-file commits anchored to stable symbols; heavy logic in self-contained blocks; `sim_table`/`sim_crypto` fork-local; only one 15693 call site changes. | +| **CONSUME one-shots** not reset until re-UPLOAD. | Session-reset emits `RESELECT`; where a CONSUME challenge must re-arm, Python re-uploads/re-adds; documented constraint. | + +## Open decisions + +1. **`FLAG_SIM_TRACE` bit:** `sim_session.py` uses `0x2000`, the 14a live-trace plan doc + says `0x4000`; confirm the firmware's actual bit before wiring the async relay. + *(Note: reconciled this session — firmware F1 landed `FLAG_SIM_TRACE 0x2000`; the doc's + `0x4000` is stale. Verify against `pm3_cmd.h`.)* +2. **Multi-sector EML layout:** confirm the EM BigBuf `CARD_MEMORY` fits sector0+1+3 + (~2KB + config) and that `CMD_HF_MIFARE_EML_MEMSET` writes at the offset the sector graft + expects; else add a dedicated `CMD_HF_ISO14443A_EML_SETMEM` (fallback id `0x033A`). +3. **NTAG I2C no-rollover fidelity:** native READ may roll over at emulator-memory end, not + sector end. Decide pre-zeroing vs an explicit sector-length clamp in the READ branch. +4. **NTAG21x first-READ NFC counter bump** is a side effect the native/static path loses. + Accept as documented deviation, or force that first READ through a CONSUME entry (same + for EV1 INCR_CNT edges). +5. **UL-C 3DES fixed nonce vs model random `RndB`:** acceptable for sim; decide whether + firmware 3DES-with-supplied-nonce is wanted or a downscope note suffices. +6. **L4 chaining scope for v1:** minimum viable = single-frame APDUs + DESFire `0xAF` + + tag→reader FSD split. Confirm whether reader→tag chaining is needed for any target + reader in the first cut or can be deferred. +7. **Empirical reader behaviour:** do target NTAG I2C reader stacks actually issue + SECTOR_SELECT for sector 1, or only read sector-0 NDEF? If the latter, sector-1 can be + validated later without blocking the L3 ship. +8. **Apply hooks to `SimulateIso14443aTagAID` (`:4716`)?** Deferred; `TagEx` is the primary + `FLAG_SIM_TRACE` path. + +## Provenance + +Produced by the `plan-14a-custom-commands` design workflow (13 agents: 6 readers → 3 +designs → 3 adversarial judges → synthesis). Judge outcome: static-table-first and hybrid +"adopt-with-changes" (reuse 5/5, fdt_safety 3-4); relay-first "reject" (FDT-unsafe at L3). +The recommended architecture is static-table-first L3 grafted with a firmware sector +projection to resolve the SECTOR_SELECT/entry-explosion risk, plus a WTX relay for dynamic +L4 crypto. diff --git a/docs/plans/2026-07-05-14a-live-trace-plan.md b/docs/plans/2026-07-05-14a-live-trace-plan.md new file mode 100644 index 0000000..27334cf --- /dev/null +++ b/docs/plans/2026-07-05-14a-live-trace-plan.md @@ -0,0 +1,76 @@ +# Live trace for ISO14443-A sim (14a-3, extensible to 14a-4) + +Add a live reader↔tag trace to the ISO14443-A card simulation, mirroring the +working ISO15693 sim trace, designed so 14a-4 (ISO-DEP / Layer 4) reuses the +same firmware mechanism. Backed by the feasibility eval (verdict: +**feasible with care** — 86µs FDT does not break it). + +## Design invariants (why 86µs is not "too fast") +1. **Response is never touched** — table/precompiled, FPGA-timed via + `EmSendCmd14443aRaw` (SSP-clock alignment + FPGA delay-line). Tracing is + architecturally incapable of affecting FDT. +2. **Both trace pushes deferred to *after* transmit** → zero added work inside + the 86µs window. (A push is a ≤128B memcpy ≈ <200 cycles / <4.2µs anyway.) +3. **Flush ≤1 ring entry per idle iteration** (NOT 15693's bulk-drain) — the + only USB-touching part must never overlap a reader-command edge, because + `reply_ng` and the Miller software-UART receive share the ARM thread. +4. **Ring + drop-on-full** → lossy under anticollision bursts, but the sim + never stalls. RF timing is preserved; trace is merely lossy in tight gaps. + +## P0 — Prerequisite (standalone bug): fix `start_14a` sim-start payload +`sim_session.py:101` builds `payload = atqa + sak + uid` then calls +`send_ng_no_response(HF_ISO14443A_SIMULATE)` with **no payload** (`:123` TODO). +The sim never starts correctly today. Fix: send the proper `HF_ISO14443A_SIMULATE` +NG struct — the exact `` layout already used by the fixed +`hf.iso14a.sim()` (tagtype, flags, uid[10], exitAfter, rats…). Land first; +14a sim is unusable without it, trace or not. + +## Firmware (atomic single-file commits + build) +- **F1 `include/pm3_cmd.h`:** add `CMD_HF_ISO14443A_SIM_TRACE` (default id + `0x0339`, next to the 15693 one at `0x0336`); reserve a `FLAG_SIM_TRACE` bit + in the 14a sim `flags` word (default `0x4000`, a free high bit) so the host + opts trace in/out — mirrors 15693's `flags & 0x02`. +- **F2+F3 `armsrc/iso14443a.c`:** copy the `TRACE_PUSH`/`TRACE_FLUSH` ring block + from `iso15693.c:2423` but **`ENTRY_MAX` → 128** (ISO-DEP I-blocks exceed 64B, + needed for 14a-4) and ring → 16; `trace_enabled` from the new flag. Hook into + `SimulateIso14443aTagEx` (`:1725`): buffer received cmd after `:1851` (don't + push); after `EmSendPrecompiledCmd` (`:2554`) do `TRACE_PUSH(0,cmd)` then + `TRACE_PUSH(1,response)`; `TRACE_FLUSH_ONE()` at loop top (`:1844`); full + drain on loop exit. +- **F4:** build `fullimage PLATFORM=PM3GENERIC` — must be clean. + +## pm3py (reuses the generic scriptable-trace plumbing) +- **Y1 `protocol.py`:** add `Cmd.HF_ISO14443A_SIM_TRACE`. +- **Y2 `sim_session.py:284`:** generalize the trace-reader guard + (`cmd == CMD_HF_ISO15693_SIM_TRACE`) into a `{cmd: (protocol, decoder)}` + dispatch; add the 14a branch (`dir=data[0]&0x0F`, `crc_fail=data[0]&0x80`, + `protocol=1`, `decode_14443a`). +- **Y3 `start_14a`:** synchronous trace path mirroring `start_15693` (raw + pyserial + the existing `_trace_reader` thread) + `trace`/`on_frame`/`quiet` + kwargs. `on_frame`/`quiet`/`entries` + `TraceFormatter(mode='sim')` unchanged. + +## I/O model decision (the one real design fork) +- **14a-3:** sync trace path (no WTX relay at Layer 3) → avoids dual-reader-on- + one-fd contention. +- **14a-4 (later):** the async `_relay_loop` owns WTX; route `SIM_TRACE` through + *it* (single async reader), not a second thread. Firmware F1–F3 are identical + for both layers — only the pm3py consumer differs. WTX creates ms-scale idle + windows ideal for flushing. + +## Tests +- **T1:** mocked-transport unit test — synthetic `CMD_HF_ISO14443A_SIM_TRACE` + frames → assert `TraceFormatter`/`on_frame`/`entries` (mirror + `test_trace_scripting.py`). +- **T2 (bench):** phone doing anticollision + SELECT + RATS → verify (a) card + never drops, (b) frames stream, tolerating loss during anticollision. + +## Risks (+ mitigations) +- Ring fill under heavy Layer-3 polling → lossy by design; ring=16, accept loss + in anticollision (uninteresting traffic). +- Flush overlapping a reader edge → single-entry-per-idle at loop top only; + never bulk-drain post-transmit. A missed command = reader retry, not teardown. +- `ENTRY_MAX=64` too small for ISO-DEP → 128 (RAM ≈ 1KB static; check budget). +- pm3py dual reader on one fd → 14a-3 sync only; 14a-4 converges to async relay. + +## Sequence +`P0 → F1 → F2+F3 → F4 → Y1 → Y2 → Y3 → T1 → (T2 on bench)` diff --git a/docs/plans/README.md b/docs/plans/README.md new file mode 100644 index 0000000..19541d5 --- /dev/null +++ b/docs/plans/README.md @@ -0,0 +1,61 @@ +# pm3py roadmap — design & implementation plans + +This directory is the project roadmap: dated design docs (`*-design.md`) and +implementation plans (`*-plan.md`). Each file is a self-contained spec — problem, +approach, concrete steps, tests, risks. + +> **Status column is best-effort**, inferred from the codebase and memory at the time of +> writing (2026-07-05). Correct it if a plan's real state differs. Legend: +> ✅ landed · 🔄 in progress · 📋 planned · 📎 reference/ongoing. + +## ISO 14443-A sim expansion + +| Plan | Status | Summary | +|------|--------|---------| +| [2026-07-05-14a-live-trace-plan](2026-07-05-14a-live-trace-plan.md) | 🔄 | Live reader↔tag trace for the 14a sim (firmware ring + `EmLogTrace` capture + pm3py consumer). Code complete; flush + direct-send-capture fixes awaiting hardware verify. | +| [2026-07-05-14a-custom-commands-design](2026-07-05-14a-custom-commands-design.md) | 📋 | **Design:** how to serve custom 14a-3 (NTAG I2C SECTOR_SELECT + cross-sector reads) and 14a-4 (vendor ISO-DEP APDUs) commands — table-first L3 + WTX-L4 hybrid layered on the native-EML sim. | +| [2026-07-05-14a-custom-commands-plan](2026-07-05-14a-custom-commands-plan.md) | 📋 | **Plan:** sequenced L3 + L4 tracks — `sim_table` hook, sector-offset graft, WTX relay, `TableCompiler` extension, tests. | + +## Sim framework & response table + +| Plan | Status | Summary | +|------|--------|---------| +| [2026-03-17-stateful-table-firmware-design](2026-03-17-stateful-table-firmware-design.md) | ✅ | Response-table sim in firmware BigBuf: stateful `sim_table` + `sim_crypto`, group/auth state. | +| [2026-03-17-stateful-table-firmware-plan](2026-03-17-stateful-table-firmware-plan.md) | ✅ | Implementation of the above (15693 sim fully working: inventory, NDEF, NXP custom commands). | +| [2026-03-17-table-compiler-update-plan](2026-03-17-table-compiler-update-plan.md) | ✅ | `TableCompiler` / `ResponseTable` — compile per-IC responses into the 120-byte entry format. | + +## Trace & sniff + +| Plan | Status | Summary | +|------|--------|---------| +| [2026-03-17-trace-formatter-design](2026-03-17-trace-formatter-design.md) | ✅ | `TraceFormatter` — colored, decoded, column-wrapped trace output (sim/reader/sniff modes). | +| [2026-03-17-trace-formatter-plan](2026-03-17-trace-formatter-plan.md) | ✅ | Implementation of the trace formatter. | +| [2026-03-18-ndef-trace-decode](2026-03-18-ndef-trace-decode.md) | ✅ | NDEF TLV/record decode for trace annotation. | +| [2026-03-19-live-sniff-design](2026-03-19-live-sniff-design.md) | ✅ | Live streaming sniff (firmware `CMD_HF_SNIFF_STREAM` + `SniffSession`). | +| [2026-03-19-live-sniff-plan](2026-03-19-live-sniff-plan.md) | ✅ | Implementation of live sniff. | + +## Package refactor & migration + +| Plan | Status | Summary | +|------|--------|---------| +| [2026-03-18-refactor-design](2026-03-18-refactor-design.md) | ✅ | Split the monolith into `core/`, `trace/`, `sniff/`, `sim/`, `transponders/`. | +| [2026-03-18-refactor-progress](2026-03-18-refactor-progress.md) | ✅ | Progress log for the refactor (complete, all on master). | +| [2026-03-18-sim-migration](2026-03-18-sim-migration.md) | ✅ | Migrate the sim framework into the new package layout. | + +## Firmware maintenance + +| Plan | Status | Summary | +|------|--------|---------| +| [2026-07-03-firmware-upstream-rebase-plan](2026-07-03-firmware-upstream-rebase-plan.md) | 📎 | Rebase the `proxmark3-pm3py` fork against upstream PM3 (atomic single-file-commit workflow). | + +--- + +### Conventions + +- **Naming:** `YYYY-MM-DD--{design,plan}.md`. A design doc explains *why* and *what*; + a plan doc gives sequenced, implementation-grade steps. Small features may have only a + `-plan.md`. +- **Firmware changes** land as atomic single-file commits anchored to stable symbols (not + line numbers) for easy rebase against upstream — see the firmware-upstream-rebase plan. +- **Provenance:** plans produced with multi-agent design workflows note it in a Provenance + section.