scalesApp/.claude/plans/multi-protocol-scales.md

# Multi-Protocol Scale Support

## Context

The port runs 10 scales from 6 manufacturers (Mettler-Toledo, Esit, Elikom, Libra, 3 no-name), each speaking a different serial protocol. ScalesApp currently assumes one format and parses it with a regex in `Main.jsx`.

The hard constraint is **operational, not technical**: installation and upgrades must need minimal involvement from the developer. So the design question is not "how do we parse 6 formats" but **"where does protocol knowledge live so that adding a scale never means visiting 10 PCs with a new binary."**

**Answer: the bridge knows nothing about protocols.** It is pure transport — open the port, optionally send a poll string on an interval, stream raw lines. Protocol definitions live in Django, are cached by the frontend, and parsing happens in the browser where the weight is displayed. A new scale type is a web-UI change; the bridge binary never changes for it.

**Decisions made with the user:**
- Bridge = transport only. It learns its poll string at runtime from the UI, so a scale with a different command (`%p`, `SI`, `#w`) needs no bridge update.
- Protocol definitions live in Django, cached in the frontend (same pattern as `NomenclatureDataContext`).
- Live weight keeps going **bridge → UI directly** on localhost. Django stays out of the hot path, preserving the independence the project CLAUDE.md calls out.
- Admin gets a COM monitor (raw text + hex) and a fixed-column protocol builder. Exotic protocols stay code.
- Four protocols seeded: Simulator (`ST`/`US`), fixed-column `%p`, Toledo Continuous, Mettler-Toledo SICS.

---

## Why the browser, not the bridge, parses

The bridge is a PyInstaller binary on 10 station PCs. Anything it knows is expensive to change. The frontend is served from one place and updates on reload.

Polling is the exception that must stay in the bridge: a browser cannot be the heartbeat. Background tabs get timer-throttled by Chrome, a closed tab stops the scale, two open tabs double the rate. So the **bridge owns the poll loop, the UI owns the poll string.**

---

## Bridge changes (`serial_bridge/app.py`)

Note `serial_reader.py` and `backend_client.py` in that folder are dead code — nothing imports them. Leave them alone; all live code is in `app.py`.

### 1. Runtime poll configuration

```
POST /poll   {"command": "%p", "interval": 0.2}   -> bridge stores and runs the loop
POST /poll   {"command": ""}                       -> continuous mode, no polling
GET  /poll                                         -> current setting
```

The UI pushes this on connect, from the station's protocol definition. The bridge writes the bytes verbatim — `%p` bare, **no terminator** (confirmed against the original Delphi, where `TComPort.WriteStr('%p')` appends nothing).

**Persist the last setting to a local file** (`poll_config.json` beside the executable) and restore it on startup. Otherwise a bridge restarted with no browser open sits idle in a confusing dead state.

### 2. One-off command endpoint — for the admin monitor

```
POST /send   {"command": "%p"}
```

Writes once, does not change the poll loop. This is the probe tool for identifying an unknown scale.

### 3. Raw bytes alongside text

`read_data()` currently does `.decode('utf-8', errors='ignore')`, which **silently corrupts binary frames** — Toledo Continuous starts with `STX` (0x02) and packs status into bit fields. Add a `hex` field to every SSE payload:

```python
{'port': ..., 'data': '<utf-8, lossy>', 'hex': '02 20 30 ...', 'timestamp': ...}
```

The UI parses text protocols from `data` and binary ones from `hex`. Cheap now, painful to retrofit once something depends on `data`.

### 4. Polling loop

Extend `read_serial_data()`: when a poll command is set, write it, wait briefly, then read. Keep the existing buffer-drain behaviour for continuous mode (it discards stale lines and keeps the newest — added earlier today; essential for streaming scales, harmless for polled ones).

### 5. `.env` shrinks to machine-local hardware facts

```
COM_PORT=COM3
BAUD_RATE=9600
TIMEOUT=1
PORT=5000
LOG_LEVEL=INFO
```

No protocol, no poll command, no `STATION_ID`. COM port and baud must stay local — Django cannot know this PC's wiring.

### 6. Version reporting

Bridge reports its version in `/status`. Surfaced in the UI so stale stations are visible at a glance rather than discovered when one misbehaves.

---

## Backend: protocol definitions

### Model — `api/models.py`

```python
class ScaleProtocol(models.Model):
    name = models.CharField(max_length=100, unique=True)
    kind = models.CharField(max_length=20, choices=[...])   # fixed_column | regex | toledo_continuous | sics
    poll_command = models.CharField(max_length=20, blank=True)   # '' = continuous
    poll_interval = models.FloatField(default=0.2)
    config = models.JSONField(default=dict)   # kind-specific parameters
    is_builtin = models.BooleanField(default=False)   # seeded; not user-deletable
```

`config` for `fixed_column` (the case the builder targets, and what covers the no-names):

```json
{"weight_start": 4, "weight_length": 8,
 "stable_pos": 21, "stable_char": " ", "stable_means_balanced": true,
 "decimals": 0}
```

**Offsets stored 0-based**, converted for display in the builder. The `%p` scale's "weight from char 5, length 8; stability last char" is 1-based Delphi — so `weight_start: 4`, `stable_pos: 21`.

Seed four via data migration: Simulator, `%p` fixed-column, Toledo Continuous, SICS. Marked `is_builtin`.

ViewSet + route `scale-protocols` in `api/urls.py`, following the existing router convention.

---

## Frontend

### 1. Protocol cache — `contexts/ScaleProtocolContext.jsx`

Fetch `/api/scale-protocols/` on mount, cache, subscribe to SSE updates. Mirrors `NomenclatureDataContext`, which already does exactly this; reuse its shape rather than inventing a new one.

### 2. Parsers — `services/scaleProtocols.js`

One function per kind, all returning the same shape:

```js
parse(reading, protocol) -> { weight: number|null, balanced: boolean, error: string|null }
```

- **`fixed_column`** — slice by offsets. Must **not** trim before reading `stable_pos`: in the `%p` format a *space* means balanced, so trailing-whitespace stripping would invert the signal and make every reading look unstable.
- **`toledo_continuous`** — parse from `hex`. STX + SWA/SWB/SWC + 6 weight + 6 tare digits + CR. Stability is the **motion bit in SWB** (set = moving), decimal position from SWA.
- **`sics`** — `S S      16480 kg` → stable; `S D` → dynamic; `S I` / `S +` / `S -` → error states.
- **`simulator`** — trailing `ST`/`US`, the current behaviour.

**Return an explicit error rather than a plausible weight** when a frame doesn't match. Toledo and SICS are written from documented structure without hardware to test against, so they must fail loudly — a silently wrong weight on a weighbridge is worse than no weight.

### 3. `Main.jsx` — replace the inline regex

The current `currentWeight` / `isBalanced` memos parse the reading directly. Replace with a call to the selected protocol's parser. Everything downstream (the red/green colouring, the Measure button) already consumes `{weight, balanced}` and is unaffected.

The UI POSTs `/poll` to the bridge on connect with the station's protocol command and interval.

### 4. Admin: COM monitor + protocol builder

New page, gated on `is_admin`:
- **Live raw view** — each incoming line as text and hex, scrolling, pause/capture to file. Hex matters: it's the only way to see `STX` or trailing spaces.
- **Send command** — free-text box posting `/send`, for probing an unknown scale.
- **Fixed-column builder** — form for weight start/length, stability position and character; **applied live against the captured lines** so the admin sees the parsed result before saving. This is what makes it usable without the developer.

---

## Which station uses which protocol

Deployment is flexible (one central Django at the port; single-machine elsewhere), so the browser identifies its station implicitly: **it talks to whichever bridge is on localhost:5000.** No cookie, no per-browser state — cookies were rejected because they are per-browser, wiped by clearing site data, and unreadable by the bridge.

The protocol selection for a station is held in Django and chosen in the UI. With one scale per PC this is a single selection; it does not require a full Station entity yet.

---

## Edge cases

| Case | Behaviour |
|---|---|
| Frame doesn't match the protocol | Parser returns `error`; UI shows unparseable rather than a wrong number |
| Binary frame through `data` | Use `hex`; `data` is lossy by design |
| `%p` reply shorter than 22 chars | Guard offsets; return error, never slice past the end |
| Bridge restarted, no browser open | Restores poll config from its local file and keeps polling |
| Two tabs open | Both POST `/poll` with the same values — idempotent, loop stays single |
| Django down | Live weight unaffected; UI uses cached protocol definitions |

---

## Verification

1. **Simulator unchanged** — existing test writer keeps working end to end; weight displays and colours as today. This is the regression guard for the whole refactor.
2. **Fixed-column against a synthetic `%p`** — extend the test writer to emit a 22-char frame; confirm weight and the space-means-balanced rule, including that a trailing space survives.
3. **Toledo/SICS** — unit-test the parsers against hand-built frames. Mark clearly as **unvalidated against real hardware** until a real scale confirms them.
4. **Poll loop** — POST `/poll`, confirm the bridge writes on interval; empty command stops it; setting survives a bridge restart.
5. **Hex field** — verify a frame containing 0x02 arrives intact in `hex` while `data` is lossy.
6. **Builder** — capture lines from the simulator, build a fixed-column protocol in the UI, confirm the live preview parses correctly before save.
7. **Offsets** — the `%p` 1-based → 0-based conversion, explicitly tested. One position off silently clips a digit.

---

## Files

**Bridge** — `serial_bridge/app.py` (poll endpoints, `/send`, hex field, poll loop, version), `serial_bridge/.env`

**Backend** — `api/models.py`, `api/serializers.py`, `api/views.py`, `api/urls.py`, migration + seed data migration

**Frontend** — `contexts/ScaleProtocolContext.jsx`, `services/scaleProtocols.js`, `components/Main.jsx`, new admin monitor/builder component

**Not touched** — `serial_reader.py`, `backend_client.py` (dead code)

---

## Deliberately out of scope

- **Station entity** — one scale per PC makes it unnecessary now; add it if a PC ever drives two scales.
- **Bridge self-update** — version reporting only. Real update machinery is speculative until you see how often bridges actually change.
- **Regex/delimited builder kinds** — the model supports the `kind` field, but only fixed-column gets builder UI. Add others if a real scale needs one.

---

## Implementation notes (built 2026-09-17)

All three tiers built and verified. Deviations and details worth recording:

**Protocol selection is per-browser (`localStorage`), not a Station model.** With one scale per PC the browser talks to its own bridge on localhost, so a full Station entity was not needed. If a PC ever drives two scales, that changes.

**Bridge version 1.1.0** — reported in `/status` and shown in the monitor toolbar.

**`poll_config.json` sits beside the bridge executable** and is gitignored along with `serial_bridge.log`. Both are per-machine runtime state.

**Fallback protocol**: if none is chosen, the frontend uses the Simulator definition so a fresh install shows a weight rather than an error.

### Verified

- **27 parser unit tests pass** (scratchpad `test_protocols.mjs`), covering all four kinds plus error paths. Notably: space-means-balanced for `%p`, inverted stability, `decimals` scaling, Toledo motion bit and sign bit, SICS `I`/`+`/`-` error states, and `"16480 kg BEST"` correctly NOT read as stable.
- **Backend API**: seeded 4 protocols; custom create/delete; built-ins protected from deletion; invalid offsets and non-positive poll intervals rejected.
- **Bridge endpoints**: `/poll` get/set, persistence across restart, empty command = continuous, bad intervals rejected, `/send` 503 when disconnected, `/status` reports version and poll state.
- **Byte handling**: a binary Toledo frame survives byte-exact in `hex` (STX intact); a 22-char `%p` line keeps its significant trailing space while CRLF is stripped; the backlog drain still keeps only the newest line.

### Still unvalidated

**Toledo Continuous and SICS have never seen real hardware.** Both are written from documented structure. They return explicit errors rather than guesses on malformed frames, and the monitor shows the raw hex needed to correct them. Expect the first real Toledo to need a tweak — most likely the SWA decimal-point table or a SWB bit position.