scalesApp/.claude/plans/multi-protocol-scales.md

Multi-Protocol Scale Support

Context

The port runs 10 scales from 6 manufacturers (Mettler-Toledo, Esit, Elikom, Libra, 3 no-name), each speaking a different serial protocol. ScalesApp currently assumes one format and parses it with a regex in Main.jsx.

The hard constraint is operational, not technical: installation and upgrades must need minimal involvement from the developer. So the design question is not "how do we parse 6 formats" but "where does protocol knowledge live so that adding a scale never means visiting 10 PCs with a new binary."

Answer: the bridge knows nothing about protocols. It is pure transport — open the port, optionally send a poll string on an interval, stream raw lines. Protocol definitions live in Django, are cached by the frontend, and parsing happens in the browser where the weight is displayed. A new scale type is a web-UI change; the bridge binary never changes for it.

Decisions made with the user:

• Bridge = transport only. It learns its poll string at runtime from the UI, so a scale with a different command (%p, SI, #w) needs no bridge update.
• Protocol definitions live in Django, cached in the frontend (same pattern as NomenclatureDataContext).
• Live weight keeps going bridge → UI directly on localhost. Django stays out of the hot path, preserving the independence the project CLAUDE.md calls out.
• Admin gets a COM monitor (raw text + hex) and a fixed-column protocol builder. Exotic protocols stay code.
• Four protocols seeded: Simulator (ST/US), fixed-column %p, Toledo Continuous, Mettler-Toledo SICS.

---

Why the browser, not the bridge, parses

The bridge is a PyInstaller binary on 10 station PCs. Anything it knows is expensive to change. The frontend is served from one place and updates on reload.

Polling is the exception that must stay in the bridge: a browser cannot be the heartbeat. Background tabs get timer-throttled by Chrome, a closed tab stops the scale, two open tabs double the rate. So the bridge owns the poll loop, the UI owns the poll string.

---

Bridge changes (serial_bridge/app.py)

Note serial_reader.py and backend_client.py in that folder are dead code — nothing imports them. Leave them alone; all live code is in app.py.

1. Runtime poll configuration

POST /poll   {"command": "%p", "interval": 0.2}   -> bridge stores and runs the loop
POST /poll   {"command": ""}                       -> continuous mode, no polling
GET  /poll                                         -> current setting

The UI pushes this on connect, from the station's protocol definition. The bridge writes the bytes verbatim — %p bare, no terminator (confirmed against the original Delphi, where TComPort.WriteStr('%p') appends nothing).

Persist the last setting to a local file (poll_config.json beside the executable) and restore it on startup. Otherwise a bridge restarted with no browser open sits idle in a confusing dead state.

2. One-off command endpoint — for the admin monitor

POST /send   {"command": "%p"}

Writes once, does not change the poll loop. This is the probe tool for identifying an unknown scale.

3. Raw bytes alongside text

read_data() currently does .decode('utf-8', errors='ignore'), which silently corrupts binary frames — Toledo Continuous starts with STX (0x02) and packs status into bit fields. Add a hex field to every SSE payload:

{'port': ..., 'data': '<utf-8, lossy>', 'hex': '02 20 30 ...', 'timestamp': ...}

The UI parses text protocols from data and binary ones from hex. Cheap now, painful to retrofit once something depends on data.

4. Polling loop

Extend read_serial_data(): when a poll command is set, write it, wait briefly, then read. Keep the existing buffer-drain behaviour for continuous mode (it discards stale lines and keeps the newest — added earlier today; essential for streaming scales, harmless for polled ones).

5. .env shrinks to machine-local hardware facts

COM_PORT=COM3
BAUD_RATE=9600
TIMEOUT=1
PORT=5000
LOG_LEVEL=INFO

No protocol, no poll command, no STATION_ID. COM port and baud must stay local — Django cannot know this PC's wiring.

6. Version reporting

Bridge reports its version in /status. Surfaced in the UI so stale stations are visible at a glance rather than discovered when one misbehaves.

---

Backend: protocol definitions

Model — api/models.py

class ScaleProtocol(models.Model):
    name = models.CharField(max_length=100, unique=True)
    kind = models.CharField(max_length=20, choices=[...])   # fixed_column | regex | toledo_continuous | sics
    poll_command = models.CharField(max_length=20, blank=True)   # '' = continuous
    poll_interval = models.FloatField(default=0.2)
    config = models.JSONField(default=dict)   # kind-specific parameters
    is_builtin = models.BooleanField(default=False)   # seeded; not user-deletable

config for fixed_column (the case the builder targets, and what covers the no-names):

{"weight_start": 4, "weight_length": 8,
 "stable_pos": 21, "stable_char": " ", "stable_means_balanced": true,
 "decimals": 0}

Offsets stored 0-based, converted for display in the builder. The %p scale's "weight from char 5, length 8; stability last char" is 1-based Delphi — so weight_start: 4, stable_pos: 21.

Seed four via data migration: Simulator, %p fixed-column, Toledo Continuous, SICS. Marked is_builtin.

ViewSet + route scale-protocols in api/urls.py, following the existing router convention.

---

Frontend

1. Protocol cache — contexts/ScaleProtocolContext.jsx

Fetch /api/scale-protocols/ on mount, cache, subscribe to SSE updates. Mirrors NomenclatureDataContext, which already does exactly this; reuse its shape rather than inventing a new one.

2. Parsers — services/scaleProtocols.js

One function per kind, all returning the same shape:

parse(reading, protocol) -> { weight: number|null, balanced: boolean, error: string|null }

• fixed_column — slice by offsets. Must not trim before reading stable_pos: in the %p format a space means balanced, so trailing-whitespace stripping would invert the signal and make every reading look unstable.
• toledo_continuous — parse from hex. STX + SWA/SWB/SWC + 6 weight + 6 tare digits + CR. Stability is the motion bit in SWB (set = moving), decimal position from SWA.
• sics — S S 16480 kg → stable; S D → dynamic; S I / S + / S - → error states.
• simulator — trailing ST/US, the current behaviour.

Return an explicit error rather than a plausible weight when a frame doesn't match. Toledo and SICS are written from documented structure without hardware to test against, so they must fail loudly — a silently wrong weight on a weighbridge is worse than no weight.

3. Main.jsx — replace the inline regex

The current currentWeight / isBalanced memos parse the reading directly. Replace with a call to the selected protocol's parser. Everything downstream (the red/green colouring, the Measure button) already consumes {weight, balanced} and is unaffected.

The UI POSTs /poll to the bridge on connect with the station's protocol command and interval.

4. Admin: COM monitor + protocol builder

New page, gated on is_admin:

• Live raw view — each incoming line as text and hex, scrolling, pause/capture to file. Hex matters: it's the only way to see STX or trailing spaces.
• Send command — free-text box posting /send, for probing an unknown scale.
• Fixed-column builder — form for weight start/length, stability position and character; applied live against the captured lines so the admin sees the parsed result before saving. This is what makes it usable without the developer.

---

Which station uses which protocol

Deployment is flexible (one central Django at the port; single-machine elsewhere), so the browser identifies its station implicitly: it talks to whichever bridge is on localhost:5000. No cookie, no per-browser state — cookies were rejected because they are per-browser, wiped by clearing site data, and unreadable by the bridge.

The protocol selection for a station is held in Django and chosen in the UI. With one scale per PC this is a single selection; it does not require a full Station entity yet.

---

Edge cases

Case	Behaviour
Frame doesn't match the protocol	Parser returns error; UI shows unparseable rather than a wrong number
Binary frame through data	Use hex; data is lossy by design
%p reply shorter than 22 chars	Guard offsets; return error, never slice past the end
Bridge restarted, no browser open	Restores poll config from its local file and keeps polling
Two tabs open	Both POST /poll with the same values — idempotent, loop stays single
Django down	Live weight unaffected; UI uses cached protocol definitions

---

Verification

1. Simulator unchanged — existing test writer keeps working end to end; weight displays and colours as today. This is the regression guard for the whole refactor.
2. Fixed-column against a synthetic %p — extend the test writer to emit a 22-char frame; confirm weight and the space-means-balanced rule, including that a trailing space survives.
3. Toledo/SICS — unit-test the parsers against hand-built frames. Mark clearly as unvalidated against real hardware until a real scale confirms them.
4. Poll loop — POST /poll, confirm the bridge writes on interval; empty command stops it; setting survives a bridge restart.
5. Hex field — verify a frame containing 0x02 arrives intact in hex while data is lossy.
6. Builder — capture lines from the simulator, build a fixed-column protocol in the UI, confirm the live preview parses correctly before save.
7. Offsets — the %p 1-based → 0-based conversion, explicitly tested. One position off silently clips a digit.

---

Files

Bridge — serial_bridge/app.py (poll endpoints, /send, hex field, poll loop, version), serial_bridge/.env

Backend — api/models.py, api/serializers.py, api/views.py, api/urls.py, migration + seed data migration

Frontend — contexts/ScaleProtocolContext.jsx, services/scaleProtocols.js, components/Main.jsx, new admin monitor/builder component

Not touched — serial_reader.py, backend_client.py (dead code)

---

Deliberately out of scope

• Station entity — one scale per PC makes it unnecessary now; add it if a PC ever drives two scales.
• Bridge self-update — version reporting only. Real update machinery is speculative until you see how often bridges actually change.
• Regex/delimited builder kinds — the model supports the kind field, but only fixed-column gets builder UI. Add others if a real scale needs one.

---

Implementation notes (built 2026-09-17)

All three tiers built and verified. Deviations and details worth recording:

Protocol selection is per-browser (localStorage), not a Station model. With one scale per PC the browser talks to its own bridge on localhost, so a full Station entity was not needed. If a PC ever drives two scales, that changes.

Bridge version 1.1.0 — reported in /status and shown in the monitor toolbar.

poll_config.json sits beside the bridge executable and is gitignored along with serial_bridge.log. Both are per-machine runtime state.

Fallback protocol: if none is chosen, the frontend uses the Simulator definition so a fresh install shows a weight rather than an error.

Verified

• 27 parser unit tests pass (scratchpad test_protocols.mjs), covering all four kinds plus error paths. Notably: space-means-balanced for %p, inverted stability, decimals scaling, Toledo motion bit and sign bit, SICS I/+/- error states, and "16480 kg BEST" correctly NOT read as stable.
• Backend API: seeded 4 protocols; custom create/delete; built-ins protected from deletion; invalid offsets and non-positive poll intervals rejected.
• Bridge endpoints: /poll get/set, persistence across restart, empty command = continuous, bad intervals rejected, /send 503 when disconnected, /status reports version and poll state.
• Byte handling: a binary Toledo frame survives byte-exact in hex (STX intact); a 22-char %p line keeps its significant trailing space while CRLF is stripped; the backlog drain still keeps only the newest line.

Still unvalidated

Toledo Continuous and SICS have never seen real hardware. Both are written from documented structure. They return explicit errors rather than guesses on malformed frames, and the monitor shows the raw hex needed to correct them. Expect the first real Toledo to need a tweak — most likely the SWA decimal-point table or a SWB bit position.