Building 27 Pantry Survey

SHEET 1

Floor plan

Galley room, ~8–9 × 18–20 ft. The monitored surface is one continuous brick wall (top): counter, backstock cabinets, and the cubby unit, ending at the fridge bank. Power exists at both ends. Fridge-bank wall orientation remains marked verify.

SHEET 2

Cubby wall elevation

All detector regions of interest lie on this surface. Two rows of ~22–24 × 13–14 in openings; produce sits in wire baskets, bread in a crate on the counter; those zones need their own occupancy heuristics. Cans are faced 1–3 deep: the camera measures facade fullness, and true stock-out can hide in the cabinets below.

SHEET 3

Camera coverage

Coverage options at the measured standoff. Option A with a 6-column scope is the lowest-risk configuration; option B or a 4K camera covers the full wall. Both power points (Sheet 1) can feed either mount.

SHEET 4

Annotated site frames

Real frames from the site video with feature tags F1–F10 keyed to the trainable-feature inventory below. Green = detector targets and states, amber = confusers and special-case zones, dark = structure reference. Faces blurred / cropped per project privacy rules.

Straight-on view of three cubby bays with annotations: full can facade, sparse cereal bay, wire produce baskets, and the fruit-cutout wall decorations flagged as confusers — Bay structure, straight-on view. The full-state can facade (F1), the sparse-by-design cereal bay (F2), wire-basket produce (F3), and the fruit-shaped wall decoration (F4) directly above the shelves.

Left end of the pantry wall showing the sparse glass case with bare lower shelves, the wall outlet, the Rosarita bay, rice bag stacks, and the bread crate on the counter — Left wall. Bare lower case shelves (F5); wall outlet at counter height (F6); bread crate on the counter, outside the cubby grid (F7).

Right wall bays with Campbells soup and Jif rows, the dark void behind shallow Barilla stacks tagged as facade depth, and a banana basket — Facade depth (F8): a front row reads as stocked while the shelf behind it is empty. The camera measures facade fullness; occupancy estimates carry this limitation.

Freezer interior with a completely bare wire shelf directly above a stocked shelf, LED strip lighting visible — Bare freezer shelf (F9) directly above a stocked shelf under identical lighting. LED strips, reflections, and condensation (F10) motivate excluding cold storage from v1.

SHEET 5

Trainable-feature inventory

Tag	Feature	Role in training
F9, F5	Bare shelf / bare cubby: exposed shelf board or wire with nothing on it	The primary gap class. High-contrast, well-defined edges, consistent with the pooled retail gap data. Eight instances occur in this footage; all are labeled in the v1 set.
F1	Dense can facade: uniform front-faced rows	The clean "full" state. Purchased-in-bulk stock makes this pantry more retail-like than the literature predicted; transfer should be good here.
F8	Facade void: dark depth behind a 1-deep front row	Labeling policy: facade-only for v1 (matches what staff see at a glance) rather than visible-depth voids, applied consistently.
F2	Sparse-by-design stock: 2 boxes in a 24" opening is normal	Why occupancy thresholds must be per-ROI, not global: "2 boxes" is FULL for the cereal bay and LOW for the can bay. Calibrate per cubby at install.
F3	Wire-basket produce: loose fruit in black mesh baskets	Not a gap-detection problem: an emptying basket never exposes shelf board the same way. Give baskets their own ROI + heuristic (basket fill fraction or simple classification) in v2; exclude from v1 gap labels.
F7	On-counter stock: bread crate, bag stacks	Needs its own ROI outside the cubby grid; crate walls partially occlude contents. v2.
F4	Fruit-cutout wall decor: fruit-shaped, fruit-colored, above the shelves	Permanent fixture; a produce-trained model can false-positive here (confirmed in EXP-001). Mitigated by ROI cropping (decor sits above the cubby line); verified in EXP-002.
F10	Glass + LED cold cases: reflections, condensation, cold color temperature	Out of v1 scope. Different exposure regime from the dry wall; monitoring it is its own project phase.
F6	People and hands: client-choice shopping, mid-hours restock	Never a training target. Frames with people are transient occlusion: skip inference when motion is detected, or accept occluded readings and let the persistence filter absorb them. Frames are discarded either way.

v1 training target: the single class gap (bare shelf regions, facade-only policy) inside per-cubby ROIs on the dry-goods wall. Baskets, counter stock, cold cases, and decoration are handled by ROI design rather than model classes; the remainder is deferred to v2 with on-site data.

SHEET 6

Adaptation results (EXP-001 vs EXP-002)

EXP-001 tested the retail-trained detector (0.90 mAP50 on supermarket data) on real pantry imagery; EXP-002 fine-tuned it on 15 hand-labeled frames from this room (7 gap boxes + 8 negatives, ~2 min of training on a laptop) and re-ran the same test on held-out images. Left column = before, right = after.

Before and after on the decor-heavy wall: the retail model fires on wall decoration at 0.70 confidence and on empty counter at 0.35; the fine-tuned model produces zero detections — Figure: decor-heavy wall, held-out negative. Before: false positives on wall decoration (0.70) and counter (0.35). After: no detections.

Before and after on the left wall: the retail model false-alarms on the counter and misses the bare case shelves; the fine-tuned model finds the bare case shelves at 0.29 with no false alarms — Figure: left wall, excluded from training. Before: counter false positive; bare case shelves missed. After: bare shelves detected at 0.29, no false positives.

Before and after on a fully stocked wall screenshot: the retail model fires three times on video player interface elements; the fine-tuned model produces zero detections — Figure: fully stocked wall. Before: three false positives on video-player interface elements, the same dark-region error as the decoration. After: none. Freezer gaps through glass were detected both before (0.45, 0.59) and after (0.41, 0.36) fine-tuning; no regression.

Interpretation: a detector with strong retail metrics produces systematic errors in the pantry; 15 labeled frames from the deployment site corrected every observed failure. The full-scale version of this loop (EXP-003, frames from the installed camera) runs in Phase 2.

SHEET 7

Monitoring loop, demonstrated in simulation

The on-device pipeline (repo: monitor/ + scripts/06_monitor.py) was implemented and exercised on 2026-08-15 using video frames as a stand-in camera. Frames are analyzed in memory and discarded; the only outputs ever written are occupancy numbers, shelf state, and the digest.

Runtime loop. The out-of-distribution guard runs before the detector: frames failing the brightness or scene-match checks produce a "no reading" record instead of an occupancy number, and the affected shelves are reported as unreadable rather than guessed.

Test 1: moving camera (30 walkthrough frames)

1 valid reading, 29 refused
reason: scene mismatch, camera
moved or blocked

CAMERA COULD NOT READ
(check lens/lighting):
  cubby top c1: no recent reading
  cubby top c2: no recent reading
  ...

Calibrated on one viewpoint, then fed 29 frames from other viewpoints. The guard refused all 29 rather than emit occupancy numbers for a scene it was not calibrated on. This is the intended behavior for a bumped or blocked camera.

Test 2: fixed viewpoint (identical-view pair)

Pantry shelf digest - Sat Aug 15

All 13 shelves OK.
Nothing needs attention.

occupancy: 13/13 ROIs = 1.00
(wall fully stocked: correct)

Both frames pass the guard; all 13 regions read fully stocked, matching the ground truth for that wall. The digest renders the steady-state output a healthy pantry should produce.

Deployment status: every software stage between camera and digest now exists and has been exercised. Jetson bring-up reduces to: flash JetPack, build the TensorRT engine, calibrate the guard once on the mounted camera, run the loop.

TABLE 1

Dimension register

Element	Estimate	Basis	Confidence
Cubby opening, interior	~22–24" W × 13–14" H	7–8 × 3.0" cans per row; 12" cereal box + clearance; 5 brick courses	HIGH
Countertop height	36"	standard + person reference	HIGH
Openings per column	2	all frames	HIGH
Shelf-pin pitch on gables	32 mm system	close-up frame	HIGH
Fridge bank width	54" + ~27" ≈ 6.8 ft	SABA standard reach-in sizes	HIGH
Cubby depth	~12–14"	cans 2–3 deep + box depth	MED
Counter-to-cubby gap	~16–21"	brick count disagrees between frames (6–8 courses)	MED
Unit top above floor	~84–85"	stack-up + full-overhead-reach test	MED
Room size	~8–9 × 18–20 ft	fixture chaining, door + fridge yardsticks	MED
Aisle width	~5.5–6.5 ft	room width − counter depth	MED
Column count	8–10	angles disagree (6 / 8 / 9–10 counted)	VERIFY
Fridge-bank wall orientation	far end, exact wall unclear	perspective ambiguity across frames	VERIFY

TABLE 2

Stock presentation as filmed

Format	How it presents	Detector implication
Canned goods	front-faced rows, 7–8 wide, 1–3 deep	closest to retail training data; facade can hide empty depth
Boxes (cereal, pasta)	2–3 upright per opening	sparse-by-design; “2 boxes” is normal, not low
Bagged staples (rice)	bricked flat, 3–4 layers	reads as texture, not objects; gap detection fits
Loose produce	black wire baskets in openings (incl. two-tier)	basket fill ≠ shelf gap; needs its own heuristic
Bread	wooden crate on countertop	on-counter ROI, separate from cubby grid
Cold items	glass-door fridge/freezer, LED-lit	out of prototype scope; reflections + exposure fight

Real empty states captured on film: a bare lower cubby near the entry, a half-empty bean shelf, a completely bare freezer middle shelf, a ~60%-empty upper shelf; genuine pantry-domain gap examples exist in this footage.
Restocking happens mid-open-hours from the cabinets below, with shoppers present - the persistence-based alert design (“low for N hours”) is the right one.

The pantry already measures visits (check-in tablet) and donations (weighed at the desk). Shelf-level stock is not currently measured; this system adds that layer.

TABLE 3

Site-visit verification list

Count cubby columns (8, 9, or 10?) and tape-measure one opening + the counter-to-cubby gap.
Confirm which wall the fridge bank sits on; photograph the fridge–cubby corner.
Test both outlets; ask about mounting on brick (masonry anchors need permission) vs. drywall N wall vs. ceiling grid.
Check for any network drop; confirm none is needed (system runs offline by design).
Shoot one straight-on calibration photo of the wall from each candidate mount point.