de8370481f
- Update COMPONENT_AUDIT.md: replace htmx reference with React SPA - Add server.read_only to config.example.yaml and CONFIGURATION.md
18 KiB
Component Audit Tool
Last Updated: 2026-02-01 Status: Design
Problem
The parts database has grown organically. Many items were created with only a part number, description, and category. The property schema defines dozens of fields per category (material, finish, manufacturer, supplier, cost, etc.) but most items have few or none of these populated. There is no way to see which items are missing data or to prioritize what needs filling in.
Currently, adding or updating properties requires either:
- Editing each item individually through the web UI detail panel
- Bulk CSV export, manual editing, re-import
- The Calc extension (new, not yet widely used)
None of these approaches give visibility into what's missing across the database. Engineers don't know which items need attention until they encounter a blank field during a design review or procurement cycle.
Goals
- Show a per-item completeness score based on the property schema
- Surface the least-complete items so they can be prioritized
- Let users fill in missing fields directly from the audit view
- Filter by project, category, completeness threshold
- Track improvement over time
Design
The audit tool is a page in the web UI (/audit), built with the React SPA
(same architecture as the items, projects, and schemas pages). It adds one
new API endpoint for the completeness data and reuses existing endpoints for
updates.
Completeness Scoring
Each item's completeness is computed against its category's property schema. The schema defines both global defaults (12 fields, all categories) and category-specific properties (varies: 9 fields for fasteners, 20+ for motion components, etc.).
Score formula:
score = sum(weight for each filled field) / sum(weight for all applicable fields)
Score is 0.0 to 1.0, displayed as a percentage. Fields are weighted differently depending on sourcing type.
Purchased parts (sourcing_type = "purchased"):
| Weight | Fields | Rationale |
|---|---|---|
| 3 | manufacturer_pn, sourcing_link | Can't procure without these |
| 2 | manufacturer, supplier, supplier_pn, standard_cost | Core procurement data |
| 1 | description, sourcing_type, lead_time_days, minimum_order_qty, lifecycle_status | Important but less blocking |
| 1 | All category-specific properties | Engineering detail |
| 0.5 | rohs_compliant, country_of_origin, notes, long_description | Nice to have |
Manufactured parts (sourcing_type = "manufactured"):
| Weight | Fields | Rationale |
|---|---|---|
| 3 | has_bom (at least one BOM child) | Can't manufacture without a BOM |
| 2 | description, standard_cost | Core identification |
| 1 | All category-specific properties | Engineering detail |
| 0.5 | manufacturer, supplier, notes, long_description | Less relevant for in-house |
The has_bom check for manufactured parts queries the relationships
table for at least one rel_type = 'component' child. This is not a
property field -- it's a structural check. A manufactured part with no BOM
children is flagged as critically incomplete regardless of how many other
fields are filled.
Assemblies (categories A01-A07):
Assembly scores are partially computed from children:
| Field | Source | Notes |
|---|---|---|
| weight | Sum of child weights | Computed if all children have weight |
| standard_cost | Sum of child (cost * qty) | Computed from BOM |
| component_count | Count of BOM children | Always computable |
| has_bom | BOM children exist | Required (weight 3) |
A computed field counts as "filled" if the data needed to compute it is available. If a computed value exists, it is shown alongside the stored value so engineers can verify or override.
Assembly-specific properties that cannot be computed (assembly_time, test_procedure, ip_rating, dimensions) are scored normally.
Field filled criteria:
- String fields: non-empty after trimming
- Number fields: non-null and non-zero
- Boolean fields: non-null (false is a valid answer)
- has_bom: at least one component relationship exists
Item-level fields (description, sourcing_type, sourcing_link,
standard_cost, long_description) are checked on the items table.
Property fields (manufacturer, material, etc.) are checked on the
current revision's properties JSONB column. BOM existence is checked
on the relationships table.
Tiers
Items are grouped into completeness tiers for dashboard display:
| Tier | Range | Color | Label |
|---|---|---|---|
| Critical | 0-25% | Red | Missing critical data |
| Low | 25-50% | Orange | Needs attention |
| Partial | 50-75% | Yellow | Partially complete |
| Good | 75-99% | Light green | Nearly complete |
| Complete | 100% | Green | All fields populated |
API
GET /api/audit/completeness
Returns completeness scores for all items (or filtered subset).
Query parameters:
| Param | Type | Description |
|---|---|---|
project |
string | Filter by project code |
category |
string | Filter by category prefix (e.g. F, F01) |
max_score |
float | Only items below this score (e.g. 0.5) |
min_score |
float | Only items above this score |
sort |
string | score_asc (default), score_desc, part_number, updated_at |
limit |
int | Pagination limit (default 100) |
offset |
int | Pagination offset |
Response:
{
"items": [
{
"part_number": "F01-0042",
"description": "M3x10 Socket Head Cap Screw",
"category": "F01",
"category_name": "Screws and Bolts",
"sourcing_type": "purchased",
"projects": ["3DX10", "PROTO"],
"score": 0.41,
"weighted_filled": 12.5,
"weighted_total": 30.5,
"has_bom": false,
"bom_children": 0,
"missing_critical": ["manufacturer_pn", "sourcing_link"],
"missing": [
"manufacturer_pn",
"sourcing_link",
"supplier",
"supplier_pn",
"finish",
"strength_grade",
"torque_spec"
],
"updated_at": "2026-01-15T10:30:00Z"
},
{
"part_number": "A01-0003",
"description": "3DX10 Line Assembly",
"category": "A01",
"category_name": "Mechanical Assembly",
"sourcing_type": "manufactured",
"projects": ["3DX10"],
"score": 0.68,
"weighted_filled": 15.0,
"weighted_total": 22.0,
"has_bom": true,
"bom_children": 12,
"computed_fields": {
"standard_cost": 7538.61,
"component_count": 12,
"weight": null
},
"missing_critical": [],
"missing": ["assembly_time", "test_procedure", "weight", "ip_rating"],
"updated_at": "2026-01-28T14:20:00Z"
}
],
"summary": {
"total_items": 847,
"avg_score": 0.42,
"manufactured_without_bom": 31,
"by_tier": {
"critical": 123,
"low": 298,
"partial": 251,
"good": 142,
"complete": 33
},
"by_category": {
"F": {"count": 156, "avg_score": 0.51},
"C": {"count": 89, "avg_score": 0.38},
"R": {"count": 201, "avg_score": 0.29}
}
}
}
GET /api/audit/completeness/{partNumber}
Single-item detail with field-by-field breakdown.
{
"part_number": "F01-0042",
"description": "M3x10 Socket Head Cap Screw",
"category": "F01",
"sourcing_type": "purchased",
"score": 0.41,
"has_bom": false,
"fields": [
{"key": "description", "source": "item", "weight": 1, "value": "M3x10 Socket Head Cap Screw", "filled": true},
{"key": "sourcing_type", "source": "item", "weight": 1, "value": "purchased", "filled": true},
{"key": "standard_cost", "source": "item", "weight": 2, "value": 0.12, "filled": true},
{"key": "sourcing_link", "source": "item", "weight": 3, "value": "", "filled": false},
{"key": "manufacturer", "source": "property", "weight": 2, "value": null, "filled": false},
{"key": "manufacturer_pn", "source": "property", "weight": 3, "value": null, "filled": false},
{"key": "supplier", "source": "property", "weight": 2, "value": null, "filled": false},
{"key": "supplier_pn", "source": "property", "weight": 2, "value": null, "filled": false},
{"key": "material", "source": "property", "weight": 1, "value": "18-8 Stainless Steel", "filled": true},
{"key": "finish", "source": "property", "weight": 1, "value": null, "filled": false},
{"key": "thread_size", "source": "property", "weight": 1, "value": "M3", "filled": true},
{"key": "thread_pitch", "source": "property", "weight": 1, "value": null, "filled": false},
{"key": "length", "source": "property", "weight": 1, "value": "10mm", "filled": true},
{"key": "head_type", "source": "property", "weight": 1, "value": "Socket", "filled": true}
]
}
For assemblies, the detail response includes a computed_fields section
showing values derived from children (cost rollup, weight rollup,
component count). These are presented alongside stored values in the UI
so engineers can compare and choose to accept the computed value.
Existing PUT /api/items/{pn} and revision property updates handle writes.
Web UI
Audit Page (/audit)
New page accessible from the top navigation bar (fourth tab after Items, Projects, Schemas).
Layout:
+------------------------------------------------------------------+
| Items | Projects | Schemas | Audit |
+------------------------------------------------------------------+
| [Project: ___] [Category: ___] [Max Score: ___] [Search] |
+------------------------------------------------------------------+
| Summary Bar |
| [===Critical: 123===|===Low: 298===|==Partial: 251==|Good|Done] |
+------------------------------------------------------------------+
| Score | PN | Description | Category | Missing|
|-------|-----------|--------------------------|----------|--------|
| 12% | R01-0003 | Bearing, Deep Groove 6205| Bearings | 18 |
| 15% | E14-0001 | NTC Thermistor 10K | Sensors | 16 |
| 23% | C03-0012 | 1/4" NPT Ball Valve SS | Valves | 14 |
| 35% | F01-0042 | M3x10 Socket Head Cap | Screws | 7 |
| ... | | | | |
+------------------------------------------------------------------+
Interactions:
- Click a row to open an inline edit panel (right side, same split-panel pattern as the items page)
- The edit panel shows all applicable fields for the category, with empty fields highlighted
- Editing a field and pressing Enter/Tab saves immediately via API
- Score updates live after each save
- Summary bar updates as items are completed
- Click a tier segment in the summary bar to filter to that tier
Inline Edit Panel
+----------------------------------+
| F01-0042 Score: 35% |
| M3x10 Socket Head Cap Screw |
+----------------------------------+
| -- Required -- |
| Description [M3x10 Socket H..] |
| Sourcing [purchased v ] |
+----------------------------------+
| -- Procurement -- |
| Manufacturer [________________] |
| Mfr PN [________________] |
| Supplier [________________] |
| Supplier PN [________________] |
| Cost [$0.12 ] |
| Sourcing Link[________________] |
| Lead Time [____ days ] |
+----------------------------------+
| -- Fastener Properties -- |
| Material [18-8 Stainless ] |
| Finish [________________] |
| Thread Size [M3 ] |
| Thread Pitch [________________] |
| Length [10mm ] |
| Head Type [Socket ] |
| Drive Type [________________] |
| Strength [________________] |
| Torque Spec [________________] |
+----------------------------------+
| [Save All] |
+----------------------------------+
Fields are grouped into sections: Required, Procurement (global defaults), and category-specific properties. Empty fields have a subtle red left border. Filled fields have a green left border. The score bar at the top updates as fields are filled in.
Implementation Plan
Phase 1: API endpoint and scoring engine
New file: internal/api/audit_handlers.go
HandleAuditCompleteness-- query items, join current revision properties, compute scores against schema, return paginated JSONHandleAuditItemDetail-- single item with field-by-field breakdown- Scoring logic in a helper function that takes item fields + revision properties + category schema and returns score + missing list
Register routes:
GET /api/audit/completeness(viewer role)GET /api/audit/completeness/{partNumber}(viewer role)
Phase 2: Web UI page
New template: internal/api/templates/audit.html
- Same base template, Catppuccin Mocha theme, nav bar with Audit tab
- Summary bar with tier counts (colored segments)
- Sortable, filterable table
- Split-panel detail view on row click
- Vanilla JS fetch calls to audit and item update endpoints
Update internal/api/web.go:
- Add
HandleAuditPagehandler - Register
GET /auditroute
Update internal/api/templates/base.html:
- Add Audit tab to navigation
Phase 3: Inline editing
- Field save on blur/Enter via
PUT /api/items/{pn}for item fields - Property updates via
POST /api/items/{pn}/revisionswith updated properties map - Live score recalculation after save (re-fetch from audit detail endpoint)
- Batch "Save All" button for multiple field changes
Phase 4: Tracking and reporting
- Store periodic score snapshots (daily cron or on-demand) in a new
audit_snapshotstable for trend tracking - Dashboard chart showing completeness improvement over time
- Per-project completeness summary on the projects page
- CSV export of audit results for offline review
Phase 5: Batch AI assistance
Server-side OpenRouter integration for bulk property inference from existing sourcing data. This extends the Calc extension's AI client pattern to the backend.
Workflow:
- Audit page shows items with sourcing links but missing properties
- Engineer selects items (or filters to a category/project) and clicks "AI Fill Properties"
- Server fetches each item's sourcing link page content (or uses the seller description from the item's metadata)
- OpenRouter API call per item: system prompt describes the category's property schema, user prompt provides the scraped/stored description
- AI returns structured JSON with suggested property values
- Results shown in a review table: item, field, current value, suggested value, confidence indicator
- Engineer checks/unchecks suggestions, clicks "Apply Selected"
- Server writes accepted values as property updates (new revision)
AI prompt structure:
System: You are a parts data specialist. Given a product description
and a list of property fields with types, extract values for as many
fields as possible. Return JSON only.
User:
Category: F01 (Screws and Bolts)
Product: {seller_description or scraped page text}
Fields to extract:
- material (string): Material specification
- finish (string): Surface finish
- thread_size (string): Thread size designation
- thread_pitch (string): Thread pitch
- length (string): Fastener length with unit
- head_type (string): Head style
- drive_type (string): Drive type
- strength_grade (string): Strength/property class
Rate limiting: Queue items and process in batches of 10 with 1s delay between batches to stay within OpenRouter rate limits. Show progress bar in the UI.
Cost control: Use openai/gpt-4.1-nano by default (cheapest). Show
estimated cost before starting batch. Allow model override in settings.
Database Changes
Phase 1: None
Completeness is computed at query time from existing items +
revisions.properties data joined against the in-memory schema definition.
No new tables needed for the core feature.
Phase 4: New table
CREATE TABLE IF NOT EXISTS audit_snapshots (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
captured_at TIMESTAMPTZ NOT NULL DEFAULT now(),
total_items INTEGER NOT NULL,
avg_score DECIMAL(5,4) NOT NULL,
by_tier JSONB NOT NULL,
by_category JSONB NOT NULL,
by_project JSONB NOT NULL
);
Phase 5: None
AI suggestions are ephemeral (computed per request, not stored). Accepted suggestions are written through the existing revision/property update path.
Scoring Examples
Purchased Fastener (F01)
Weighted total: ~30.5 points
| Field | Weight | Filled? | Points |
|---|---|---|---|
| manufacturer_pn | 3 | no | 0/3 |
| sourcing_link | 3 | no | 0/3 |
| manufacturer | 2 | no | 0/2 |
| supplier | 2 | no | 0/2 |
| supplier_pn | 2 | no | 0/2 |
| standard_cost | 2 | yes | 2/2 |
| description | 1 | yes | 1/1 |
| sourcing_type | 1 | yes | 1/1 |
| material | 1 | yes | 1/1 |
| thread_size | 1 | yes | 1/1 |
| length | 1 | yes | 1/1 |
| head_type | 1 | yes | 1/1 |
| drive_type | 1 | no | 0/1 |
| finish | 1 | no | 0/1 |
| ... (remaining) | 0.5-1 | no | 0/... |
Score: 8/30.5 = 26% -- "Low" tier, flagged because weight-3 fields (manufacturer_pn, sourcing_link) are missing.
Manufactured Assembly (A01)
Weighted total: ~22 points
| Field | Weight | Source | Points |
|---|---|---|---|
| has_bom | 3 | BOM query | 3/3 (12 children) |
| description | 2 | item | 2/2 |
| standard_cost | 2 | computed from children | 2/2 |
| component_count | 1 | computed (= 12) | 1/1 |
| weight | 1 | computed (needs children) | 0/1 (not all children have weight) |
| assembly_time | 1 | property | 0/1 |
| test_procedure | 1 | property | 0/1 |
| dimensions | 1 | property | 0/1 |
| ip_rating | 1 | property | 0/1 |
| ... (globals) | 0.5-1 | property | .../... |
Score: ~15/22 = 68% -- "Partial" tier, mostly complete because BOM and cost are covered through children.
Motor (R01) -- highest field count
30+ applicable fields across global defaults + motion-specific properties (load, speed, power, voltage, current, torque, encoder, gear ratio...). A motor with only description + cost + sourcing_type scores under 10% because of the large denominator. Motors are the category most likely to benefit from batch AI extraction from datasheets.