kindred/silo
1
1
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
test-coverage-batch
silo/docs/ROADMAP.md
Forbes 414a5cf3d6 docs: update roadmap status for completed Tier 0 items 
- Unit test suite: complete (137 tests across 20 files)
- Date segment type: complete (#79)
- Part number validation: complete (#80)
- Link remaining tasks to issue numbers (#81-#91)
- Update Appendix B checklists and Partially Implemented table
2026-02-14 08:54:14 -06:00

446 lines
21 KiB
Markdown
Raw Permalink Blame History

# Silo Platform Roadmap
**Version:** 2.0
**Date:** February 2026
Silo is the server component of the Kindred ecosystem. Its core function is storing and version-controlling engineering data (parts, assemblies, BOMs). This roadmap describes the expansion of Silo from a PDM server into a modular platform -- comparable to how Gitea/GitHub extend Git hosting with Actions, Wikis, Packages, and webhooks.
For a detailed comparison against SOLIDWORKS PDM, see [GAP_ANALYSIS.md](GAP_ANALYSIS.md).
---
## Guiding Principles
- **Modular architecture.** Every capability beyond core PDM is a module. Modules register against a central API endpoint registry and declare their menu entries, views, dependencies, and routes via a module manifest.
- **Odoo-aligned UX.** The web UI follows Odoo's navigation patterns: a top-level app launcher grid, breadcrumb navigation (`Module > List > Record > Sub-view`), and standard view types (list, form, kanban, calendar, pivot). This alignment provides a familiar experience for shops already using Odoo as their ERP, and a clean integration path for those who adopt it later.
- **Open by default.** Silo and all modules are open-source. Enterprise customers can fork, extend, and self-host. Developer tools for building and distributing custom Create forks are available to everyone, not just Kindred.
- **Odoo as reference ERP.** For shops on Odoo, a bridge module syncs Silo data to Odoo models (`mrp.bom`, `mrp.production`, `quality.check`, etc.). For shops on other ERPs, the open API serves as a documented integration surface. Silo's web UI is fully self-sufficient with no ERP dependency required.
---
## Foundational Contracts
### The .kc File Format
Silo introduces the `.kc` file format as an enhanced superset of FreeCAD's `.fcstd`. Both are ZIP bundles. A `.kc` file contains everything an `.fcstd` does, plus a `silo/` directory with platform metadata.
#### Standard FCStd contents (preserved as-is)
- `Document.xml`, `GuiDocument.xml`
- BREP geometry files (`.brp`)
- `thumbnails/`
#### Added .kc entries
| Path | Purpose |
|------|---------|
| `silo/manifest.json` | Silo instance origin, part UUID, revision hash, .kc schema version |
| `silo/metadata.json` | Custom schema field values, tags, lifecycle state |
| `silo/history.json` | Local revision log (lightweight; full history is server-side) |
| `silo/approvals.json` | ECO/approval state snapshot |
| `silo/dependencies.json` | Assembly link references by Silo UUID (not filepath) |
| `silo/macros/` | Embedded macro references or inline scripts bound to this part |
| `silo/inspection/` | GD&T annotations, tolerance data, CMM linkage metadata |
| `silo/thumbnails/` | Silo-generated renderings (separate from FreeCAD's built-in thumbnail) |
#### Interoperability
- **FCStd -> Silo:** On import, the `silo/` directory is generated with defaults. A UUID is assigned and the user is prompted for schema fields.
- **Silo -> FCStd:** On export, the `silo/` directory is stripped. The remaining contents are a valid `.fcstd`.
- **Round-trip safety:** FreeCAD ignores the `silo/` directory on save, so there is no risk of FreeCAD corrupting Silo metadata.
- **Schema versioning:** `silo/manifest.json` carries a format version for forward-compatible migrations.
### Module Manifest
Each module ships a manifest declaring its integration surface:
```
id, name, version, description
dependencies (other module IDs)
menu_entries (app launcher icon, label, route)
view_declarations (list, form, kanban, etc.)
api_routes (REST endpoints the module registers)
hooks (events the module listens to or emits)
permissions (required roles/scopes)
```
The exact format (JSON, TOML, or Python-based a la Odoo's `__manifest__.py`) is TBD. The contract is: a module is anything that provides a valid manifest and registers against the endpoint registry.
### Web UI Shell
The Silo web application provides the chrome that all modules render within.
- **App launcher:** Top-level grid of installed module icons. Driven by the API endpoint registry -- only enabled modules appear. Disabled modules show greyed with an "Enable" action for discoverability.
- **Breadcrumbs:** Every view follows `Module > List > Record > Sub-view`. Consistent across all modules.
- **View types:** List, form, kanban, calendar, pivot/reporting. Modules declare supported views in their manifest.
- **Schema-driven forms:** The user-customizable schema engine maps directly to form views, enabling end-users to define part metadata fields through the web UI without code changes.
---
## Dependency Tiers
Modules are organized into tiers based on what they depend on. Lower tiers must be stable before higher tiers are built.
### Tier 0 -- Foundation
Everything depends on these. They define what Silo *is*.
| Component | Description | Status |
|-----------|-------------|--------|
| **Core Silo** | Part/assembly storage, version control, auth, base REST API | Complete |
| **.kc Format Spec** | File format contract between Create and Silo | Not Started |
| **API Endpoint Registry** | Module discovery, dynamic UI rendering, health checks | Not Started |
| **Web UI Shell** | App launcher, breadcrumbs, view framework, module rendering | Partial |
| **Python Scripting Engine** | Server-side hook execution, module extension point | Not Started |
| **Job Queue Infrastructure** | Redis/NATS shared async service for all compute modules | Not Started |
### Tier 1 -- Core Services
Broad downstream dependencies. These should be built early because retrofitting is painful.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Headless Create** | API-driven FreeCAD instance for file manipulation, geometry queries, format conversion, rendering | Core Silo, Job Queue | Not Started |
| **Notifications & Subscriptions** | Per-part watch lists, lifecycle event hooks, webhook delivery | Core Silo, Registry | Not Started |
| **Audit Trail / Compliance** | ITAR, ISO 9001, AS9100 traceability; module-level event journaling | Core Silo | Partial |
### Tier 2 -- File Intelligence & Collaboration
High-visibility features. Mostly low-hanging fruit once Tier 1 is solid.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Intelligent FCStd Diffing** | XML-based structural diff of .kc bundles | Headless Create | Not Started |
| **Thumbnail Generation** | Auto-rendered part/assembly previews | Headless Create | Not Started |
| **Macro Store** | Shared macro library across Create instances | Core Silo, Registry | Not Started |
| **Theme & Addon Manager** | Centralized distribution of UI themes and workbench addons | Core Silo, Registry | Not Started |
| **User-Customizable Schemas** | End-user defined part/form metadata via web UI | Core Silo, Scripting Engine | Not Started |
### Tier 3 -- Compute
Heavy async workloads. All route through the shared job queue.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Batch Jobs (CPU/GPU)** | FEA, CFD, rendering, bulk export | Job Queue, Headless Create | Not Started |
| **AI Broker** | LLM tasks (Ollama), GNN constraint optimization, appearance AI | Job Queue | Not Started |
| **Reporting & Analytics** | Part reuse, revision frequency, compute usage dashboards, cost roll-ups | Audit Trail, Core Silo | Not Started |
### Tier 4 -- Engineering Workflow
Process modules that formalize how engineering work moves through an organization.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Approval / ECO Workflow** | Engineering change orders, multi-stage review gates, digital signatures | Notifications, Audit Trail, Schemas | Not Started |
| **Shop Floor Drawing Distribution** | Controlled push-to-production drawings; web-based appliance displays on the floor | Headless Create, Approval Workflow | Not Started |
| **Import/Export Bridge** | STEP, IGES, 3MF connectors; SOLIDWORKS migration tooling; ERP adapters | Headless Create | Not Started |
| **Multi-tenant / Org Management** | Org boundaries, role-based permissioning, storage quotas | Core Auth, Audit Trail | Not Started |
### Tier 5 -- Manufacturing & Quality
Deep domain modules. Heavy spec work required independent of software dependencies.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **MES Module** | Manufacturing execution -- internal module or bridge to external MES | Approval Workflow, Schemas, Shop Floor Drawings | Not Started |
| **Quality / Tolerance Stackup** | Inspection data ingestion, CMM device linking, statistical tolerance analysis, material mapping | Schemas, Import Bridge | Not Started |
| **Inspection Plan Generator** | Auto-generate CMM programs or inspection checklists from GD&T drawings | Headless Create, Quality Module | Not Started |
| **BIM Inventory / Receiving** | Live facility model with real-time inventory location, explorable in a custom BIM-MES workbench in Create | Custom BIM-MES Workbench, Schemas, Notifications | Not Started |
### Tier 6 -- Platform & Ecosystem
Modules that serve the broader community and long-horizon use cases.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Developer Tools** | Managed Gitea instance for in-house Create fork development; CI/CD to build and distribute fork updates to configured clients | Tier 0-1 stability | Not Started |
| **Digital Twin Sync** | Live sensor data mapped onto BIM/assembly models; operational monitoring | BIM Inventory, Reporting | Not Started |
| **ERP Adapters (Odoo, SAP, etc.)** | Bidirectional sync of parts, BOMs, ECOs, production orders to external ERP | Import/Export Bridge, MES, Schemas | Partial (Odoo stubs) |
---
## Near-Term Priorities
These are the concrete tasks that map to Tier 0 completion and the first steps into Tier 1. They replace the older Phase 1-6 calendar-based timelines.
### Tier 0 Completion
Complete MVP and stabilize core functionality.
| Task | Description | Status |
|------|-------------|--------|
| Unit test suite | Core API, database, partnum, file, CSV/ODS handler tests | Complete (137 tests) |
| Date segment type | Implement `date` segment with strftime-style formatting | Complete (#79) |
| Part number validation | Validate format against schema on creation | Complete (#80) |
| Location CRUD API | Expose location hierarchy via REST | Not Started (#81) |
| Inventory API | Expose inventory operations via REST | Not Started (#82) |
**Success metrics:**
- All existing tests pass
- File upload/download works end-to-end
- FreeCAD users can checkout, modify, commit parts
### Multi-User Enablement
Enable team collaboration (feeds into Tier 1 and Tier 4).
| Task | Description | Status |
|------|-------------|--------|
| Check-out locking | Pessimistic locks with timeout | Not Started (#87) |
| User/group management | Create, assign, manage users and groups | Not Started (#88) |
| Folder permissions | Read/write/delete per folder/project hierarchy | Not Started (#89) |
**Success metrics:**
- 5+ concurrent users supported
- No data corruption under concurrent access
- Audit log captures all modifications
### Workflow Engine
Implement engineering change processes (Tier 4: Approval/ECO Workflow).
| Task | Description | Status |
|------|-------------|--------|
| Workflow designer | YAML-defined state machines | Not Started |
| State transitions | Configurable transition rules with permissions | Not Started |
| Approval workflows | Single and parallel approver gates | Not Started |
| Email notifications | SMTP integration for alerts on state changes | Not Started |
**Success metrics:**
- Engineering change process completable in Silo
- Email notifications delivered reliably
- Workflow state visible in web UI
### Search & Discovery
Improve findability and navigation (Tier 0 Web UI Shell).
| Task | Description | Status |
|------|-------------|--------|
| Advanced search UI | Web interface with filters and operators | Not Started (#90) |
| Saved searches | User-defined query favorites | Not Started (#91) |
**Success metrics:**
- Search returns results in <2 seconds
- Where-used queries complete in <5 seconds
---
## Gap Summary
For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ANALYSIS.md#appendix-c-solidworks-pdm-comparison).
### Completed (Previously Critical/High)
1. ~~User authentication~~ -- local, LDAP, OIDC
2. ~~Role-based permissions~~ -- 3-tier role model (admin/editor/viewer)
3. ~~Audit trail~~ -- audit_log table with completeness scoring
4. ~~Where-used search~~ -- reverse parent lookup API
5. ~~Multi-level BOM API~~ -- recursive expansion with configurable depth
6. ~~BOM export~~ -- CSV and ODS formats
### Critical Gaps (Required for Team Use)
1. **Workflow engine** -- state machines with transitions and approvals
2. **Check-out locking** -- pessimistic locking for CAD files
### High Priority Gaps (Significant Value)
1. **Email notifications** -- alert users on state changes
2. **Web UI search** -- advanced search interface with saved searches
3. **Folder/state permissions** -- granular access control beyond role model
### Medium Priority Gaps (Nice to Have)
1. **Saved searches** -- frequently used queries
2. **File preview/thumbnails** -- visual browsing
3. **Reporting** -- activity and inventory reports
4. **Scheduled tasks** -- background automation
5. **BOM comparison** -- revision diff for assemblies
---
## Priority Notes
- **Headless Create** is the single highest-leverage Tier 1 item. It unblocks diffing, thumbnails, batch export, drawing distribution, and inspection plan generation.
- **Audit Trail** is unglamorous but critical to build early. Retrofitting compliance logging after modules ship is expensive and error-prone.
- **Tier 2** delivers visible, demo-able value quickly -- diffing, thumbnails, and the macro store are features users immediately understand.
- **Tiers 5-6** carry heavy domain complexity. They need detailed specification and industry consultation well before implementation begins.
- The **.kc format** and **module manifest** are the two foundational contracts. Getting these right determines how cleanly everything above them composes.
---
## Open Questions
1. **Module manifest format** -- JSON, TOML, or Python-based? Tradeoffs between simplicity and expressiveness.
2. **.kc thumbnail policy** -- Single canonical thumbnail vs. multi-view renders. Impacts file size and generation cost.
3. **Job queue technology** -- Redis Streams vs. NATS. Redis is already in the stack; NATS offers better pub/sub semantics for event-driven modules.
4. **Headless Create deployment** -- Sidecar container per Silo instance, or pool of workers behind the job queue?
5. **BIM-MES workbench scope** -- How much of FreeCAD BIM is reusable vs. needs to be purpose-built for inventory/facility modeling?
6. **Offline .kc workflow** -- How much of the `silo/` metadata is authoritative when disconnected? Reconciliation strategy on reconnect.
---
## Appendix A: Current Project Inventory
### Implemented Features (MVP Complete)
#### Core Database System
- PostgreSQL schema with 13 migrations
- UUID-based identifiers throughout
- Soft delete support via `archived_at` timestamps
- Atomic sequence generation for part numbers
#### Part Number Generation
- YAML schema parser with validation
- Segment types: `string`, `enum`, `serial`, `constant`
- Scope templates for serial counters (e.g., `{category}`, `{project}`)
- Format templates for custom output
#### Item Management
- Full CRUD operations for items
- Item types: part, assembly, drawing, document, tooling, purchased, electrical, software
- Custom properties via JSONB storage
- Project tagging with many-to-many relationships
#### Revision Control
- Append-only revision history
- Revision metadata: properties, file reference, checksum, comment
- Status tracking: draft, review, released, obsolete
- Labels/tags per revision
- Revision comparison (diff)
- Rollback functionality
#### File Management
- MinIO integration with versioning
- File upload/download via REST API
- SHA256 checksums for integrity
- Storage path: `items/{partNumber}/rev{N}.FCStd`
#### Bill of Materials (BOM)
- Relationship types: component, alternate, reference
- Multi-level BOM (recursive expansion with configurable depth)
- Where-used queries (reverse parent lookup)
- BOM CSV and ODS export/import with cycle detection
- Reference designators for electronics
- Quantity tracking with units
- Revision-specific child linking
#### Project Management
- Project CRUD operations
- Unique project codes (2-10 characters)
- Item-to-project tagging
- Project-filtered queries
#### Data Import/Export
- CSV export with configurable properties
- CSV import with dry-run validation
- ODS spreadsheet import/export (items, BOMs, project sheets)
- Template generation for import formatting
#### API & Web Interface
- REST API with 78 endpoints
- Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak
- Role-based access control (admin > editor > viewer)
- API token management (SHA-256 hashed)
- Session management (PostgreSQL-backed, 24h lifetime)
- CSRF protection (nosurf on web forms)
- Middleware: logging, CORS, recovery, request ID
- Web UI -- React SPA (Vite + TypeScript, Catppuccin Mocha theme)
- Fuzzy search
- Health and readiness probes
#### Audit & Completeness
- Audit logging (database table with user/action/resource tracking)
- Item completeness scoring with weighted fields
- Category-specific property validation
- Tier classification (critical/low/partial/good/complete)
#### Configuration
- YAML configuration with environment variable overrides
- Multi-schema support
- Docker Compose deployment ready
### Partially Implemented
| Feature | Status | Notes |
|---------|--------|-------|
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull sync operations are stubs |
| Date segment type | Complete | strftime-style formatting via Go time layout (#79) |
| Part number validation | Complete | Validates against schema on creation (#80) |
| Location hierarchy CRUD | Schema only | Tables exist, no API endpoints (#81) |
| Inventory tracking | Schema only | Tables exist, no API endpoints (#82) |
| Unit tests | Complete | 137 tests across 20 files covering api, db, ods, partnum, schema packages |
---
## Appendix B: Phase 1 Detailed Tasks
### 1.1 MinIO Integration -- COMPLETE
- [x] MinIO service configured in Docker Compose
- [x] File upload via REST API
- [x] File download via REST API (latest and by revision)
- [x] SHA256 checksums on upload
### 1.2 Authentication & Authorization -- COMPLETE
- [x] Local authentication (bcrypt)
- [x] LDAP/FreeIPA authentication
- [x] OIDC/Keycloak authentication
- [x] Role-based access control (admin/editor/viewer)
- [x] API token management (SHA-256 hashed)
- [x] Session management (PostgreSQL-backed)
- [x] CSRF protection (nosurf)
- [x] Audit logging (database table)
### 1.3 Multi-level BOM & Export -- COMPLETE
- [x] Recursive BOM expansion with configurable depth
- [x] Where-used reverse lookup
- [x] BOM CSV export/import with cycle detection
- [x] BOM ODS export
- [x] ODS item export/import/template
### 1.4 Unit Test Suite -- COMPLETE
- [x] Database connection and transaction tests
- [x] Item CRUD operation tests (including edge cases: duplicate keys, pagination, search)
- [x] Revision creation, retrieval, compare, rollback tests
- [x] Part number generation tests (including date segments, validation)
- [x] File upload/download handler tests
- [x] CSV import/export tests (dry-run, commit, BOM export)
- [x] ODS import/export tests (export, template, project sheet)
- [x] API endpoint tests (revisions, schemas, audit, auth tokens)
- [x] Item file CRUD tests
- [x] BOM handler tests (get, flat, cost, add, delete)
### 1.5 Missing Segment Types -- COMPLETE
- [x] Implement date segment type
- [x] Add strftime-style format support
### 1.6 Location & Inventory APIs
- [ ] `GET /api/locations` - List locations
- [ ] `POST /api/locations` - Create location
- [ ] `GET /api/locations/{path}` - Get location
- [ ] `DELETE /api/locations/{path}` - Delete location
- [ ] `GET /api/inventory/{partNumber}` - Get inventory
- [ ] `POST /api/inventory/{partNumber}/adjust` - Adjust quantity
- [ ] `POST /api/inventory/{partNumber}/move` - Move between locations
---
## Appendix C: References
### SOLIDWORKS PDM Documentation
- [SOLIDWORKS PDM Product Page](https://www.solidworks.com/product/solidworks-pdm)
- [What's New in SOLIDWORKS PDM 2025](https://blogs.solidworks.com/solidworksblog/2024/10/whats-new-in-solidworks-pdm-2025.html)
- [Top 5 Enhancements in SOLIDWORKS PDM 2024](https://blogs.solidworks.com/solidworksblog/2023/10/top-5-enhancements-in-solidworks-pdm-2024.html)
- [SOLIDWORKS PDM Workflow Transitions](https://help.solidworks.com/2023/english/EnterprisePDM/Admin/c_workflow_transition.htm)
- [Ultimate Guide to SOLIDWORKS PDM Permissions](https://www.goengineer.com/blog/ultimate-guide-to-solidworks-pdm-permissions)
- [Searching in SOLIDWORKS PDM](https://help.solidworks.com/2021/english/EnterprisePDM/fileexplorer/c_searches.htm)
- [SOLIDWORKS PDM API Getting Started](https://3dswym.3dexperience.3ds.com/wiki/solidworks-news-info/getting-started-with-the-solidworks-pdm-api-solidpractices_gBCYaM75RgORBcpSO1m_Mw)
### Silo Documentation
- [Specification](SPECIFICATION.md)
- [Development Status](STATUS.md)
- [Deployment Guide](DEPLOYMENT.md)
- [Gap Analysis](GAP_ANALYSIS.md)
Reference in New Issue View Git Blame Copy Permalink