kindred/silo
1
1
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fc0eb6d2be5d98fca6a289691245e175a894f86d
silo/ROADMAP.md
Forbes f39aef0fc7 update documentation and specs
2026-01-29 13:10:12 -06:00

500 lines
21 KiB
Markdown
Raw Blame History

# Silo Roadmap
**Version:** 1.0
**Date:** January 2026
**Purpose:** Project inventory, SOLIDWORKS PDM gap analysis, and development roadmap
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Current Project Inventory](#current-project-inventory)
3. [SOLIDWORKS PDM Gap Analysis](#solidworks-pdm-gap-analysis)
4. [Feature Roadmap](#feature-roadmap)
5. [Implementation Phases](#implementation-phases)
---
## Executive Summary
Silo is an R&D-oriented item database and part management system designed for FreeCAD integration. It provides configurable part number generation, revision tracking, BOM management, and file versioning through MinIO storage.
This document compares Silo's current capabilities against SOLIDWORKS PDM—the industry-leading product data management solution—to identify gaps and prioritize future development.
### Key Differentiators
| Aspect | Silo | SOLIDWORKS PDM |
|--------|------|----------------|
| **Target CAD** | FreeCAD / Kindred Create (open source) | SOLIDWORKS (proprietary) |
| **Part Numbering** | Schema-as-configuration (YAML) | Fixed format with some customization |
| **Licensing** | Open source / Kindred Proprietary | Commercial ($3,000-$10,000+ per seat) |
| **Storage** | PostgreSQL + MinIO (S3-compatible) | SQL Server + File Archive |
| **Philosophy** | R&D-oriented, lightweight | Enterprise-grade, comprehensive |
---
## Current Project Inventory
### Implemented Features (MVP Complete)
#### Core Database System
- PostgreSQL schema with 7 migrations
- UUID-based identifiers throughout
- Soft delete support via `archived_at` timestamps
- Atomic sequence generation for part numbers
#### Part Number Generation
- YAML schema parser with validation
- Segment types: `string`, `enum`, `serial`, `constant`
- Scope templates for serial counters (e.g., `{category}`, `{project}`)
- Format templates for custom output
#### Item Management
- Full CRUD operations for items
- Item types: part, assembly, drawing, document, tooling, purchased, electrical, software
- Custom properties via JSONB storage
- Project tagging with many-to-many relationships
#### Revision Control
- Append-only revision history
- Revision metadata: properties, file reference, checksum, comment
- Status tracking: draft, review, released, obsolete
- Labels/tags per revision
- Revision comparison (diff)
- Rollback functionality
#### File Management
- MinIO integration with versioning
- File upload/download via REST API
- SHA256 checksums for integrity
- Storage path: `items/{partNumber}/rev{N}.FCStd`
#### Bill of Materials (BOM)
- Relationship types: component, alternate, reference
- Reference designators for electronics
- Quantity tracking with units
- Revision-specific child linking
#### Project Management
- Project CRUD operations
- Unique project codes (2-10 characters)
- Item-to-project tagging
- Project-filtered queries
#### Data Import/Export
- CSV export with configurable properties
- CSV import with dry-run validation
- Template generation for import formatting
#### API & Web Interface
- Comprehensive REST API (30+ endpoints)
- Middleware: logging, CORS, recovery, request ID
- Web UI foundation with htmx
- Health and readiness probes
#### Configuration
- YAML configuration with environment variable overrides
- Multi-schema support
- Docker Compose deployment ready
### Partially Implemented
| Feature | Status | Notes |
|---------|--------|-------|
| FreeCAD Workbench | ~80% | 8 commands implemented (save, commit, pull, push, info, register, open, browse), needs end-to-end testing |
| Date segment type | Not started | Schema parser placeholder exists |
| Part number validation | Not started | API accepts but doesn't validate format |
| Location hierarchy CRUD | Schema only | Tables exist, no API endpoints |
| Inventory tracking | Schema only | Tables exist, no API endpoints |
| Unit tests | Not started | Critical for production use |
### Infrastructure Status
| Component | Status |
|-----------|--------|
| PostgreSQL | Running (psql.kindred.internal) |
| MinIO | Configured in Docker Compose |
| Silo API Server | Builds successfully |
| Docker Compose | Complete (dev and production) |
| systemd service | Unit file and env template ready |
| Deployment scripts | setup-host, deploy, init-db, setup-ipa-nginx |
---
## SOLIDWORKS PDM Gap Analysis
This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps are categorized by priority and implementation complexity.
### Legend
- **Silo Status:** Full / Partial / None
- **Priority:** Critical / High / Medium / Low
- **Complexity:** Simple / Moderate / Complex
---
### 1. Version Control & Revision Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Check-in/check-out | Full pessimistic locking | None | High | Moderate |
| Version history | Complete with branching | Full (linear) | - | - |
| Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - |
| Rollback/restore | Full | Full | - | - |
| Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex |
| Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple |
**Gap Analysis:**
Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization.
---
### 2. Workflow Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Custom workflows | Full visual designer | None | Critical | Complex |
| State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex |
| Parallel approvals | Multiple approvers required | None | High | Complex |
| Automatic transitions | Timer/condition-based | None | Medium | Moderate |
| Email notifications | On state change | None | High | Moderate |
| ECO process | Built-in change management | None | High | Complex |
| Child state conditions | Block parent if children invalid | None | Medium | Moderate |
**Gap Analysis:**
Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes.
---
### 3. User Management & Security
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| User authentication | Windows AD, LDAP | None (single-user) | Critical | Moderate |
| Role-based permissions | Granular per folder/state | None | Critical | Complex |
| Group management | Full | None | Critical | Moderate |
| Folder permissions | Read/write/delete per folder | None | High | Moderate |
| State permissions | Actions allowed per state | None | High | Moderate |
| Audit trail | Complete action logging | None | High | Moderate |
| Private files | Pre-check-in visibility control | None | Low | Simple |
**Gap Analysis:**
Silo currently operates as single-user with no authentication. Multi-user deployment requires authentication (LDAP planned), authorization (role-based access), and audit logging. This is prerequisite for any team environment.
---
### 4. Search & Discovery
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Metadata search | Full with custom cards | Partial (API query params) | High | Moderate |
| Full-text content search | iFilters for Office, CAD | None | Medium | Complex |
| Quick search | Toolbar with history | None (API only) | Medium | Simple |
| Saved searches | User-defined favorites | None | Medium | Simple |
| Advanced operators | AND, OR, NOT, wildcards | None | Medium | Simple |
| Multi-variable search | Search across multiple fields | None | Medium | Simple |
| Where-used search | Find all assemblies using part | None | High | Moderate |
**Gap Analysis:**
Silo has basic API-level filtering but lacks a rich search interface. Content search (searching within CAD files) would require FreeCAD file parsing. "Where-used" queries are particularly valuable for impact analysis.
---
### 5. BOM Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Single-level BOM | Yes | Full | - | - |
| Multi-level BOM | Indented/exploded views | Schema only (no API) | High | Moderate |
| BOM comparison | Between revisions | None | Medium | Moderate |
| BOM export | Excel, XML, ERP formats | None | High | Simple |
| Calculated BOMs | Quantities rolled up | None | Medium | Moderate |
| Reference designators | Full support | Full | - | - |
| Alternate parts | Substitute tracking | Full | - | - |
**Gap Analysis:**
BOM structure exists in the database but API endpoints for multi-level BOM retrieval and export are missing. BOM comparison between revisions would be valuable for change impact analysis.
---
### 6. CAD Integration
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Native CAD add-in | Deep SOLIDWORKS integration | FreeCAD workbench (partial) | High | Complex |
| Property mapping | Bi-directional sync | Planned | High | Moderate |
| Task pane | Embedded in CAD UI | None | Medium | Complex |
| Lightweight components | Handle without full load | N/A for FreeCAD | - | - |
| Drawing/model linking | Automatic association | Manual | Medium | Moderate |
| Multi-CAD support | Third-party formats | FreeCAD only | Low | - |
**Gap Analysis:**
FreeCAD workbench commands exist but need thorough testing. Property synchronization between Silo database and FreeCAD document properties is planned but not implemented.
---
### 7. External Integrations
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| ERP integration | SAP, Dynamics, etc. | None | Medium | Complex |
| API access | Full COM/REST API | Full REST API | - | - |
| Dispatch scripts | Automation without coding | None | Medium | Moderate |
| Task scheduler | Background processing | None | Medium | Moderate |
| Email system | SMTP integration | None | High | Simple |
| Web portal | Browser access | Partial (basic UI) | High | Moderate |
**Gap Analysis:**
Silo has a solid REST API foundation. ERP integration can be built on this API but would require specific adapters per target system. Email notifications and scheduled tasks would enhance workflow automation.
---
### 8. Reporting & Analytics
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Standard reports | Inventory, usage, activity | None | Medium | Moderate |
| Custom reports | User-defined queries | None | Medium | Moderate |
| Dashboard | Visual KPIs | None | Low | Moderate |
| Export formats | PDF, Excel, CSV | CSV only | Medium | Simple |
**Gap Analysis:**
Reporting capabilities are absent. Basic reports (item counts, revision activity, where-used) would provide immediate value.
---
### 9. File Handling
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| File versioning | Automatic | Full (MinIO) | - | - |
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
| File conversion | PDF, DXF generation | None | Medium | Complex |
| Replication | Multi-site sync | None | Low | Complex |
| File copy with refs | Copy tree with references | None | Medium | Moderate |
**Gap Analysis:**
File storage works well. Thumbnail generation and file preview would significantly improve the web UI experience. Automatic conversion to PDF/DXF is valuable for sharing with non-CAD users.
---
### Gap Summary by Priority
#### Critical Gaps (Required for Team Use)
1. **User authentication** - LDAP/SSO integration
2. **Role-based permissions** - Folder and state-level access control
3. **Workflow engine** - State machines with transitions and approvals
4. **Check-out locking** - Pessimistic locking for CAD files
#### High Priority Gaps (Significant Value)
1. **Email notifications** - Alert users on state changes
2. **Where-used search** - Impact analysis for changes
3. **Multi-level BOM API** - Retrieve full assembly structure
4. **BOM export** - Excel/CSV/XML for downstream systems
5. **Audit trail** - Log all user actions
6. **Web UI search** - User-friendly search interface
#### Medium Priority Gaps (Nice to Have)
1. **Saved searches** - Frequently used queries
2. **File preview/thumbnails** - Visual browsing
3. **Reporting** - Activity and inventory reports
4. **Scheduled tasks** - Background automation
5. **BOM comparison** - Revision diff for assemblies
---
## Feature Roadmap
### Phase 1: Foundation (Current - Q2 2026)
*Complete MVP and stabilize core functionality*
| Feature | Description | Status |
|---------|-------------|--------|
| MinIO integration | File upload/download with versioning and checksums | Complete |
| FreeCAD workbench | 8 toolbar commands implemented | Needs Testing |
| Revision control | Rollback, comparison, status/labels | Complete |
| CSV import/export | Dry-run validation, template generation | Complete |
| Project management | CRUD, many-to-many item tagging | Complete |
| Unit tests | Core API and database operations | Not Started |
| Date segment type | Support date-based part number segments | Not Started |
| Part number validation | Validate format on creation | Not Started |
| Location CRUD API | Expose location hierarchy via REST | Not Started |
| Inventory API | Expose inventory operations via REST | Not Started |
### Phase 2: Multi-User (Q2-Q3 2026)
*Enable team collaboration*
| Feature | Description | Complexity |
|---------|-------------|------------|
| LDAP authentication | Integrate with FreeIPA/Active Directory | Moderate |
| User/group management | Create, assign, manage users and groups | Moderate |
| Folder permissions | Read/write/delete per folder hierarchy | Moderate |
| Check-out locking | Pessimistic locks with timeout | Moderate |
| Audit logging | Record all user actions with timestamps | Moderate |
| Session management | Token-based API authentication | Moderate |
### Phase 3: Workflow Engine (Q3-Q4 2026)
*Implement engineering change processes*
| Feature | Description | Complexity |
|---------|-------------|------------|
| Workflow designer | YAML-defined state machines | Complex |
| State transitions | Configurable transition rules | Complex |
| Transition permissions | Who can execute which transitions | Moderate |
| Single approvals | Basic approval workflow | Moderate |
| Parallel approvals | Multi-approver gates | Complex |
| Automatic transitions | Timer and condition-based | Complex |
| Email notifications | SMTP integration for alerts | Simple |
| Child state conditions | Block parent transitions | Moderate |
### Phase 4: Search & Discovery (Q4 2026 - Q1 2027)
*Improve findability and navigation*
| Feature | Description | Complexity |
|---------|-------------|------------|
| Advanced search UI | Web interface with filters | Moderate |
| Search operators | AND, OR, NOT, wildcards | Simple |
| Saved searches | User favorites | Simple |
| Where-used queries | Find parent assemblies | Moderate |
| Quick search | Toolbar search box | Simple |
| Content search | Search within file content | Complex |
### Phase 5: BOM & Reporting (Q1-Q2 2027)
*Enhanced BOM management and analytics*
| Feature | Description | Complexity |
|---------|-------------|------------|
| Multi-level BOM API | Recursive assembly retrieval | Moderate |
| BOM export | Excel, CSV, XML formats | Simple |
| BOM comparison | Diff between revisions | Moderate |
| Standard reports | Activity, inventory, usage | Moderate |
| Custom queries | User-defined report builder | Moderate |
| Dashboard | Visual KPIs and metrics | Moderate |
### Phase 6: Advanced Features (Q2-Q4 2027)
*Enterprise capabilities*
| Feature | Description | Complexity |
|---------|-------------|------------|
| File preview | Thumbnail generation | Complex |
| File conversion | Auto-generate PDF/DXF | Complex |
| ERP integration | Adapter framework | Complex |
| Task scheduler | Background job processing | Moderate |
| Webhooks | Event notifications to external systems | Moderate |
| API rate limiting | Protect against abuse | Simple |
---
## Implementation Phases
### Phase 1 Detailed Tasks
#### 1.1 MinIO Integration -- COMPLETE
- [x] MinIO service configured in Docker Compose
- [x] File upload via REST API
- [x] File download via REST API (latest and by revision)
- [x] SHA256 checksums on upload
#### 1.2 FreeCAD Workbench -- Needs End-to-End Testing
- [x] Silo_Save, Silo_Commit, Silo_Pull, Silo_Push implemented
- [x] Silo_Info, Silo_Register, Silo_Open, Silo_Browse implemented
- [ ] End-to-end testing with running Silo instance
- [ ] Document workbench installation
#### 1.3 Unit Test Suite
- [ ] Database connection and transaction tests
- [ ] Item CRUD operation tests
- [ ] Revision creation and retrieval tests
- [ ] Part number generation tests
- [ ] File upload/download tests
- [ ] CSV import/export tests
- [ ] API endpoint tests
#### 1.4 Missing Segment Types
- [ ] Implement date segment type
- [ ] Add strftime-style format support
#### 1.5 Location & Inventory APIs
- [ ] `GET /api/locations` - List locations
- [ ] `POST /api/locations` - Create location
- [ ] `GET /api/locations/{path}` - Get location
- [ ] `DELETE /api/locations/{path}` - Delete location
- [ ] `GET /api/inventory/{partNumber}` - Get inventory
- [ ] `POST /api/inventory/{partNumber}/adjust` - Adjust quantity
- [ ] `POST /api/inventory/{partNumber}/move` - Move between locations
---
## Success Metrics
### Phase 1 (Foundation)
- All existing tests pass
- File upload/download works end-to-end
- FreeCAD users can checkout, modify, commit parts
### Phase 2 (Multi-User)
- 5+ concurrent users supported
- No data corruption under concurrent access
- Audit log captures all modifications
### Phase 3 (Workflow)
- Engineering change process completable in Silo
- Email notifications delivered reliably
- Workflow state visible in web UI
### Phase 4+ (Advanced)
- Search returns results in <2 seconds
- Where-used queries complete in <5 seconds
- BOM export matches assembly structure
---
## References
### SOLIDWORKS PDM Documentation
- [SOLIDWORKS PDM Product Page](https://www.solidworks.com/product/solidworks-pdm)
- [What's New in SOLIDWORKS PDM 2025](https://blogs.solidworks.com/solidworksblog/2024/10/whats-new-in-solidworks-pdm-2025.html)
- [Top 5 Enhancements in SOLIDWORKS PDM 2024](https://blogs.solidworks.com/solidworksblog/2023/10/top-5-enhancements-in-solidworks-pdm-2024.html)
- [SOLIDWORKS PDM Workflow Transitions](https://help.solidworks.com/2023/english/EnterprisePDM/Admin/c_workflow_transition.htm)
- [Ultimate Guide to SOLIDWORKS PDM Permissions](https://www.goengineer.com/blog/ultimate-guide-to-solidworks-pdm-permissions)
- [Searching in SOLIDWORKS PDM](https://help.solidworks.com/2021/english/EnterprisePDM/fileexplorer/c_searches.htm)
- [SOLIDWORKS PDM API Getting Started](https://3dswym.3dexperience.3ds.com/wiki/solidworks-news-info/getting-started-with-the-solidworks-pdm-api-solidpractices_gBCYaM75RgORBcpSO1m_Mw)
### Silo Documentation
- [Specification](docs/SPECIFICATION.md)
- [Development Status](docs/STATUS.md)
- [Deployment Guide](docs/DEPLOYMENT.md)
- [Gap Analysis](docs/GAP_ANALYSIS.md)
---
## Appendix: Feature Comparison Matrix
| Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned |
|----------|---------|-----------------|------------|--------------|--------------|
| **Version Control** | Check-in/out | Yes | Yes | No | Phase 2 |
| | Version history | Yes | Yes | Yes | - |
| | Rollback | Yes | Yes | Yes | - |
| | Revision labels/status | Yes | Yes | Yes | - |
| | Revision comparison | Yes | Yes | Yes (metadata) | - |
| **Workflow** | Custom workflows | Limited | Yes | No | Phase 3 |
| | Parallel approval | No | Yes | No | Phase 3 |
| | Notifications | No | Yes | No | Phase 3 |
| **Security** | User auth | Windows | Windows/LDAP | No | Phase 2 |
| | Permissions | Basic | Granular | No | Phase 2 |
| | Audit trail | Basic | Full | No | Phase 2 |
| **Search** | Metadata search | Yes | Yes | Partial (API) | Phase 4 |
| | Content search | No | Yes | No | Phase 4 |
| | Where-used | Yes | Yes | No | Phase 4 |
| **BOM** | Single-level | Yes | Yes | Yes | - |
| | Multi-level | Yes | Yes | Schema only | Phase 5 |
| | BOM export | Yes | Yes | No | Phase 5 |
| **Data** | CSV import/export | Yes | Yes | Yes | - |
| | Project management | Yes | Yes | Yes | - |
| **Integration** | API | Limited | Full | Full REST (35+) | - |
| | ERP connectors | No | Yes | No | Phase 6 |
| | Web access | No | Yes | Yes (htmx) | Phase 4 |
| **Files** | Versioning | Yes | Yes | Yes | - |
| | Preview | Yes | Yes | No | Phase 6 |
| | Multi-site | No | Yes | No | Not Planned |
Reference in New Issue View Git Blame Copy Permalink