diff --git a/docs/src/silo-server/AUTH_USER_GUIDE.md b/docs/src/silo-server/AUTH_USER_GUIDE.md index 6968700b5a..a34dc98e52 100644 --- a/docs/src/silo-server/AUTH_USER_GUIDE.md +++ b/docs/src/silo-server/AUTH_USER_GUIDE.md @@ -38,7 +38,7 @@ API tokens allow the FreeCAD plugin, scripts, and CI pipelines to authenticate w ### Creating a Token (CLI) ```sh -export SILO_API_URL=https://silo.kindred.internal +export SILO_API_URL=https://silo.example.internal export SILO_API_TOKEN=silo_ silo token create --name "CI pipeline" @@ -140,7 +140,7 @@ auth: ldap: enabled: true - url: "ldaps://ipa.kindred.internal" + url: "ldaps://ipa.example.internal" base_dn: "dc=kindred,dc=internal" user_search_dn: "cn=users,cn=accounts,dc=kindred,dc=internal" user_attr: "uid" @@ -170,10 +170,10 @@ auth: oidc: enabled: true - issuer_url: "https://keycloak.kindred.internal/realms/silo" + issuer_url: "https://keycloak.example.internal/realms/silo" client_id: "silo" client_secret: "" # Set via SILO_OIDC_CLIENT_SECRET - redirect_url: "https://silo.kindred.internal/auth/callback" + redirect_url: "https://silo.example.internal/auth/callback" scopes: ["openid", "profile", "email"] admin_role: "silo-admin" editor_role: "silo-editor" @@ -186,7 +186,7 @@ auth: auth: cors: allowed_origins: - - "https://silo.kindred.internal" + - "https://silo.example.internal" ``` ## Environment Variables @@ -254,4 +254,4 @@ UPDATE users SET password_hash = '', is_active = true WHERE usernam - Verify the token is set in FreeCAD preferences or `SILO_API_TOKEN` - Check the API URL points to the correct server -- Test with curl: `curl -H "Authorization: Bearer silo_..." https://silo.kindred.internal/api/items` +- Test with curl: `curl -H "Authorization: Bearer silo_..." https://silo.example.internal/api/items` diff --git a/docs/src/silo-server/DEPLOYMENT.md b/docs/src/silo-server/DEPLOYMENT.md index 04f449e2a3..d6e2e3f64d 100644 --- a/docs/src/silo-server/DEPLOYMENT.md +++ b/docs/src/silo-server/DEPLOYMENT.md @@ -1,5 +1,9 @@ # Silo Production Deployment Guide +> **First-time setup?** See the [Installation Guide](INSTALL.md) for step-by-step +> instructions. This document covers ongoing maintenance and operations for an +> existing deployment. + This guide covers deploying Silo to a dedicated VM using external PostgreSQL and MinIO services. ## Table of Contents @@ -17,7 +21,7 @@ This guide covers deploying Silo to a dedicated VM using external PostgreSQL and ``` ┌─────────────────────────────────────────────────────────────────┐ -│ silo.kindred.internal │ +│ silo.example.internal │ │ ┌───────────────────────────────────────────────────────────┐ │ │ │ silod │ │ │ │ (Silo API Server) │ │ @@ -27,7 +31,7 @@ This guide covers deploying Silo to a dedicated VM using external PostgreSQL and │ │ ▼ ▼ ┌─────────────────────────┐ ┌─────────────────────────────────┐ -│ psql.kindred.internal │ │ minio.kindred.internal │ +│ psql.example.internal │ │ minio.example.internal │ │ PostgreSQL 16 │ │ MinIO S3 │ │ :5432 │ │ :9000 (API) │ │ │ │ :9001 (Console) │ @@ -40,8 +44,8 @@ The following external services are already configured: | Service | Host | Database/Bucket | User | |---------|------|-----------------|------| -| PostgreSQL | psql.kindred.internal:5432 | silo | silo | -| MinIO | minio.kindred.internal:9000 | silo-files | silouser | +| PostgreSQL | psql.example.internal:5432 | silo | silo | +| MinIO | minio.example.internal:9000 | silo-files | silouser | Migrations have been applied to the database. @@ -53,10 +57,10 @@ For a fresh VM, run these commands: ```bash # 1. SSH to the target host -ssh root@silo.kindred.internal +ssh root@silo.example.internal # 2. Download and run setup script -curl -fsSL https://gitea.kindred.internal/kindred/silo-0062/raw/branch/main/scripts/setup-host.sh | bash +curl -fsSL https://git.kindred-systems.com/kindred/silo/raw/branch/main/scripts/setup-host.sh | bash # 3. Configure credentials nano /etc/silo/silod.env @@ -69,16 +73,16 @@ nano /etc/silo/silod.env ## Initial Setup -Run the setup script once on `silo.kindred.internal` to prepare the host: +Run the setup script once on `silo.example.internal` to prepare the host: ```bash # Option 1: If you have the repo locally -scp scripts/setup-host.sh root@silo.kindred.internal:/tmp/ -ssh root@silo.kindred.internal 'bash /tmp/setup-host.sh' +scp scripts/setup-host.sh root@silo.example.internal:/tmp/ +ssh root@silo.example.internal 'bash /tmp/setup-host.sh' # Option 2: Direct on the host -ssh root@silo.kindred.internal -curl -fsSL https://git.kindred.internal/kindred/silo/raw/branch/main/scripts/setup-host.sh -o /tmp/setup-host.sh +ssh root@silo.example.internal +curl -fsSL https://git.kindred-systems.com/kindred/silo/raw/branch/main/scripts/setup-host.sh -o /tmp/setup-host.sh bash /tmp/setup-host.sh ``` @@ -100,10 +104,10 @@ sudo nano /etc/silo/silod.env Fill in the values: ```bash -# Database credentials (psql.kindred.internal) +# Database credentials (psql.example.internal) SILO_DB_PASSWORD=your-database-password -# MinIO credentials (minio.kindred.internal) +# MinIO credentials (minio.example.internal) SILO_MINIO_ACCESS_KEY=silouser SILO_MINIO_SECRET_KEY=your-minio-secret-key ``` @@ -114,10 +118,10 @@ Before deploying, verify connectivity to external services: ```bash # Test PostgreSQL -psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1' +psql -h psql.example.internal -U silo -d silo -c 'SELECT 1' # Test MinIO -curl -I http://minio.kindred.internal:9000/minio/health/live +curl -I http://minio.example.internal:9000/minio/health/live ``` --- @@ -129,7 +133,7 @@ curl -I http://minio.kindred.internal:9000/minio/health/live To deploy or update Silo, run the deploy script on the target host: ```bash -ssh root@silo.kindred.internal +ssh root@silo.example.internal /opt/silo/src/scripts/deploy.sh ``` @@ -165,7 +169,7 @@ sudo /opt/silo/src/scripts/deploy.sh --status You can override the git repository URL and branch: ```bash -export SILO_REPO_URL=https://git.kindred.internal/kindred/silo.git +export SILO_REPO_URL=https://git.kindred-systems.com/kindred/silo.git export SILO_BRANCH=main sudo -E /opt/silo/src/scripts/deploy.sh ``` @@ -247,7 +251,7 @@ curl http://localhost:8080/ready To update to the latest version: ```bash -ssh root@silo.kindred.internal +ssh root@silo.example.internal /opt/silo/src/scripts/deploy.sh ``` @@ -269,7 +273,7 @@ When new migrations are added, run them manually: ls -la /opt/silo/src/migrations/ # Run a specific migration -psql -h psql.kindred.internal -U silo -d silo -f /opt/silo/src/migrations/008_new_feature.sql +psql -h psql.example.internal -U silo -d silo -f /opt/silo/src/migrations/008_new_feature.sql ``` --- @@ -303,13 +307,13 @@ psql -h psql.kindred.internal -U silo -d silo -f /opt/silo/src/migrations/008_ne 1. Test network connectivity: ```bash - nc -zv psql.kindred.internal 5432 + nc -zv psql.example.internal 5432 ``` 2. Test credentials: ```bash source /etc/silo/silod.env - PGPASSWORD=$SILO_DB_PASSWORD psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1' + PGPASSWORD=$SILO_DB_PASSWORD psql -h psql.example.internal -U silo -d silo -c 'SELECT 1' ``` 3. Check `pg_hba.conf` on PostgreSQL server allows connections from this host. @@ -318,12 +322,12 @@ psql -h psql.kindred.internal -U silo -d silo -f /opt/silo/src/migrations/008_ne 1. Test network connectivity: ```bash - nc -zv minio.kindred.internal 9000 + nc -zv minio.example.internal 9000 ``` 2. Test with curl: ```bash - curl -I http://minio.kindred.internal:9000/minio/health/live + curl -I http://minio.example.internal:9000/minio/health/live ``` 3. Check SSL settings in config match MinIO setup: @@ -340,8 +344,8 @@ curl -v http://localhost:8080/health curl -v http://localhost:8080/ready # If ready fails but health passes, check external services -psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1' -curl http://minio.kindred.internal:9000/minio/health/live +psql -h psql.example.internal -U silo -d silo -c 'SELECT 1' +curl http://minio.example.internal:9000/minio/health/live ``` ### Build Fails @@ -391,14 +395,14 @@ This script: getcert list ``` -2. The silo config is already updated to use `https://silo.kindred.internal` as base URL. Restart silo: +2. The silo config is already updated to use `https://silo.example.internal` as base URL. Restart silo: ```bash sudo systemctl restart silod ``` 3. Test the setup: ```bash - curl https://silo.kindred.internal/health + curl https://silo.example.internal/health ``` ### Certificate Management @@ -422,7 +426,7 @@ For clients to trust the Silo HTTPS certificate, they need the IPA CA: ```bash # Download CA cert -curl -o /tmp/ipa-ca.crt https://ipa.kindred.internal/ipa/config/ca.crt +curl -o /tmp/ipa-ca.crt https://ipa.example.internal/ipa/config/ca.crt # Ubuntu/Debian sudo cp /tmp/ipa-ca.crt /usr/local/share/ca-certificates/ipa-ca.crt diff --git a/docs/src/silo-server/GAP_ANALYSIS.md b/docs/src/silo-server/GAP_ANALYSIS.md index a34783341c..bf0c0c7e75 100644 --- a/docs/src/silo-server/GAP_ANALYSIS.md +++ b/docs/src/silo-server/GAP_ANALYSIS.md @@ -365,7 +365,7 @@ internal/ handlers.go # Items, schemas, projects, revisions middleware.go # Auth middleware odoo_handlers.go # Odoo integration endpoints - routes.go # Route registration (75 endpoints) + routes.go # Route registration (78 endpoints) search.go # Fuzzy search auth/ auth.go # Auth service: local, LDAP, OIDC diff --git a/docs/src/silo-server/INSTALL.md b/docs/src/silo-server/INSTALL.md new file mode 100644 index 0000000000..9429928dcd --- /dev/null +++ b/docs/src/silo-server/INSTALL.md @@ -0,0 +1,518 @@ +# Installing Silo + +This guide covers two installation methods: + +- **[Option A: Docker Compose](#option-a-docker-compose)** — self-contained stack with all services. Recommended for evaluation, small teams, and environments where Docker is the standard. +- **[Option B: Daemon Install](#option-b-daemon-install-systemd--external-services)** — systemd service with external PostgreSQL, MinIO, and optional LDAP/nginx. Recommended for production deployments integrated with existing infrastructure. + +Both methods produce the same result: a running Silo server with a web UI, REST API, and authentication. + +--- + +## Table of Contents + +- [Prerequisites](#prerequisites) +- [Option A: Docker Compose](#option-a-docker-compose) + - [A.1 Prerequisites](#a1-prerequisites) + - [A.2 Clone the Repository](#a2-clone-the-repository) + - [A.3 Run the Setup Script](#a3-run-the-setup-script) + - [A.4 Start the Stack](#a4-start-the-stack) + - [A.5 Verify the Installation](#a5-verify-the-installation) + - [A.6 LDAP Users and Groups](#a6-ldap-users-and-groups) + - [A.7 Optional: Enable Nginx Reverse Proxy](#a7-optional-enable-nginx-reverse-proxy) + - [A.8 Stopping, Starting, and Upgrading](#a8-stopping-starting-and-upgrading) +- [Option B: Daemon Install (systemd + External Services)](#option-b-daemon-install-systemd--external-services) + - [B.1 Architecture Overview](#b1-architecture-overview) + - [B.2 Prerequisites](#b2-prerequisites) + - [B.3 Set Up External Services](#b3-set-up-external-services) + - [B.4 Prepare the Host](#b4-prepare-the-host) + - [B.5 Configure Credentials](#b5-configure-credentials) + - [B.6 Deploy](#b6-deploy) + - [B.7 Set Up Nginx and TLS](#b7-set-up-nginx-and-tls) + - [B.8 Verify the Installation](#b8-verify-the-installation) + - [B.9 Upgrading](#b9-upgrading) +- [Post-Install Configuration](#post-install-configuration) +- [Further Reading](#further-reading) + +--- + +## Prerequisites + +Regardless of which method you choose: + +- **Git** to clone the repository +- A machine with at least **2 GB RAM** and **10 GB free disk** +- Network access to pull container images or download Go/Node toolchains + +--- + +## Option A: Docker Compose + +A single Docker Compose file runs everything: PostgreSQL, MinIO, OpenLDAP, and Silo. An optional nginx container can be enabled for reverse proxying. + +### A.1 Prerequisites + +- [Docker Engine](https://docs.docker.com/engine/install/) 24+ with the [Compose plugin](https://docs.docker.com/compose/install/) (v2) +- `openssl` (used by the setup script to generate secrets) + +Verify your installation: + +```bash +docker --version # Docker Engine 24+ +docker compose version # Docker Compose v2+ +``` + +### A.2 Clone the Repository + +```bash +git clone https://git.kindred-systems.com/kindred/silo.git +cd silo +``` + +### A.3 Run the Setup Script + +The setup script generates credentials and configuration files: + +```bash +./scripts/setup-docker.sh +``` + +It prompts for: +- Server domain (default: `localhost`) +- PostgreSQL password (auto-generated if you press Enter) +- MinIO credentials (auto-generated) +- OpenLDAP admin password and initial user (auto-generated) +- Silo local admin account (fallback when LDAP is unavailable) + +For automated/CI environments, use non-interactive mode: + +```bash +./scripts/setup-docker.sh --non-interactive +``` + +The script writes two files: +- `deployments/.env` — secrets for Docker Compose +- `deployments/config.docker.yaml` — Silo server configuration + +### A.4 Start the Stack + +```bash +docker compose -f deployments/docker-compose.allinone.yaml up -d +``` + +Wait for all services to become healthy: + +```bash +docker compose -f deployments/docker-compose.allinone.yaml ps +``` + +You should see `silo-postgres`, `silo-minio`, `silo-openldap`, and `silo-api` all in a healthy state. + +View logs: + +```bash +# All services +docker compose -f deployments/docker-compose.allinone.yaml logs -f + +# Silo only +docker compose -f deployments/docker-compose.allinone.yaml logs -f silo +``` + +### A.5 Verify the Installation + +```bash +# Health check +curl http://localhost:8080/health + +# Readiness check (includes database and storage connectivity) +curl http://localhost:8080/ready +``` + +Open http://localhost:8080 in your browser. Log in with either: + +- **LDAP account**: the username and password shown by the setup script (default: `siloadmin`) +- **Local admin**: the local admin credentials shown by the setup script (default: `admin`) + +The credentials were printed at the end of the setup script output and are stored in `deployments/.env`. + +### A.6 LDAP Users and Groups + +The Docker stack includes an OpenLDAP server with three preconfigured groups that map to Silo roles: + +| LDAP Group | Silo Role | Access Level | +|------------|-----------|-------------| +| `cn=silo-admins,ou=groups,dc=silo,dc=local` | admin | Full access | +| `cn=silo-users,ou=groups,dc=silo,dc=local` | editor | Create and modify items | +| `cn=silo-viewers,ou=groups,dc=silo,dc=local` | viewer | Read-only | + +The initial LDAP user (default: `siloadmin`) is added to `silo-admins`. + +**Add a new LDAP user:** + +```bash +# From the host (using the exposed port) +ldapadd -x -H ldap://localhost:1389 \ + -D "cn=admin,dc=silo,dc=local" \ + -w "YOUR_LDAP_ADMIN_PASSWORD" << EOF +dn: cn=jdoe,ou=users,dc=silo,dc=local +objectClass: inetOrgPerson +cn: jdoe +sn: Doe +userPassword: changeme +mail: jdoe@example.com +EOF +``` + +**Add a user to a group:** + +```bash +ldapmodify -x -H ldap://localhost:1389 \ + -D "cn=admin,dc=silo,dc=local" \ + -w "YOUR_LDAP_ADMIN_PASSWORD" << EOF +dn: cn=silo-users,ou=groups,dc=silo,dc=local +changetype: modify +add: member +member: cn=jdoe,ou=users,dc=silo,dc=local +EOF +``` + +**List all users:** + +```bash +ldapsearch -x -H ldap://localhost:1389 \ + -b "ou=users,dc=silo,dc=local" \ + -D "cn=admin,dc=silo,dc=local" \ + -w "YOUR_LDAP_ADMIN_PASSWORD" "(objectClass=inetOrgPerson)" cn mail memberOf +``` + +### A.7 Optional: Enable Nginx Reverse Proxy + +To place nginx in front of Silo (for TLS termination or to serve on port 80): + +```bash +docker compose -f deployments/docker-compose.allinone.yaml --profile nginx up -d +``` + +By default nginx listens on ports 80 and 443 and proxies to the Silo container. The configuration is at `deployments/nginx/nginx.conf`. + +**To enable HTTPS**, edit `deployments/docker-compose.allinone.yaml` and uncomment the TLS certificate volume mounts in the `nginx` service, then uncomment the HTTPS server block in `deployments/nginx/nginx.conf`. See the comments in those files for details. + +If you already have your own reverse proxy or load balancer, skip the nginx profile and point your proxy at port 8080. + +### A.8 Stopping, Starting, and Upgrading + +```bash +# Stop the stack (data is preserved in Docker volumes) +docker compose -f deployments/docker-compose.allinone.yaml down + +# Start again +docker compose -f deployments/docker-compose.allinone.yaml up -d + +# Stop and delete all data (WARNING: destroys database, files, and LDAP data) +docker compose -f deployments/docker-compose.allinone.yaml down -v +``` + +**To upgrade to a newer version:** + +```bash +cd silo +git pull +docker compose -f deployments/docker-compose.allinone.yaml up -d --build +``` + +The Silo container is rebuilt from the updated source. Database migrations in `migrations/` are applied automatically on container startup via the PostgreSQL init mechanism. + +--- + +## Option B: Daemon Install (systemd + External Services) + +This method runs Silo as a systemd service on a dedicated host, connecting to externally managed PostgreSQL, MinIO, and optionally LDAP services. + +### B.1 Architecture Overview + +``` + ┌──────────────────────┐ + │ Silo Host │ + │ ┌────────────────┐ │ + HTTPS (443) ──►│ │ nginx │ │ + │ └───────┬────────┘ │ + │ │ :8080 │ + │ ┌───────▼────────┐ │ + │ │ silod │ │ + │ │ (API server) │ │ + │ └──┬─────────┬───┘ │ + └─────┼─────────┼──────┘ + │ │ + ┌───────────▼──┐ ┌───▼──────────────┐ + │ PostgreSQL 16│ │ MinIO (S3) │ + │ :5432 │ │ :9000 API │ + └──────────────┘ │ :9001 Console │ + └──────────────────┘ +``` + +### B.2 Prerequisites + +- Linux host (Debian/Ubuntu or RHEL/Fedora/AlmaLinux) +- Root or sudo access +- Network access to your PostgreSQL and MinIO servers + +The setup script installs Go and other build dependencies automatically. + +### B.3 Set Up External Services + +#### PostgreSQL 16 + +Install PostgreSQL and create the Silo database: + +- [PostgreSQL downloads](https://www.postgresql.org/download/) + +```bash +# After installing PostgreSQL, create the database and user: +sudo -u postgres createuser silo +sudo -u postgres createdb -O silo silo +sudo -u postgres psql -c "ALTER USER silo WITH PASSWORD 'your-password';" +``` + +Ensure the Silo host can connect (check `pg_hba.conf` on the PostgreSQL server). + +Verify: + +```bash +psql -h YOUR_PG_HOST -U silo -d silo -c 'SELECT 1' +``` + +#### MinIO + +Install MinIO and create a bucket and service account: + +- [MinIO quickstart](https://min.io/docs/minio/linux/index.html) + +```bash +# Using the MinIO client (mc): +mc alias set local http://YOUR_MINIO_HOST:9000 minioadmin minioadmin +mc mb local/silo-files +mc admin user add local silouser YOUR_MINIO_SECRET +mc admin policy attach local readwrite --user silouser +``` + +Verify: + +```bash +curl -I http://YOUR_MINIO_HOST:9000/minio/health/live +``` + +#### LDAP / FreeIPA (Optional) + +For LDAP authentication, you need an LDAP server with user and group entries. Options: + +- [FreeIPA](https://www.freeipa.org/page/Quick_Start_Guide) — full identity management (recommended for organizations already using it) +- [OpenLDAP](https://www.openldap.org/doc/admin26/) — lightweight LDAP server + +Silo needs: +- A base DN (e.g., `dc=example,dc=com`) +- Users under a known OU (e.g., `cn=users,cn=accounts,dc=example,dc=com`) +- Groups that map to Silo roles (`admin`, `editor`, `viewer`) +- The `memberOf` overlay enabled (so user entries have `memberOf` attributes) + +See [CONFIGURATION.md — LDAP](CONFIGURATION.md#ldap--freeipa) for the full LDAP configuration reference. + +### B.4 Prepare the Host + +Run the setup script on the target host: + +```bash +# Copy and run the script +scp scripts/setup-host.sh root@YOUR_HOST:/tmp/ +ssh root@YOUR_HOST 'bash /tmp/setup-host.sh' +``` + +Or directly on the host: + +```bash +sudo bash scripts/setup-host.sh +``` + +The script: +1. Installs dependencies (git, Go 1.24) +2. Creates the `silo` system user +3. Creates directories (`/opt/silo`, `/etc/silo`) +4. Clones the repository +5. Creates the environment file template + +To override the default service hostnames: + +```bash +SILO_DB_HOST=db.example.com SILO_MINIO_HOST=s3.example.com sudo -E bash scripts/setup-host.sh +``` + +### B.5 Configure Credentials + +Edit the environment file with your service credentials: + +```bash +sudo nano /etc/silo/silod.env +``` + +```bash +# Database +SILO_DB_PASSWORD=your-database-password + +# MinIO +SILO_MINIO_ACCESS_KEY=silouser +SILO_MINIO_SECRET_KEY=your-minio-secret + +# Authentication +SILO_SESSION_SECRET=generate-a-long-random-string +SILO_ADMIN_USERNAME=admin +SILO_ADMIN_PASSWORD=your-admin-password +``` + +Generate a session secret: + +```bash +openssl rand -hex 32 +``` + +Review the server configuration: + +```bash +sudo nano /etc/silo/config.yaml +``` + +Update `database.host`, `storage.endpoint`, `server.base_url`, and authentication settings for your environment. See [CONFIGURATION.md](CONFIGURATION.md) for all options. + +### B.6 Deploy + +Run the deploy script: + +```bash +sudo /opt/silo/src/scripts/deploy.sh +``` + +The script: +1. Pulls latest code from git +2. Builds the `silod` binary and React frontend +3. Installs files to `/opt/silo` and `/etc/silo` +4. Runs database migrations +5. Installs and starts the systemd service + +Deploy options: + +```bash +# Skip git pull (use current checkout) +sudo /opt/silo/src/scripts/deploy.sh --no-pull + +# Skip build (use existing binary) +sudo /opt/silo/src/scripts/deploy.sh --no-build + +# Just restart the service +sudo /opt/silo/src/scripts/deploy.sh --restart-only + +# Check service status +sudo /opt/silo/src/scripts/deploy.sh --status +``` + +To override the target host or database host: + +```bash +SILO_DEPLOY_TARGET=silo.example.com SILO_DB_HOST=db.example.com sudo -E scripts/deploy.sh +``` + +### B.7 Set Up Nginx and TLS + +#### With FreeIPA (automated) + +If your organization uses FreeIPA, the included script handles nginx setup, IPA enrollment, and certificate issuance: + +```bash +sudo /opt/silo/src/scripts/setup-ipa-nginx.sh +``` + +Override the hostname if needed: + +```bash +SILO_HOSTNAME=silo.example.com sudo -E /opt/silo/src/scripts/setup-ipa-nginx.sh +``` + +The script installs nginx, enrolls the host in FreeIPA, requests a TLS certificate from the IPA CA (auto-renewed by certmonger), and configures nginx as an HTTPS reverse proxy. + +#### Manual nginx setup + +Install nginx and create a config: + +```bash +sudo apt install nginx # or: sudo dnf install nginx +``` + +Use the template at `deployments/nginx/nginx.conf` as a starting point. Copy it to `/etc/nginx/sites-available/silo`, update the `server_name` and certificate paths, then enable it: + +```bash +sudo ln -sf /etc/nginx/sites-available/silo /etc/nginx/sites-enabled/silo +sudo nginx -t +sudo systemctl reload nginx +``` + +After enabling HTTPS, update `server.base_url` in `/etc/silo/config.yaml` to use `https://` and restart Silo: + +```bash +sudo systemctl restart silod +``` + +### B.8 Verify the Installation + +```bash +# Service status +sudo systemctl status silod + +# Health check +curl http://localhost:8080/health + +# Readiness check +curl http://localhost:8080/ready + +# Follow logs +sudo journalctl -u silod -f +``` + +Open your configured base URL in a browser and log in. + +### B.9 Upgrading + +```bash +# Pull latest code and redeploy +sudo /opt/silo/src/scripts/deploy.sh + +# Or deploy a specific version +cd /opt/silo/src +git fetch --all --tags +git checkout v1.2.3 +sudo /opt/silo/src/scripts/deploy.sh --no-pull +``` + +New database migrations are applied automatically during deployment. + +--- + +## Post-Install Configuration + +After a successful installation: + +- **Authentication**: Configure LDAP, OIDC, or local auth backends. See [CONFIGURATION.md — Authentication](CONFIGURATION.md#authentication). +- **Schemas**: Part numbering schemas are loaded from YAML files. See the `schemas/` directory and [CONFIGURATION.md — Schemas](CONFIGURATION.md#schemas). +- **Read-only mode**: Toggle write protection at runtime with `kill -USR1 $(pidof silod)` or by setting `server.read_only: true` in the config. +- **Ongoing maintenance**: See [DEPLOYMENT.md](DEPLOYMENT.md) for service management, log viewing, troubleshooting, and the security checklist. + +--- + +## Further Reading + +| Document | Description | +|----------|-------------| +| [CONFIGURATION.md](CONFIGURATION.md) | Complete `config.yaml` reference | +| [DEPLOYMENT.md](DEPLOYMENT.md) | Operations guide: maintenance, troubleshooting, security | +| [AUTH.md](AUTH.md) | Authentication system design | +| [AUTH_USER_GUIDE.md](AUTH_USER_GUIDE.md) | User guide for login, tokens, and roles | +| [SPECIFICATION.md](SPECIFICATION.md) | Full design specification and API reference | +| [STATUS.md](STATUS.md) | Implementation status | +| [GAP_ANALYSIS.md](GAP_ANALYSIS.md) | Gap analysis and revision control roadmap | +| [COMPONENT_AUDIT.md](COMPONENT_AUDIT.md) | Component audit tool design | diff --git a/docs/src/silo-server/ROADMAP.md b/docs/src/silo-server/ROADMAP.md index 4f281342aa..dc6db61213 100644 --- a/docs/src/silo-server/ROADMAP.md +++ b/docs/src/silo-server/ROADMAP.md @@ -39,7 +39,7 @@ This document compares Silo's current capabilities against SOLIDWORKS PDM—the ### Implemented Features (MVP Complete) #### Core Database System -- PostgreSQL schema with 11 migrations +- PostgreSQL schema with 13 migrations - UUID-based identifiers throughout - Soft delete support via `archived_at` timestamps - Atomic sequence generation for part numbers @@ -92,7 +92,7 @@ This document compares Silo's current capabilities against SOLIDWORKS PDM—the - Template generation for import formatting #### API & Web Interface -- REST API with 75 endpoints +- REST API with 78 endpoints - Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak - Role-based access control (admin > editor > viewer) - API token management (SHA-256 hashed) @@ -129,7 +129,7 @@ This document compares Silo's current capabilities against SOLIDWORKS PDM—the | Component | Status | |-----------|--------| -| PostgreSQL | Running (psql.kindred.internal) | +| PostgreSQL | Running (psql.example.internal) | | MinIO | Configured in Docker Compose | | Silo API Server | Builds successfully | | Docker Compose | Complete (dev and production) | @@ -255,14 +255,14 @@ CAD integration is maintained in separate repositories ([silo-mod](https://git.k | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex | -| API access | Full COM/REST API | Full REST API (75 endpoints) | - | - | +| API access | Full COM/REST API | Full REST API (78 endpoints) | - | - | | Dispatch scripts | Automation without coding | None | Medium | Moderate | | Task scheduler | Background processing | None | Medium | Moderate | | Email system | SMTP integration | None | High | Simple | | Web portal | Browser access | Full (React SPA + auth) | - | - | **Gap Analysis:** -Silo has a comprehensive REST API (75 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation. +Silo has a comprehensive REST API (78 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation. --- diff --git a/docs/src/silo-server/SPECIFICATION.md b/docs/src/silo-server/SPECIFICATION.md index 8f89a938a3..ac8a93d93a 100644 --- a/docs/src/silo-server/SPECIFICATION.md +++ b/docs/src/silo-server/SPECIFICATION.md @@ -37,7 +37,7 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb ▼ ┌─────────────────────────────────────────────────────────────┐ │ Silo Server (silod) │ -│ - REST API (75 endpoints) │ +│ - REST API (78 endpoints) │ │ - Authentication (local, LDAP, OIDC) │ │ - Schema parsing and validation │ │ - Part number generation engine │ @@ -50,7 +50,7 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb ▼ ▼ ┌─────────────────────────┐ ┌─────────────────────────────┐ │ PostgreSQL │ │ MinIO │ -│ (psql.kindred.internal)│ │ - File storage │ +│ (psql.example.internal)│ │ - File storage │ │ - Item metadata │ │ - Versioned objects │ │ - Relationships │ │ - Thumbnails │ │ - Revision history │ │ │ @@ -63,7 +63,7 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb | Component | Technology | Notes | |-----------|------------|-------| -| Database | PostgreSQL 16 | Existing instance at psql.kindred.internal | +| Database | PostgreSQL 16 | Existing instance at psql.example.internal | | File Storage | MinIO | S3-compatible, versioning enabled | | CLI & API Server | Go (1.24) | chi/v5 router, pgx/v5 driver, zerolog | | Authentication | Multi-backend | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak | @@ -598,7 +598,7 @@ See [AUTH.md](AUTH.md) for full architecture details and [AUTH_USER_GUIDE.md](AU ## 11. API Design -### 11.1 REST Endpoints (75 Implemented) +### 11.1 REST Endpoints (78 Implemented) ``` # Health (no auth) @@ -615,6 +615,9 @@ GET /auth/callback # OIDC callback # Public API (no auth required) GET /api/auth/config # Auth backend configuration (for login UI) +# Server-Sent Events (require auth) +GET /api/events # SSE stream for real-time updates + # Auth API (require auth) GET /api/auth/me # Current authenticated user GET /api/auth/tokens # List user's API tokens @@ -627,7 +630,7 @@ POST /api/uploads/presign # Get presigned MinI # Schemas (read: viewer, write: editor) GET /api/schemas # List all schemas GET /api/schemas/{name} # Get schema details -GET /api/schemas/{name}/properties # Get property schema for category +GET /api/schemas/{name}/form # Get form descriptor (field groups, widgets, category picker) POST /api/schemas/{name}/segments/{segment}/values # Add enum value [editor] PUT /api/schemas/{name}/segments/{segment}/values/{code} # Update enum value [editor] DELETE /api/schemas/{name}/segments/{segment}/values/{code} # Delete enum value [editor] @@ -644,6 +647,7 @@ DELETE /api/projects/{code} # Delete project [ed # Items (read: viewer, write: editor) GET /api/items # List/filter items GET /api/items/search # Fuzzy search +GET /api/items/by-uuid/{uuid} # Get item by UUID GET /api/items/export.csv # Export items to CSV GET /api/items/template.csv # CSV import template GET /api/items/export.ods # Export items to ODS @@ -689,6 +693,7 @@ GET /api/items/{partNumber}/bom/export.csv # Export BOM as CSV GET /api/items/{partNumber}/bom/export.ods # Export BOM as ODS POST /api/items/{partNumber}/bom # Add BOM entry [editor] POST /api/items/{partNumber}/bom/import # Import BOM from CSV [editor] +POST /api/items/{partNumber}/bom/merge # Merge BOM from ODS with conflict resolution [editor] PUT /api/items/{partNumber}/bom/{childPartNumber} # Update BOM entry [editor] DELETE /api/items/{partNumber}/bom/{childPartNumber} # Remove BOM entry [editor] @@ -734,11 +739,11 @@ POST /api/inventory/{partNumber}/move ### 12.1 Implemented -- [x] PostgreSQL database schema (11 migrations) +- [x] PostgreSQL database schema (13 migrations) - [x] YAML schema parser for part numbering - [x] Part number generation engine - [x] CLI tool (`cmd/silo`) -- [x] API server (`cmd/silod`) with 75 endpoints +- [x] API server (`cmd/silod`) with 78 endpoints - [x] MinIO integration for file storage with versioning - [x] BOM relationships (component, alternate, reference) - [x] Multi-level BOM (recursive expansion with configurable depth) diff --git a/docs/src/silo-server/STATUS.md b/docs/src/silo-server/STATUS.md index 43235c2663..cac9bbb11e 100644 --- a/docs/src/silo-server/STATUS.md +++ b/docs/src/silo-server/STATUS.md @@ -10,10 +10,10 @@ | Component | Status | Notes | |-----------|--------|-------| -| PostgreSQL schema | Complete | 11 migrations applied | +| PostgreSQL schema | Complete | 13 migrations applied | | YAML schema parser | Complete | Supports enum, serial, constant, string segments | | Part number generator | Complete | Scoped sequences, category-based format | -| API server (`silod`) | Complete | 75 REST endpoints via chi/v5 | +| API server (`silod`) | Complete | 78 REST endpoints via chi/v5 | | CLI tool (`silo`) | Complete | Item registration and management | | MinIO file storage | Complete | Upload, download, versioning, checksums | | Revision control | Complete | Append-only history, rollback, comparison, status/labels | @@ -55,7 +55,7 @@ FreeCAD workbench and LibreOffice Calc extension are maintained in separate repo | Service | Host | Status | |---------|------|--------| -| PostgreSQL | psql.kindred.internal:5432 | Running | +| PostgreSQL | psql.example.internal:5432 | Running | | MinIO | localhost:9000 (API) / :9001 (console) | Configured | | Silo API | localhost:8080 | Builds successfully | @@ -92,5 +92,7 @@ The schema defines 170 category codes across 10 groups: | 007_revision_status.sql | Revision status and labels | | 008_odoo_integration.sql | Odoo ERP integration tables (integrations, sync_log) | | 009_auth.sql | Authentication system (users, api_tokens, sessions, audit_log, user tracking columns) | -| 010_item_extended_fields.sql | Extended item fields (sourcing_type, sourcing_link, standard_cost, long_description) | +| 010_item_extended_fields.sql | Extended item fields (sourcing_type, long_description) | | 011_item_files.sql | Item file attachments (item_files table, thumbnail_key column) | +| 012_bom_source.sql | BOM entry source tracking | +| 013_move_cost_sourcing_to_props.sql | Move sourcing_link and standard_cost from item columns to revision properties | diff --git a/docs/src/silo-server/frontend-spec.md b/docs/src/silo-server/frontend-spec.md index 4e2962925b..6eb6ab3908 100644 --- a/docs/src/silo-server/frontend-spec.md +++ b/docs/src/silo-server/frontend-spec.md @@ -1,6 +1,6 @@ # Silo Frontend Specification -Current as of 2026-02-08. Documents the React + Vite + TypeScript frontend (migration from Go templates is complete). +Current as of 2026-02-11. Documents the React + Vite + TypeScript frontend (migration from Go templates is complete). ## Overview @@ -68,6 +68,7 @@ web/ │ └── AuthContext.tsx AuthProvider with login/logout/refresh methods ├── hooks/ │ ├── useAuth.ts Context consumer hook + │ ├── useFormDescriptor.ts Fetches form descriptor from /api/schemas/{name}/form (replaces useCategories) │ ├── useItems.ts Items fetching with search, filters, pagination, debounce │ └── useLocalStorage.ts Typed localStorage persistence hook ├── styles/ @@ -271,63 +272,81 @@ Vite dev server runs on port 5173 with proxy config in `vite.config.ts` forwardi ## New Frontend Tasks -# CreateItemPane Redesign Specification +# CreateItemPane — Schema-Driven Dynamic Form -**Date**: 2026-02-06 -**Scope**: Replace existing `CreateItemPane.tsx` with a two-column layout, multi-stage category picker, file attachment via MinIO, and full use of screen real estate. +**Date**: 2026-02-10 +**Scope**: `CreateItemPane.tsx` renders a dynamic form driven entirely by the form descriptor API (`GET /api/schemas/{name}/form`). All field groups, field types, widgets, and category-specific fields are defined in YAML and resolved server-side. **Parent**: Items page (`ItemsPage.tsx`) — renders in the detail pane area per existing in-pane CRUD pattern. --- ## Layout -The pane uses a CSS Grid two-column layout instead of the current single-column form: +Single-column scrollable form with a green header bar. Field groups are rendered dynamically from the form descriptor. Category-specific field groups appear after global groups when a category is selected. ``` -┌──────────────────────────────────────────────────────┬──────────────┐ -│ Header: "New Item" [green bar] Cancel │ Create │ │ -├──────────────────────────────────────────────────────┤ │ -│ │ Auto- │ -│ ── Identity ────────────────────────────────────── │ assigned │ -│ [Part Number *] [Type * v] │ metadata │ -│ [Description ] │ │ -│ Category * [Domain │ Group │ Subtype ] │──────────────│ -│ Mechanical│ Structural│ Bracket │ │ │ -│ Electrical│ Bearings │ Plate │ │ Attachments │ -│ ... │ ... │ ... │ │ ┌─ ─ ─ ─ ┐ │ -│ ── Sourcing ────────────────────────────────────── │ │ Drop │ │ -│ [Sourcing Type v] [Standard Cost $ ] │ │ zone │ │ -│ [Unit of Measure v] [Sourcing Link ] │ └─ ─ ─ ─ ┘ │ -│ │ file.FCStd │ -│ ── Details ─────────────────────────────────────── │ drawing.pdf │ -│ [Long Description ] │ │ -│ [Projects: [tag][tag] type to search... ] │──────────────│ -│ │ Thumbnail │ -│ │ [preview] │ -└──────────────────────────────────────────────────────┴──────────────┘ +┌──────────────────────────────────────────────────────────────────────┐ +│ Header: "New Item" [green bar] Cancel │ Create │ +├──────────────────────────────────────────────────────────────────────┤ +│ │ +│ Category * [Domain buttons: F C R S E M T A P X] │ +│ [Subcategory search + filtered list] │ +│ │ +│ ── Identity ────────────────────────────────────────────────────── │ +│ [Type * (auto-derived from category)] [Description ] │ +│ │ +│ ── Sourcing ────────────────────────────────────────────────────── │ +│ [Sourcing Type v] [Manufacturer] [MPN] [Supplier] [SPN] │ +│ [Sourcing Link] │ +│ │ +│ ── Cost & Lead Time ────────────────────────────────────────────── │ +│ [Standard Cost $] [Lead Time Days] [Min Order Qty] │ +│ │ +│ ── Status ──────────────────────────────────────────────────────── │ +│ [Lifecycle Status v] [RoHS Compliant ☐] [Country of Origin] │ +│ │ +│ ── Details ─────────────────────────────────────────────────────── │ +│ [Long Description ] │ +│ [Projects: [tag][tag] type to search... ] │ +│ [Notes ] │ +│ │ +│ ── Fastener Specifications (category-specific) ─────────────────── │ +│ [Material] [Finish] [Thread Size] [Head Type] [Drive Type] ... │ +│ │ +└──────────────────────────────────────────────────────────────────────┘ ``` -Grid definition: `grid-template-columns: 1fr 320px`. The left column scrolls independently if content overflows. The right sidebar is a flex column with sections separated by `--ctp-surface1` borders. +## Data Source — Form Descriptor API + +All form structure is fetched from `GET /api/schemas/kindred-rd/form`, which returns: + +- `category_picker`: Multi-stage picker config (domain → subcategory) +- `item_fields`: Definitions for item-level fields (description, item_type, sourcing_type, etc.) +- `field_groups`: Ordered groups with resolved field metadata (Identity, Sourcing, Cost, Status, Details) +- `category_field_groups`: Per-category-prefix groups (e.g., Fastener Specifications for `F` prefix) +- `field_overrides`: Widget hints (currency, url, select, checkbox) + +The YAML schema (`schemas/kindred-rd.yaml`) is the single source of truth. Adding a new field or category in YAML propagates to all clients with no code changes. ## File Location -`web/src/components/items/CreateItemPane.tsx` (replaces existing file) +`web/src/components/items/CreateItemPane.tsx` -New supporting files: +Supporting files: | File | Purpose | |------|---------| -| `web/src/components/items/CategoryPicker.tsx` | Multi-stage category selector | +| `web/src/components/items/CategoryPicker.tsx` | Multi-stage domain/subcategory selector | | `web/src/components/items/FileDropZone.tsx` | Drag-and-drop file upload with MinIO presigned URLs | | `web/src/components/items/TagInput.tsx` | Multi-select tag input for projects | -| `web/src/hooks/useCategories.ts` | Fetches category tree from schema data | +| `web/src/hooks/useFormDescriptor.ts` | Fetches and caches form descriptor from `/api/schemas/{name}/form` | | `web/src/hooks/useFileUpload.ts` | Manages presigned URL upload flow | ## Component Breakdown ### CreateItemPane -Top-level orchestrator. Manages form state, submission, and layout. +Top-level orchestrator. Renders dynamic form from the form descriptor. **Props** (unchanged interface): @@ -341,68 +360,64 @@ interface CreateItemPaneProps { **State**: ```typescript -const [form, setForm] = useState({ - part_number: '', - item_type: 'part', - description: '', - category_path: [], // e.g. ['Mechanical', 'Structural', 'Bracket'] - sourcing_type: 'manufactured', - standard_cost: '', - unit_of_measure: 'ea', - sourcing_link: '', - long_description: '', - project_ids: [], -}); -const [attachments, setAttachments] = useState([]); -const [thumbnail, setThumbnail] = useState(null); +const { descriptor, categories, loading } = useFormDescriptor(); +const [category, setCategory] = useState(''); // selected category code, e.g. "F01" +const [fields, setFields] = useState>({}); // all field values keyed by name const [error, setError] = useState(null); const [submitting, setSubmitting] = useState(false); ``` +A single `fields` record holds all form values (both item-level and property fields). The `ITEM_LEVEL_FIELDS` set (`description`, `item_type`, `sourcing_type`, `long_description`) determines which fields go into the top-level request vs. the `properties` map on submission. + +**Auto-derivation**: When a category is selected, `item_type` is automatically set based on the `derived_from_category` mapping in the form descriptor (e.g., category prefix `A` → `assembly`, `T` → `tooling`, default → `part`). + +**Dynamic rendering**: A `renderField()` function maps each field's `widget` type to the appropriate input: + +| Widget | Rendered As | +|--------|-------------| +| `text` | `` | +| `number` | `` | +| `textarea` | `