silo/docs/INSTALL.md

# Installing Silo

This guide covers two installation methods:

- **[Option A: Docker Compose](#option-a-docker-compose)** — self-contained stack with all services. Recommended for evaluation, small teams, and environments where Docker is the standard.
- **[Option B: Daemon Install](#option-b-daemon-install-systemd--external-services)** — systemd service with external PostgreSQL and optional LDAP/nginx. Files are stored on the local filesystem. Recommended for production deployments integrated with existing infrastructure.

Both methods produce the same result: a running Silo server with a web UI, REST API, and authentication.

---

## Table of Contents

- [Prerequisites](#prerequisites)
- [Option A: Docker Compose](#option-a-docker-compose)
  - [A.1 Prerequisites](#a1-prerequisites)
  - [A.2 Clone the Repository](#a2-clone-the-repository)
  - [A.3 Run the Setup Script](#a3-run-the-setup-script)
  - [A.4 Start the Stack](#a4-start-the-stack)
  - [A.5 Verify the Installation](#a5-verify-the-installation)
  - [A.6 LDAP Users and Groups](#a6-ldap-users-and-groups)
  - [A.7 Optional: Enable Nginx Reverse Proxy](#a7-optional-enable-nginx-reverse-proxy)
  - [A.8 Stopping, Starting, and Upgrading](#a8-stopping-starting-and-upgrading)
- [Option B: Daemon Install (systemd + External Services)](#option-b-daemon-install-systemd--external-services)
  - [B.1 Architecture Overview](#b1-architecture-overview)
  - [B.2 Prerequisites](#b2-prerequisites)
  - [B.3 Set Up External Services](#b3-set-up-external-services)
  - [B.4 Prepare the Host](#b4-prepare-the-host)
  - [B.5 Configure Credentials](#b5-configure-credentials)
  - [B.6 Deploy](#b6-deploy)
  - [B.7 Set Up Nginx and TLS](#b7-set-up-nginx-and-tls)
  - [B.8 Verify the Installation](#b8-verify-the-installation)
  - [B.9 Upgrading](#b9-upgrading)
- [Post-Install Configuration](#post-install-configuration)
- [Further Reading](#further-reading)

---

## Prerequisites

Regardless of which method you choose:

- **Git** to clone the repository
- A machine with at least **2 GB RAM** and **10 GB free disk**
- Network access to pull container images or download Go/Node toolchains

---

## Option A: Docker Compose

A single Docker Compose file runs everything: PostgreSQL, OpenLDAP, and Silo. Files are stored on the local filesystem. An optional nginx container can be enabled for reverse proxying.

### A.1 Prerequisites

- [Docker Engine](https://docs.docker.com/engine/install/) 24+ with the [Compose plugin](https://docs.docker.com/compose/install/) (v2)
- `openssl` (used by the setup script to generate secrets)

Verify your installation:

```bash
docker --version        # Docker Engine 24+
docker compose version  # Docker Compose v2+
```

### A.2 Clone the Repository

```bash
git clone https://git.kindred-systems.com/kindred/silo.git
cd silo
```

### A.3 Run the Setup Script

The setup script generates credentials and configuration files:

```bash
./scripts/setup-docker.sh
```

It prompts for:
- Server domain (default: `localhost`)
- PostgreSQL password (auto-generated if you press Enter)
- OpenLDAP admin password and initial user (auto-generated)
- Silo local admin account (fallback when LDAP is unavailable)

For automated/CI environments, use non-interactive mode:

```bash
./scripts/setup-docker.sh --non-interactive
```

The script writes two files:
- `deployments/.env` — secrets for Docker Compose
- `deployments/config.docker.yaml` — Silo server configuration

### A.4 Start the Stack

```bash
docker compose -f deployments/docker-compose.allinone.yaml up -d
```

Wait for all services to become healthy:

```bash
docker compose -f deployments/docker-compose.allinone.yaml ps
```

You should see `silo-postgres`, `silo-openldap`, and `silo-api` all in a healthy state.

View logs:

```bash
# All services
docker compose -f deployments/docker-compose.allinone.yaml logs -f

# Silo only
docker compose -f deployments/docker-compose.allinone.yaml logs -f silo
```

### A.5 Verify the Installation

```bash
# Health check
curl http://localhost:8080/health

# Readiness check (includes database connectivity)
curl http://localhost:8080/ready
```

Open http://localhost:8080 in your browser. Log in with either:

- **LDAP account**: the username and password shown by the setup script (default: `siloadmin`)
- **Local admin**: the local admin credentials shown by the setup script (default: `admin`)

The credentials were printed at the end of the setup script output and are stored in `deployments/.env`.

### A.6 LDAP Users and Groups

The Docker stack includes an OpenLDAP server with three preconfigured groups that map to Silo roles:

| LDAP Group | Silo Role | Access Level |
|------------|-----------|-------------|
| `cn=silo-admins,ou=groups,dc=silo,dc=local` | admin | Full access |
| `cn=silo-users,ou=groups,dc=silo,dc=local` | editor | Create and modify items |
| `cn=silo-viewers,ou=groups,dc=silo,dc=local` | viewer | Read-only |

The initial LDAP user (default: `siloadmin`) is added to `silo-admins`.

**Add a new LDAP user:**

```bash
# From the host (using the exposed port)
ldapadd -x -H ldap://localhost:1389 \
  -D "cn=admin,dc=silo,dc=local" \
  -w "YOUR_LDAP_ADMIN_PASSWORD" << EOF
dn: cn=jdoe,ou=users,dc=silo,dc=local
objectClass: inetOrgPerson
cn: jdoe
sn: Doe
userPassword: changeme
mail: jdoe@example.com
EOF
```

**Add a user to a group:**

```bash
ldapmodify -x -H ldap://localhost:1389 \
  -D "cn=admin,dc=silo,dc=local" \
  -w "YOUR_LDAP_ADMIN_PASSWORD" << EOF
dn: cn=silo-users,ou=groups,dc=silo,dc=local
changetype: modify
add: member
member: cn=jdoe,ou=users,dc=silo,dc=local
EOF
```

**List all users:**

```bash
ldapsearch -x -H ldap://localhost:1389 \
  -b "ou=users,dc=silo,dc=local" \
  -D "cn=admin,dc=silo,dc=local" \
  -w "YOUR_LDAP_ADMIN_PASSWORD" "(objectClass=inetOrgPerson)" cn mail memberOf
```

### A.7 Optional: Enable Nginx Reverse Proxy

To place nginx in front of Silo (for TLS termination or to serve on port 80):

```bash
docker compose -f deployments/docker-compose.allinone.yaml --profile nginx up -d
```

By default nginx listens on ports 80 and 443 and proxies to the Silo container. The configuration is at `deployments/nginx/nginx.conf`.

**To enable HTTPS**, edit `deployments/docker-compose.allinone.yaml` and uncomment the TLS certificate volume mounts in the `nginx` service, then uncomment the HTTPS server block in `deployments/nginx/nginx.conf`. See the comments in those files for details.

If you already have your own reverse proxy or load balancer, skip the nginx profile and point your proxy at port 8080.

### A.8 Stopping, Starting, and Upgrading

```bash
# Stop the stack (data is preserved in Docker volumes)
docker compose -f deployments/docker-compose.allinone.yaml down

# Start again
docker compose -f deployments/docker-compose.allinone.yaml up -d

# Stop and delete all data (WARNING: destroys database, files, and LDAP data)
docker compose -f deployments/docker-compose.allinone.yaml down -v
```

**To upgrade to a newer version:**

```bash
cd silo
git pull
docker compose -f deployments/docker-compose.allinone.yaml up -d --build
```

The Silo container is rebuilt from the updated source. Database migrations in `migrations/` are applied automatically on container startup via the PostgreSQL init mechanism.

---

## Option B: Daemon Install (systemd + External Services)

This method runs Silo as a systemd service on a dedicated host, connecting to externally managed PostgreSQL and optionally LDAP services. Files are stored on the local filesystem.

### B.1 Architecture Overview

```
                    ┌──────────────────────┐
                    │   Silo Host           │
                    │  ┌────────────────┐  │
     HTTPS (443) ──►│  │     nginx      │  │
                    │  └───────┬────────┘  │
                    │          │ :8080      │
                    │  ┌───────▼────────┐  │
                    │  │     silod       │  │
                    │  │  (API server)   │  │
                    │  │  Files: /opt/   │  │
                    │  │   silo/data     │  │
                    │  └──────┬─────────┘  │
                    └─────────┼────────────┘
                              │
                  ┌───────────▼──┐
                  │ PostgreSQL 16│
                  │   :5432      │
                  └──────────────┘
```

### B.2 Prerequisites

- Linux host (Debian/Ubuntu or RHEL/Fedora/AlmaLinux)
- Root or sudo access
- Network access to your PostgreSQL server

The setup script installs Go and other build dependencies automatically.

### B.3 Set Up External Services

#### PostgreSQL 16

Install PostgreSQL and create the Silo database:

- [PostgreSQL downloads](https://www.postgresql.org/download/)

```bash
# After installing PostgreSQL, create the database and user:
sudo -u postgres createuser silo
sudo -u postgres createdb -O silo silo
sudo -u postgres psql -c "ALTER USER silo WITH PASSWORD 'your-password';"
```

Ensure the Silo host can connect (check `pg_hba.conf` on the PostgreSQL server).

Verify:

```bash
psql -h YOUR_PG_HOST -U silo -d silo -c 'SELECT 1'
```

#### LDAP / FreeIPA (Optional)

For LDAP authentication, you need an LDAP server with user and group entries. Options:

- [FreeIPA](https://www.freeipa.org/page/Quick_Start_Guide) — full identity management (recommended for organizations already using it)
- [OpenLDAP](https://www.openldap.org/doc/admin26/) — lightweight LDAP server

Silo needs:
- A base DN (e.g., `dc=example,dc=com`)
- Users under a known OU (e.g., `cn=users,cn=accounts,dc=example,dc=com`)
- Groups that map to Silo roles (`admin`, `editor`, `viewer`)
- The `memberOf` overlay enabled (so user entries have `memberOf` attributes)

See [CONFIGURATION.md — LDAP](CONFIGURATION.md#ldap--freeipa) for the full LDAP configuration reference.

### B.4 Prepare the Host

Run the setup script on the target host:

```bash
# Copy and run the script
scp scripts/setup-host.sh root@YOUR_HOST:/tmp/
ssh root@YOUR_HOST 'bash /tmp/setup-host.sh'
```

Or directly on the host:

```bash
sudo bash scripts/setup-host.sh
```

The script:
1. Installs dependencies (git, Go 1.24)
2. Creates the `silo` system user
3. Creates directories (`/opt/silo`, `/etc/silo`)
4. Clones the repository
5. Creates the environment file template

To override the default database hostname:

```bash
SILO_DB_HOST=db.example.com sudo -E bash scripts/setup-host.sh
```

### B.5 Configure Credentials

Edit the environment file with your service credentials:

```bash
sudo nano /etc/silo/silod.env
```

```bash
# Database
SILO_DB_PASSWORD=your-database-password

# Authentication
SILO_SESSION_SECRET=generate-a-long-random-string
SILO_ADMIN_USERNAME=admin
SILO_ADMIN_PASSWORD=your-admin-password
```

Generate a session secret:

```bash
openssl rand -hex 32
```

Review the server configuration:

```bash
sudo nano /etc/silo/config.yaml
```

Update `database.host`, `storage.filesystem.root_dir`, `server.base_url`, and authentication settings for your environment. See [CONFIGURATION.md](CONFIGURATION.md) for all options.

### B.6 Deploy

Run the deploy script:

```bash
sudo /opt/silo/src/scripts/deploy.sh
```

The script:
1. Pulls latest code from git
2. Builds the `silod` binary and React frontend
3. Installs files to `/opt/silo` and `/etc/silo`
4. Runs database migrations
5. Installs and starts the systemd service

Deploy options:

```bash
# Skip git pull (use current checkout)
sudo /opt/silo/src/scripts/deploy.sh --no-pull

# Skip build (use existing binary)
sudo /opt/silo/src/scripts/deploy.sh --no-build

# Just restart the service
sudo /opt/silo/src/scripts/deploy.sh --restart-only

# Check service status
sudo /opt/silo/src/scripts/deploy.sh --status
```

To override the target host:

```bash
SILO_DEPLOY_TARGET=silo.example.com sudo -E scripts/deploy.sh
```

### B.7 Set Up Nginx and TLS

#### With FreeIPA (automated)

If your organization uses FreeIPA, the included script handles nginx setup, IPA enrollment, and certificate issuance:

```bash
sudo /opt/silo/src/scripts/setup-ipa-nginx.sh
```

Override the hostname if needed:

```bash
SILO_HOSTNAME=silo.example.com sudo -E /opt/silo/src/scripts/setup-ipa-nginx.sh
```

The script installs nginx, enrolls the host in FreeIPA, requests a TLS certificate from the IPA CA (auto-renewed by certmonger), and configures nginx as an HTTPS reverse proxy.

#### Manual nginx setup

Install nginx and create a config:

```bash
sudo apt install nginx   # or: sudo dnf install nginx
```

Use the template at `deployments/nginx/nginx.conf` as a starting point. Copy it to `/etc/nginx/sites-available/silo`, update the `server_name` and certificate paths, then enable it:

```bash
sudo ln -sf /etc/nginx/sites-available/silo /etc/nginx/sites-enabled/silo
sudo nginx -t
sudo systemctl reload nginx
```

After enabling HTTPS, update `server.base_url` in `/etc/silo/config.yaml` to use `https://` and restart Silo:

```bash
sudo systemctl restart silod
```

### B.8 Verify the Installation

```bash
# Service status
sudo systemctl status silod

# Health check
curl http://localhost:8080/health

# Readiness check
curl http://localhost:8080/ready

# Follow logs
sudo journalctl -u silod -f
```

Open your configured base URL in a browser and log in.

### B.9 Upgrading

```bash
# Pull latest code and redeploy
sudo /opt/silo/src/scripts/deploy.sh

# Or deploy a specific version
cd /opt/silo/src
git fetch --all --tags
git checkout v1.2.3
sudo /opt/silo/src/scripts/deploy.sh --no-pull
```

New database migrations are applied automatically during deployment.

---

## Post-Install Configuration

After a successful installation:

- **Authentication**: Configure LDAP, OIDC, or local auth backends. See [CONFIGURATION.md — Authentication](CONFIGURATION.md#authentication).
- **Schemas**: Part numbering schemas are loaded from YAML files. See the `schemas/` directory and [CONFIGURATION.md — Schemas](CONFIGURATION.md#schemas).
- **Read-only mode**: Toggle write protection at runtime with `kill -USR1 $(pidof silod)` or by setting `server.read_only: true` in the config.
- **Ongoing maintenance**: See [DEPLOYMENT.md](DEPLOYMENT.md) for service management, log viewing, troubleshooting, and the security checklist.

---

## Further Reading

| Document | Description |
|----------|-------------|
| [CONFIGURATION.md](CONFIGURATION.md) | Complete `config.yaml` reference |
| [DEPLOYMENT.md](DEPLOYMENT.md) | Operations guide: maintenance, troubleshooting, security |
| [AUTH.md](AUTH.md) | Authentication system design |
| [AUTH_USER_GUIDE.md](AUTH_USER_GUIDE.md) | User guide for login, tokens, and roles |
| [SPECIFICATION.md](SPECIFICATION.md) | Full design specification and API reference |
| [STATUS.md](STATUS.md) | Implementation status |
| [GAP_ANALYSIS.md](GAP_ANALYSIS.md) | Gap analysis and revision control roadmap |
| [MODULES.md](MODULES.md) | Module system specification |
| [WORKERS.md](WORKERS.md) | Job queue and runner system |
| [SOLVER.md](SOLVER.md) | Assembly solver service |
| [COMPONENT_AUDIT.md](COMPONENT_AUDIT.md) | Component audit tool design |