feat(revisions): configurable retention policy (keep last N) #175

New Issue

forbes · 2026-03-03T20:30:17Z

forbes commented

2026-03-03 20:30:17 +00:00

Context

From GAP_ANALYSIS.md Open Questions #3 and #5:

"Archive or hard delete revisions?"
"How long to keep old revisions?"

Decision: Archive old revisions according to a server-side "keep last N" policy configured in config.yaml. Released revisions are exempt from cleanup.

Current Behavior

All revisions are kept indefinitely. There is no retention policy, cleanup mechanism, or archival process. The revisions table and associated file storage grow without bound.

Proposed Behavior

Configuration

Add a revisions section to config.yaml:

revisions:
  keep_last_n: 50        # keep the N most recent revisions per item (0 = unlimited)
  protect_released: true  # never archive released/obsolete revisions (default true)
  cleanup_interval: 24h   # how often the background sweeper runs

Corresponding additions to internal/config/config.go:

type RevisionsConfig struct {
    KeepLastN        int    `yaml:"keep_last_n"`        // 0 = unlimited
    ProtectReleased  bool   `yaml:"protect_released"`   // default true
    CleanupInterval  string `yaml:"cleanup_interval"`   // default "24h"
}

Archive Behavior

When the sweeper runs (or when a new revision is created and the count exceeds the threshold):

Count revisions for each item
For items exceeding keep_last_n, identify candidates for archival (oldest first)
Skip revisions with status released or obsolete if protect_released is true
For archived revisions:
- Set a new archived_at timestamp column on the revision row (soft delete)
- Delete the associated file from filesystem storage to reclaim disk
- The revision metadata (properties, comment, checksums) remains in the database for audit purposes
Archived revisions are excluded from GET /api/items/{pn}/revisions by default but can be included with ?include_archived=true

API Changes

GET /api/items/{pn}/revisions — add include_archived query param (default false)
GET /api/admin/settings/revisions — expose retention config
PUT /api/admin/settings/revisions — update retention config

Files to Modify

internal/config/config.go — add RevisionsConfig struct and defaults
internal/db/items.go — add ArchiveRevision(), CountRevisions(), update list queries to filter archived
internal/api/handlers.go — update HandleListRevisions() to support include_archived param
New migration — add archived_at TIMESTAMPTZ column to revisions table
New background goroutine — revision cleanup sweeper (similar pattern to job timeout sweeper)

GAP_ANALYSIS.md Section 7, Questions 3 and 5
GAP_ANALYSIS.md Section 3.3: "No retention policy — revisions never expire"
Current revision schema: migrations/001_initial.sql

## Context From GAP_ANALYSIS.md Open Questions #3 and #5: - "Archive or hard delete revisions?" - "How long to keep old revisions?" **Decision:** Archive old revisions according to a server-side "keep last N" policy configured in `config.yaml`. Released revisions are exempt from cleanup. ## Current Behavior All revisions are kept indefinitely. There is no retention policy, cleanup mechanism, or archival process. The `revisions` table and associated file storage grow without bound. ## Proposed Behavior ### Configuration Add a `revisions` section to `config.yaml`: ```yaml revisions: keep_last_n: 50 # keep the N most recent revisions per item (0 = unlimited) protect_released: true # never archive released/obsolete revisions (default true) cleanup_interval: 24h # how often the background sweeper runs ``` Corresponding additions to `internal/config/config.go`: ```go type RevisionsConfig struct { KeepLastN int `yaml:"keep_last_n"` // 0 = unlimited ProtectReleased bool `yaml:"protect_released"` // default true CleanupInterval string `yaml:"cleanup_interval"` // default "24h" } ``` ### Archive Behavior When the sweeper runs (or when a new revision is created and the count exceeds the threshold): 1. Count revisions for each item 2. For items exceeding `keep_last_n`, identify candidates for archival (oldest first) 3. Skip revisions with status `released` or `obsolete` if `protect_released` is true 4. For archived revisions: - Set a new `archived_at` timestamp column on the revision row (soft delete) - Delete the associated file from filesystem storage to reclaim disk - The revision metadata (properties, comment, checksums) remains in the database for audit purposes 5. Archived revisions are excluded from `GET /api/items/{pn}/revisions` by default but can be included with `?include_archived=true` ### API Changes - `GET /api/items/{pn}/revisions` — add `include_archived` query param (default false) - `GET /api/admin/settings/revisions` — expose retention config - `PUT /api/admin/settings/revisions` — update retention config ## Files to Modify - `internal/config/config.go` — add `RevisionsConfig` struct and defaults - `internal/db/items.go` — add `ArchiveRevision()`, `CountRevisions()`, update list queries to filter archived - `internal/api/handlers.go` — update `HandleListRevisions()` to support `include_archived` param - New migration — add `archived_at TIMESTAMPTZ` column to `revisions` table - New background goroutine — revision cleanup sweeper (similar pattern to job timeout sweeper) ## Related - GAP_ANALYSIS.md Section 7, Questions 3 and 5 - GAP_ANALYSIS.md Section 3.3: "No retention policy — revisions never expire" - Current revision schema: `migrations/001_initial.sql`

forbes referenced this issue

2026-03-03 20:30:49 +00:00

feat(storage): explode .kc/.fcstd archives for XML-level revision diffing #176

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: kindred/silo#175