comunidadhll/ai/tasks/done/TASK-database-maintenance-cleanup-command.md

id, title, status, type, team, supporting_teams, roadmap_item, priority

id	title	status	type	team	supporting_teams	roadmap_item	priority
TASK-database-maintenance-cleanup-command	Add database maintenance cleanup command	pending	backend	Backend Senior	Arquitecto de Base de Datos Arquitecto Python	historical-data-maintenance	high

TASK-database-maintenance-cleanup-command - Add database maintenance cleanup command

Goal

Implement a safe backend cleanup command with dry-run/apply modes to remove database data that is no longer needed while preserving all data required for visible history and leaderboards.

Context

The project should keep the database bounded without manually deleting data.

The command will later be scheduled from historical-runner, but this task only implements the cleanup module and tests.

The cleanup must not rely on native PostgreSQL scheduling or a new Docker service.

The cleanup must protect:

• latest 100 closed/materialized matches;
• current month;
• previous month during the first 7 days of the new month;
• current week;
• previous week if fallback weekly logic may need it;
• child/player stats for protected matches;
• current live/RCON data.

Steps

1. Inspect the listed files first.
2. Add a backend module for database maintenance.
3. Provide a CLI entry point:
   • python -m app.database_maintenance cleanup --dry-run
   • python -m app.database_maintenance cleanup --apply
4. Make dry-run the default behavior.
5. Add JSON logs for planning, batches and completion.
6. Add tests.
7. Validate locally.
8. Commit and push the task branch.

Files to Read First

• AGENTS.md
• backend/app/config.py
• backend/app/postgres_display_storage.py
• backend/app/historical_runner.py
• backend/app/rcon_historical_read_model.py
• backend/app/rcon_historical_backfill.py
• backend/tests/

Expected Files to Modify

• backend/app/database_maintenance.py
• backend/app/config.py
• backend/tests/test_database_maintenance.py

Optional only if required by existing test helpers:

• backend/tests/conftest.py
• backend/tests/test_*.py

Do not modify frontend files in this task.

Functional Requirements

Create a cleanup command:

python -m app.database_maintenance cleanup --dry-run
python -m app.database_maintenance cleanup --apply

Supported options:

• --recent-matches-keep
• --admin-log-noncritical-retention-days
• --admin-log-critical-retention-days
• --server-snapshot-retention-days
• --batch-size
• --vacuum-analyze
• --now

Environment defaults:

• HLL_RECENT_MATCHES_KEEP=100
• HLL_ADMIN_LOG_NONCRITICAL_RETENTION_DAYS=30
• HLL_ADMIN_LOG_CRITICAL_RETENTION_DAYS=90
• HLL_SERVER_SNAPSHOT_RETENTION_DAYS=14
• HLL_DB_MAINTENANCE_BATCH_SIZE=5000

Dry-run behavior:

• calculate all candidate rows;
• do not delete anything;
• log exactly what would be deleted;
• return status ok.

Apply behavior:

• delete only candidate rows;
• delete in batches;
• use transactions safely;
• do not block indefinitely;
• log deleted row counts;
• return status ok even if deleted counts are zero.

Retention Logic

Protected materialized matches:

• latest N closed matches, default N = 100;
• current month matches;
• previous month matches if now.day <= 7;
• current week matches;
• previous week matches if weekly fallback may need them;
• any match whose end/start timestamp cannot be safely interpreted should be skipped/protected, not deleted.

Delete materialized matches only if:

• not in protected latest N;
• not in protected current/previous month window;
• not in protected current/previous week window;
• older than the relevant protected windows;
• child stats/events that depend on the match are handled first.

AdminLog critical events:

• kill
• match_start
• match_end

Critical events may only be deleted after HLL_ADMIN_LOG_CRITICAL_RETENTION_DAYS and only if they are not needed for protected materialized matches/windows.

AdminLog non-critical events may be deleted after HLL_ADMIN_LOG_NONCRITICAL_RETENTION_DAYS.

Server snapshots may be deleted after HLL_SERVER_SNAPSHOT_RETENTION_DAYS.

displayed_historical_snapshots should not be routinely deleted because it is keyed/upserted. Only skip or report suspicious legacy/corrupt records; do not aggressively delete snapshots in this task.

Safety Requirements

• If an optional table does not exist, log a controlled skip and continue.
• If PostgreSQL URL is not configured and the app is in SQLite mode, either support SQLite safely or return a clear unsupported message, depending on existing architecture.
• Do not leave orphan rows.
• Do not delete from unrelated Elo/MMR tables unless the audit proves exact dependency and task scope is explicitly updated.
• Use existing project lock/writer-lock patterns if present.
• Cleanup failure must produce JSON error output.

Required JSON Log Events

At minimum:

• database-maintenance-started
• database-maintenance-plan
• database-maintenance-table-skipped
• database-maintenance-delete-batch
• database-maintenance-completed
• database-maintenance-error

Tests

Create backend/tests/test_database_maintenance.py.

Minimum cases:

• dry-run does not delete.
• apply deletes old server snapshots.
• apply deletes old non-critical admin log events.
• apply preserves critical events within retention.
• apply preserves latest 100 materialized matches.
• apply preserves current month matches.
• apply preserves previous month when now.day <= 7.
• apply preserves current week.
• apply preserves previous week when fallback protection applies.
• apply deletes an old non-protected materialized match.
• child/player stats for deleted matches are deleted first or handled without orphans.
• missing optional tables are logged and do not crash cleanup.

Validation

Run:

PYTHONPATH=backend python -m unittest backend.tests.test_database_maintenance
python -m compileall backend/app
git diff --check

Also manually run:

docker compose exec backend python -m app.database_maintenance cleanup --dry-run

or, if backend container is unavailable:

cd backend
python -m app.database_maintenance cleanup --dry-run

Document which one was used.

Outcome

Document:

• validation commands run;
• dry-run output summary;
• any existing unrelated test failures;
• exact files changed;
• any follow-up task needed instead of expanding scope.

Codex CLI must commit and push the completed task branch.

Suggested implementation branch:

task/database-maintenance-cleanup-command

Suggested commit message:

feat: add database maintenance cleanup command

Change Budget

Prefer fewer than 5 modified files. Prefer focused backend-only changes. Split follow-up tasks if implementation exceeds safe scope.