Schedule public snapshot refreshes and optimize historical loading
This commit is contained in:
devRaGonSa
140
ai/tasks/done/TASK-220-schedule-public-snapshot-refreshes.md
Normal file
140
ai/tasks/done/TASK-220-schedule-public-snapshot-refreshes.md
Normal file
@@ -0,0 +1,140 @@
|
||||
---
|
||||
id: TASK-220
|
||||
title: Schedule public snapshot refreshes
|
||||
status: done
|
||||
type: backend
|
||||
team: Backend Senior
|
||||
supporting_teams: ["Arquitecto Python"]
|
||||
roadmap_item: foundation
|
||||
priority: high
|
||||
---
|
||||
|
||||
# TASK-220 - Schedule public snapshot refreshes
|
||||
|
||||
## Goal
|
||||
|
||||
Configure automatic regeneration of public snapshots and read models at appropriate times, so public pages do not depend on heavy runtime calculations.
|
||||
|
||||
## Context
|
||||
|
||||
Public historical, ranking and stats pages should read precomputed data. Heavy regeneration should happen inside the internal runner during low-load windows, while weekly/monthly ranking and recent matches stay fresh at shorter intervals.
|
||||
|
||||
Preserve the current product identity: Spanish-speaking HLL Vietnam community, military/Vietnam/tactical/sober visual direction and controlled repository evolution.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Inspect the listed files first.
|
||||
2. Document how the current refresh paths work.
|
||||
3. Add environment-driven scheduling for daily full refresh, frequent ranking refresh and recent match snapshot refresh.
|
||||
4. Keep refreshes idempotent and non-overlapping.
|
||||
5. Document Portainer configuration and manual emergency commands.
|
||||
6. Validate Python compilation and scheduler helpers.
|
||||
|
||||
## Files to Read First
|
||||
|
||||
- `AGENTS.md`
|
||||
- `ai/repo-context.md`
|
||||
- `ai/architecture-index.md`
|
||||
- `ai/orchestrator/backend-senior.md`
|
||||
- `ai/orchestrator/python-architect.md`
|
||||
- `backend/app/historical_runner.py`
|
||||
- `backend/app/rcon_historical_leaderboards.py`
|
||||
- `backend/app/rcon_annual_rankings.py`
|
||||
- `backend/app/rcon_historical_player_stats.py`
|
||||
- `backend/app/historical_snapshot_storage.py`
|
||||
- `backend/app/historical_snapshots.py`
|
||||
- `backend/app/payloads.py`
|
||||
- `backend/app/config.py`
|
||||
- `backend/app/main.py`
|
||||
- `docker-compose.yml`
|
||||
- `README.md`
|
||||
- `docs/PERFORMANCE_PUBLIC_QUERY_AUDIT.md`
|
||||
|
||||
## Expected Files to Modify
|
||||
|
||||
- `backend/app/config.py`
|
||||
- `backend/app/historical_runner.py`
|
||||
- `backend/app/historical_snapshots.py`
|
||||
- `backend/app/routes.py`
|
||||
- `backend/tests/test_historical_snapshot_refresh.py`
|
||||
- `docker-compose.yml`
|
||||
- `docs/public-snapshot-refresh-schedule.md`
|
||||
- `ai/tasks/in-progress/TASK-220-schedule-public-snapshot-refreshes.md`
|
||||
|
||||
## Constraints
|
||||
|
||||
- Do not execute `ai-platform run`.
|
||||
- Do not push or commit.
|
||||
- Do not touch frontend except documentation if unavoidable.
|
||||
- Do not touch weapon assets, SVGs or physical images.
|
||||
- Do not touch `ai/system-metrics.md`.
|
||||
- Do not reactivate Elo/MMR.
|
||||
- Do not reintroduce Comunidad Hispana #03.
|
||||
- Keep public requests on read models/snapshots for rankings.
|
||||
- Prefer the internal runner over host cron.
|
||||
|
||||
## Validation
|
||||
|
||||
Before completing the task ensure:
|
||||
|
||||
- Run `python -m compileall` for modified backend modules.
|
||||
- Run relevant existing backend tests if applicable.
|
||||
- Validate scheduler helper behavior for the next 06:00 Europe/Madrid.
|
||||
- Confirm public route modules still compile.
|
||||
- Review `git diff --name-only`.
|
||||
- Confirm no frontend, weapon assets, SVGs, physical images or `ai/system-metrics.md` were modified by this task.
|
||||
|
||||
## Outcome
|
||||
|
||||
Completed.
|
||||
|
||||
Current-state analysis:
|
||||
|
||||
- `ranking_snapshots` weekly/monthly were generated by `refresh_ranking_snapshots()` and invoked from `historical_runner` once per historical runner cycle. Compose forced `--hourly`, so public rankings could wait up to an hour despite the code recommending 5-15 minute freshness.
|
||||
- Annual ranking snapshots existed through `generate_annual_ranking_snapshot()` but were only manual/CLI-driven, not part of the runner schedule.
|
||||
- `player_search_index` and `player_period_stats` were refreshed from `historical_runner` once per runner cycle.
|
||||
- Recent matches were included in full/priority historical snapshots, but there was no dedicated short-cadence refresh. RCON capture exposed `materialized_matches_inserted`, which can be used as a finished-match hook.
|
||||
- `historical_runner` already existed and is the right place for internal scheduling; no host cron is required.
|
||||
- Public historical snapshot leaderboard routes in RCON mode were using a runtime materialized fast path despite being snapshot endpoints.
|
||||
|
||||
Implemented strategy:
|
||||
|
||||
- Added public refresh env configuration with safe defaults.
|
||||
- Kept the internal runner as scheduler and removed the Compose `--hourly` override so the runner can tick at the shortest public interval while still throttling the heavier historical cycle separately.
|
||||
- Added daily public full refresh logic, due once per local day after `06:00 Europe/Madrid`.
|
||||
- Added frequent weekly/monthly ranking snapshot refresh at `HLL_PUBLIC_RANKING_REFRESH_INTERVAL_SECONDS`.
|
||||
- Added recent-match-only snapshot generation and scheduled it at `HLL_PUBLIC_RECENT_MATCHES_REFRESH_INTERVAL_SECONDS`.
|
||||
- Added immediate recent-match refresh when RCON capture reports newly materialized finished matches.
|
||||
- Added structured start/end logs, durations and per-step results. Per-step failures are captured without stopping neighboring refreshes or the runner.
|
||||
- Added in-process non-overlap guards for public refresh classes.
|
||||
- Changed public historical leaderboard snapshot routes to read persisted snapshots instead of the RCON runtime fast path.
|
||||
- Documented Portainer configuration and emergency commands in `docs/public-snapshot-refresh-schedule.md`.
|
||||
|
||||
Validation:
|
||||
|
||||
- `python -m compileall backend/app/config.py backend/app/historical_runner.py backend/app/historical_snapshots.py backend/app/routes.py`
|
||||
- `python -m compileall backend/app backend/tests`
|
||||
- `$env:PYTHONPATH='backend'; python -m unittest backend.tests.test_historical_snapshot_refresh`
|
||||
- `$env:PYTHONPATH='backend'; python -m unittest backend.tests.test_annual_ranking_payload`
|
||||
- Dry-run scheduler helper check:
|
||||
- `2026-06-10T03:30:00Z` -> next refresh `2026-06-10T04:00:00Z` (`06:00 Europe/Madrid`)
|
||||
- `2026-06-10T05:00:00Z` -> next refresh `2026-06-11T04:00:00Z`
|
||||
- Route dry-run with patched builder confirmed `/api/historical/snapshots/leaderboard` resolves through `build_leaderboard_snapshot_payload`.
|
||||
- `git diff --name-only` reviewed.
|
||||
|
||||
Exclusions:
|
||||
|
||||
- Did not execute `ai-platform run`.
|
||||
- Did not push or commit.
|
||||
- Did not modify frontend files as part of this task.
|
||||
- Did not modify physical images, weapon assets, SVGs or `ai/system-metrics.md`.
|
||||
- Did not reactivate Elo/MMR.
|
||||
- Did not reintroduce Comunidad Hispana #03.
|
||||
|
||||
Note: the worktree still contains pre-existing unrelated changes in `ai/system-metrics.md`, `frontend/assets/css/styles.css`, clan image files and weapon/SVG assets. They were not touched by this task.
|
||||
|
||||
## Change Budget
|
||||
|
||||
- Prefer fewer than 5 modified files when feasible.
|
||||
- Prefer changes under 200 lines when feasible.
|
||||
- Split follow-up tasks if scope grows.
|
||||
Reference in New Issue
Block a user