Automate player read model refresh
This commit is contained in:
223
ai/tasks/done/TASK-201-automate-player-read-model-refresh.md
Normal file
223
ai/tasks/done/TASK-201-automate-player-read-model-refresh.md
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
---
|
||||||
|
id: TASK-201-automate-player-read-model-refresh
|
||||||
|
title: Automate player read model refresh
|
||||||
|
status: done
|
||||||
|
type: backend
|
||||||
|
team: Backend Senior
|
||||||
|
supporting_teams:
|
||||||
|
- Arquitecto de Base de Datos
|
||||||
|
- Arquitecto Python
|
||||||
|
roadmap_item: foundation
|
||||||
|
priority: high
|
||||||
|
---
|
||||||
|
|
||||||
|
# TASK-201 - Automate player read model refresh
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Automate the periodic refresh of `player_search_index` and `player_period_stats` inside the existing backend historical runner so the public stats endpoints can keep using their read models without depending on manual operator refreshes.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Production already validates the full read-model chain required by the public player stats flows:
|
||||||
|
|
||||||
|
- `player_search_index` exists and `/api/stats/players/search` already uses `read_model=player-search-index` with `fallback_used=false`
|
||||||
|
- `player_period_stats` exists and `/api/stats/players/{player_id}` already uses `read_model=player-period-stats` with `fallback_used=false`
|
||||||
|
- `ranking_snapshots` / `ranking_snapshot_items` already exist
|
||||||
|
- `refresh-ranking-snapshots --limit 30` already runs in the periodic historical runner
|
||||||
|
- PostgreSQL is the operational storage target
|
||||||
|
- runtime fallback must remain preserved for player stats and ranking
|
||||||
|
|
||||||
|
The remaining gap is operational automation for the two player read models. The existing periodic runner in `backend/app/historical_runner.py` is the intended scheduler surface, so this task must extend that cycle instead of inventing a new scheduler. The cycle should keep `refresh-ranking-snapshots` and add dedicated, separately reported refresh steps for player search and player period stats.
|
||||||
|
|
||||||
|
Preserve the current product identity: Spanish-speaking HLL Vietnam community, military/Vietnam/tactical/sober visual direction and controlled repository evolution.
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
1. Read the listed files first.
|
||||||
|
2. Inspect the current historical runner flow and the existing manual read-model refresh entrypoints.
|
||||||
|
3. Integrate `refresh_player_search_index(...)` into the periodic backend cycle.
|
||||||
|
4. Integrate `refresh_player_period_stats(...)` into the periodic backend cycle.
|
||||||
|
5. Keep `refresh-ranking-snapshots` in the same cycle and preserve its current behavior.
|
||||||
|
6. Run the cycle in this order unless the current runner architecture safely requires a different invariant:
|
||||||
|
- existing RCON ingestion/materialization cycle
|
||||||
|
- `player_search_index`
|
||||||
|
- `player_period_stats`
|
||||||
|
- `ranking_snapshots`
|
||||||
|
7. Report `player_search_index_result`, `player_period_stats_result` and `ranking_snapshot_result` separately.
|
||||||
|
8. Preserve visibility of failures and avoid hiding one read-model failure behind another. One failure should not necessarily prevent the remaining refresh attempts unless the runner already enforces a stricter global policy.
|
||||||
|
9. Preserve the manual CLIs and the public contracts of:
|
||||||
|
- `/api/stats/players/search`
|
||||||
|
- `/api/stats/players/{player_id}`
|
||||||
|
- `/api/ranking`
|
||||||
|
10. Document the automatic runner refresh, inherited cadence, emergency manual commands, fallback preservation and final cycle order.
|
||||||
|
|
||||||
|
## Files to Read First
|
||||||
|
|
||||||
|
- `AGENTS.md`
|
||||||
|
- `ai/repo-context.md`
|
||||||
|
- `ai/architecture-index.md`
|
||||||
|
- `backend/app/historical_runner.py`
|
||||||
|
- `backend/app/rcon_historical_player_stats.py`
|
||||||
|
- `backend/app/rcon_historical_leaderboards.py`
|
||||||
|
- `backend/app/postgres_rcon_storage.py`
|
||||||
|
- `backend/app/payloads.py`
|
||||||
|
- `docs/player-search-read-model-plan.md`
|
||||||
|
- `docs/player-period-stats-read-model-plan.md`
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- `ai/tasks/done/TASK-197-automate-ranking-snapshot-refresh.md`
|
||||||
|
- `ai/tasks/done/TASK-198-add-player-search-read-model.md`
|
||||||
|
- `ai/tasks/done/TASK-199-add-player-period-stats-read-model.md`
|
||||||
|
- `ai/tasks/done/TASK-200-fix-player-profile-runtime-postgres-grouping.md`
|
||||||
|
|
||||||
|
## Expected Files to Modify
|
||||||
|
|
||||||
|
- `backend/app/historical_runner.py`
|
||||||
|
- `docs/player-search-read-model-plan.md`
|
||||||
|
- `docs/player-period-stats-read-model-plan.md`
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- `ai/tasks/in-progress/TASK-201-automate-player-read-model-refresh.md`
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
- Keep the change minimal and backend/documentation only.
|
||||||
|
- Do not execute `ai-platform run`.
|
||||||
|
- Do not modify frontend.
|
||||||
|
- Do not change design.
|
||||||
|
- Do not touch images or assets.
|
||||||
|
- Do not reactivate Elo/MMR.
|
||||||
|
- Do not reintroduce Comunidad Hispana `#03`.
|
||||||
|
- Do not mix visual corrections into this task.
|
||||||
|
- Preserve PostgreSQL as the operational storage target.
|
||||||
|
- Preserve runtime fallback for player stats and ranking.
|
||||||
|
- Do not break the manual CLIs:
|
||||||
|
- `refresh-player-search-index`
|
||||||
|
- `refresh-player-period-stats`
|
||||||
|
- `refresh-ranking-snapshots`
|
||||||
|
- `generate-ranking-snapshot`
|
||||||
|
- Do not change public contracts of:
|
||||||
|
- `/api/stats/players/search`
|
||||||
|
- `/api/stats/players/{player_id}`
|
||||||
|
- `/api/ranking`
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
Before completing the task ensure:
|
||||||
|
|
||||||
|
- `python -m py_compile backend/app/historical_runner.py backend/app/rcon_historical_player_stats.py`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||||
|
- local/import validation proves the runner calls:
|
||||||
|
- `refresh_player_search_index(...)`
|
||||||
|
- `refresh_player_period_stats(...)`
|
||||||
|
- `refresh_ranking_snapshots(...)`
|
||||||
|
- validation proves the periodic cycle reports these results separately:
|
||||||
|
- `player_search_index_result`
|
||||||
|
- `player_period_stats_result`
|
||||||
|
- `ranking_snapshot_result`
|
||||||
|
- validation proves the manual CLIs still work:
|
||||||
|
- `refresh-player-search-index`
|
||||||
|
- `refresh-player-period-stats`
|
||||||
|
- `refresh-ranking-snapshots`
|
||||||
|
- validation proves `/api/stats/players/search` can still use `player-search-index`
|
||||||
|
- validation proves `/api/stats/players/{player_id}` can still use `player-period-stats`
|
||||||
|
- validation proves ranking snapshots are not broken
|
||||||
|
- `git diff --name-only` matches the expected scope
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Implemented:
|
||||||
|
|
||||||
|
- `backend/app/historical_runner.py`
|
||||||
|
- integrated automatic periodic refresh of `player_search_index`
|
||||||
|
- integrated automatic periodic refresh of `player_period_stats`
|
||||||
|
- preserved the existing `ranking_snapshots` refresh
|
||||||
|
- the runner now reports:
|
||||||
|
- `player_search_index_result`
|
||||||
|
- `player_period_stats_result`
|
||||||
|
- `ranking_snapshot_result`
|
||||||
|
- one read-model failure is now reported explicitly and does not prevent the runner from attempting the remaining refresh steps in the same cycle
|
||||||
|
- the runner returns `status=partial` when one of those periodic read-model refresh steps fails but the cycle continues
|
||||||
|
- `docs/player-search-read-model-plan.md`
|
||||||
|
- documents automatic runner refresh
|
||||||
|
- documents inherited cadence from `HLL_HISTORICAL_REFRESH_INTERVAL_SECONDS`
|
||||||
|
- documents the emergency manual command
|
||||||
|
- documents fallback preservation
|
||||||
|
- `docs/player-period-stats-read-model-plan.md`
|
||||||
|
- documents automatic runner refresh
|
||||||
|
- documents inherited cadence from `HLL_HISTORICAL_REFRESH_INTERVAL_SECONDS`
|
||||||
|
- documents the emergency manual command
|
||||||
|
- documents fallback preservation
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- documents the final runner order
|
||||||
|
- documents that ranking refresh remains part of the same periodic cycle
|
||||||
|
- documents cadence inheritance and fallback preservation
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- validates that the runner calls the player read-model refreshes before ranking snapshots
|
||||||
|
- validates separate result reporting
|
||||||
|
- validates that a `player_search_index` refresh failure remains visible and does not stop the remaining periodic refresh attempts
|
||||||
|
|
||||||
|
Modified files:
|
||||||
|
|
||||||
|
- `backend/app/historical_runner.py`
|
||||||
|
- `docs/player-search-read-model-plan.md`
|
||||||
|
- `docs/player-period-stats-read-model-plan.md`
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- `ai/tasks/done/TASK-201-automate-player-read-model-refresh.md`
|
||||||
|
|
||||||
|
Final runner order:
|
||||||
|
|
||||||
|
1. existing RCON ingestion/materialization cycle
|
||||||
|
2. `player_search_index`
|
||||||
|
3. `player_period_stats`
|
||||||
|
4. `ranking_snapshots`
|
||||||
|
|
||||||
|
Added cycle results/logs:
|
||||||
|
|
||||||
|
- `player_search_index_result`
|
||||||
|
- `player_period_stats_result`
|
||||||
|
- `ranking_snapshot_result`
|
||||||
|
- start events:
|
||||||
|
- `player-search-index-refresh-started`
|
||||||
|
- `player-period-stats-refresh-started`
|
||||||
|
- failure events:
|
||||||
|
- `player-search-index-refresh-failed`
|
||||||
|
- `player-period-stats-refresh-failed`
|
||||||
|
|
||||||
|
Validations executed:
|
||||||
|
|
||||||
|
- `python -m py_compile backend/app/historical_runner.py backend/app/rcon_historical_player_stats.py`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||||
|
|
||||||
|
Production validation path:
|
||||||
|
|
||||||
|
- run one controlled backend cycle:
|
||||||
|
- `python -m app.historical_runner --max-runs 1`
|
||||||
|
- verify the cycle output includes:
|
||||||
|
- `player_search_index_result`
|
||||||
|
- `player_period_stats_result`
|
||||||
|
- `ranking_snapshot_result`
|
||||||
|
- confirm the player stats endpoints still report:
|
||||||
|
- `/api/stats/players/search` -> `source.read_model=player-search-index`
|
||||||
|
- `/api/stats/players/{player_id}` -> `source.read_model=player-period-stats`
|
||||||
|
- confirm `/api/ranking` still serves ranking snapshots when ready and preserves fallback behavior when a snapshot is missing
|
||||||
|
- if emergency rebuild is needed, operators can still run:
|
||||||
|
- `python -m app.rcon_historical_player_stats refresh-player-search-index`
|
||||||
|
- `python -m app.rcon_historical_player_stats refresh-player-period-stats`
|
||||||
|
- `python -m app.rcon_historical_leaderboards refresh-ranking-snapshots --limit 30`
|
||||||
|
|
||||||
|
Pending limitations kept out of scope:
|
||||||
|
|
||||||
|
- the runner still refreshes the player read models for all supported public scopes even when a manual runner execution is limited with `--server`
|
||||||
|
- this task does not introduce a separate external scheduler or deployment-specific cron wiring
|
||||||
|
- live HTTP validation at `http://127.0.0.1:8000` was not available in this environment; route-contract validation passed via local Python imports instead
|
||||||
|
|
||||||
|
## Change Budget
|
||||||
|
|
||||||
|
- Prefer fewer than 5 modified files.
|
||||||
|
- Prefer changes under 200 lines when feasible.
|
||||||
|
- Split the work into follow-up tasks if limits are exceeded.
|
||||||
@@ -30,6 +30,10 @@ from .historical_snapshots import (
|
|||||||
generate_and_persist_priority_historical_snapshots,
|
generate_and_persist_priority_historical_snapshots,
|
||||||
)
|
)
|
||||||
from .rcon_historical_leaderboards import refresh_ranking_snapshots
|
from .rcon_historical_leaderboards import refresh_ranking_snapshots
|
||||||
|
from .rcon_historical_player_stats import (
|
||||||
|
refresh_player_period_stats,
|
||||||
|
refresh_player_search_index,
|
||||||
|
)
|
||||||
from .rcon_historical_storage import count_rcon_historical_samples_since
|
from .rcon_historical_storage import count_rcon_historical_samples_since
|
||||||
from .rcon_historical_worker import run_rcon_historical_capture
|
from .rcon_historical_worker import run_rcon_historical_capture
|
||||||
from .writer_lock import backend_writer_lock, build_writer_lock_holder
|
from .writer_lock import backend_writer_lock, build_writer_lock_holder
|
||||||
@@ -177,13 +181,29 @@ def _run_refresh_with_retries(
|
|||||||
rcon_capture_result=rcon_capture_result
|
rcon_capture_result=rcon_capture_result
|
||||||
),
|
),
|
||||||
}
|
}
|
||||||
|
player_search_index_result = refresh_periodic_player_search_index(
|
||||||
|
server_slug=server_slug,
|
||||||
|
run_number=run_number,
|
||||||
|
)
|
||||||
|
player_period_stats_result = refresh_periodic_player_period_stats(
|
||||||
|
server_slug=server_slug,
|
||||||
|
run_number=run_number,
|
||||||
|
)
|
||||||
ranking_snapshot_result = refresh_periodic_ranking_snapshots(
|
ranking_snapshot_result = refresh_periodic_ranking_snapshots(
|
||||||
server_slug=server_slug,
|
server_slug=server_slug,
|
||||||
run_number=run_number,
|
run_number=run_number,
|
||||||
)
|
)
|
||||||
maintenance_result = _maybe_run_database_maintenance()
|
maintenance_result = _maybe_run_database_maintenance()
|
||||||
return {
|
return {
|
||||||
"status": "ok",
|
"status": _resolve_refresh_cycle_status(
|
||||||
|
refresh_result=refresh_result,
|
||||||
|
snapshot_result=snapshot_result,
|
||||||
|
player_search_index_result=player_search_index_result,
|
||||||
|
player_period_stats_result=player_period_stats_result,
|
||||||
|
ranking_snapshot_result=ranking_snapshot_result,
|
||||||
|
elo_mmr_result=elo_mmr_result,
|
||||||
|
database_maintenance_result=maintenance_result,
|
||||||
|
),
|
||||||
"attempts_used": attempt,
|
"attempts_used": attempt,
|
||||||
"max_retries": max_retries,
|
"max_retries": max_retries,
|
||||||
"rcon_capture_result": rcon_capture_result,
|
"rcon_capture_result": rcon_capture_result,
|
||||||
@@ -191,6 +211,8 @@ def _run_refresh_with_retries(
|
|||||||
"classic_fallback_reason": classic_fallback_reason,
|
"classic_fallback_reason": classic_fallback_reason,
|
||||||
"refresh_result": refresh_result,
|
"refresh_result": refresh_result,
|
||||||
"snapshot_result": snapshot_result,
|
"snapshot_result": snapshot_result,
|
||||||
|
"player_search_index_result": player_search_index_result,
|
||||||
|
"player_period_stats_result": player_period_stats_result,
|
||||||
"ranking_snapshot_result": ranking_snapshot_result,
|
"ranking_snapshot_result": ranking_snapshot_result,
|
||||||
"elo_mmr_result": elo_mmr_result,
|
"elo_mmr_result": elo_mmr_result,
|
||||||
"database_maintenance_result": maintenance_result,
|
"database_maintenance_result": maintenance_result,
|
||||||
@@ -286,6 +308,102 @@ def refresh_periodic_ranking_snapshots(
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def refresh_periodic_player_search_index(
|
||||||
|
*,
|
||||||
|
server_slug: str | None = None,
|
||||||
|
run_number: int = 1,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Refresh the player search read model without aborting the remaining cycle on failure."""
|
||||||
|
_emit_json_log(
|
||||||
|
{
|
||||||
|
"event": "player-search-index-refresh-started",
|
||||||
|
"run_number": run_number,
|
||||||
|
"server_slug": server_slug,
|
||||||
|
"refresh_scope": "supported-public-player-search-scopes",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = refresh_player_search_index()
|
||||||
|
except Exception as exc: # noqa: BLE001 - one read-model failure must stay visible
|
||||||
|
failure_result = {
|
||||||
|
"status": "error",
|
||||||
|
"error_type": type(exc).__name__,
|
||||||
|
"error": str(exc),
|
||||||
|
"run_number": run_number,
|
||||||
|
"refresh_interval_seconds": get_historical_refresh_interval_seconds(),
|
||||||
|
"server_slug": server_slug,
|
||||||
|
"generation_policy": "periodic-historical-refresh-cycle",
|
||||||
|
"scope_policy": "always-refresh-supported-public-player-search-scopes",
|
||||||
|
}
|
||||||
|
_emit_json_log({"event": "player-search-index-refresh-failed", **failure_result})
|
||||||
|
return failure_result
|
||||||
|
return {
|
||||||
|
**result,
|
||||||
|
"run_number": run_number,
|
||||||
|
"refresh_interval_seconds": get_historical_refresh_interval_seconds(),
|
||||||
|
"server_slug": server_slug,
|
||||||
|
"generation_policy": "periodic-historical-refresh-cycle",
|
||||||
|
"scope_policy": "always-refresh-supported-public-player-search-scopes",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def refresh_periodic_player_period_stats(
|
||||||
|
*,
|
||||||
|
server_slug: str | None = None,
|
||||||
|
run_number: int = 1,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Refresh the player period stats read model without aborting the remaining cycle on failure."""
|
||||||
|
_emit_json_log(
|
||||||
|
{
|
||||||
|
"event": "player-period-stats-refresh-started",
|
||||||
|
"run_number": run_number,
|
||||||
|
"server_slug": server_slug,
|
||||||
|
"refresh_scope": "supported-public-player-period-scopes",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = refresh_player_period_stats()
|
||||||
|
except Exception as exc: # noqa: BLE001 - one read-model failure must stay visible
|
||||||
|
failure_result = {
|
||||||
|
"status": "error",
|
||||||
|
"error_type": type(exc).__name__,
|
||||||
|
"error": str(exc),
|
||||||
|
"run_number": run_number,
|
||||||
|
"refresh_interval_seconds": get_historical_refresh_interval_seconds(),
|
||||||
|
"server_slug": server_slug,
|
||||||
|
"generation_policy": "periodic-historical-refresh-cycle",
|
||||||
|
"scope_policy": "always-refresh-supported-public-player-period-scopes",
|
||||||
|
}
|
||||||
|
_emit_json_log({"event": "player-period-stats-refresh-failed", **failure_result})
|
||||||
|
return failure_result
|
||||||
|
return {
|
||||||
|
**result,
|
||||||
|
"run_number": run_number,
|
||||||
|
"refresh_interval_seconds": get_historical_refresh_interval_seconds(),
|
||||||
|
"server_slug": server_slug,
|
||||||
|
"generation_policy": "periodic-historical-refresh-cycle",
|
||||||
|
"scope_policy": "always-refresh-supported-public-player-period-scopes",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_refresh_cycle_status(**results: dict[str, Any]) -> str:
|
||||||
|
statuses = [
|
||||||
|
str(result.get("status") or "").strip().lower()
|
||||||
|
for result in results.values()
|
||||||
|
if isinstance(result, dict) and result.get("status") is not None
|
||||||
|
]
|
||||||
|
if not statuses:
|
||||||
|
return "ok"
|
||||||
|
if any(status == "error" for status in statuses):
|
||||||
|
ok_like_statuses = {"ok", "skipped"}
|
||||||
|
if all(status not in ok_like_statuses for status in statuses):
|
||||||
|
return "error"
|
||||||
|
return "partial"
|
||||||
|
if any(status == "partial" for status in statuses):
|
||||||
|
return "partial"
|
||||||
|
return "ok"
|
||||||
|
|
||||||
|
|
||||||
def _emit_json_log(payload: dict[str, Any]) -> None:
|
def _emit_json_log(payload: dict[str, Any]) -> None:
|
||||||
"""Print JSON logs that remain safe for Compose and log collectors."""
|
"""Print JSON logs that remain safe for Compose and log collectors."""
|
||||||
print(json.dumps(payload, ensure_ascii=True, default=str), flush=True)
|
print(json.dumps(payload, ensure_ascii=True, default=str), flush=True)
|
||||||
|
|||||||
@@ -102,6 +102,19 @@ Refresh policy:
|
|||||||
- keep the latest player name seen inside the selected period window
|
- keep the latest player name seen inside the selected period window
|
||||||
- persist ranking position by kills inside each generated period window
|
- persist ranking position by kills inside each generated period window
|
||||||
|
|
||||||
|
Automatic runner refresh:
|
||||||
|
|
||||||
|
- `backend/app/historical_runner.py` refreshes `player_period_stats` automatically
|
||||||
|
- it inherits the periodic cadence of the historical runner via `HLL_HISTORICAL_REFRESH_INTERVAL_SECONDS`
|
||||||
|
- the runner executes this step after `player_search_index` and before `ranking_snapshots`
|
||||||
|
- the public profile path keeps runtime fallback preserved if the read model is empty, incomplete or unavailable
|
||||||
|
|
||||||
|
Emergency manual command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.rcon_historical_player_stats refresh-player-period-stats
|
||||||
|
```
|
||||||
|
|
||||||
## Public Read Path
|
## Public Read Path
|
||||||
|
|
||||||
Priority for `/api/stats/players/{player_id}`:
|
Priority for `/api/stats/players/{player_id}`:
|
||||||
@@ -147,7 +160,8 @@ Indexes kept for the public profile flow:
|
|||||||
|
|
||||||
Recommended checks after refresh:
|
Recommended checks after refresh:
|
||||||
|
|
||||||
- run `python -m app.rcon_historical_player_stats refresh-player-period-stats`
|
- confirm the historical runner output reports `player_period_stats_result`
|
||||||
|
- if an emergency rebuild is needed, run `python -m app.rcon_historical_player_stats refresh-player-period-stats`
|
||||||
- confirm the command reports rows for:
|
- confirm the command reports rows for:
|
||||||
- `all-servers`
|
- `all-servers`
|
||||||
- `comunidad-hispana-01`
|
- `comunidad-hispana-01`
|
||||||
@@ -162,7 +176,7 @@ Recommended checks after refresh:
|
|||||||
|
|
||||||
## Current Limitations
|
## Current Limitations
|
||||||
|
|
||||||
- periodic refresh is still manual; there is no scheduled operational refresh yet
|
|
||||||
- the public route still exposes weekly/monthly only; yearly is prepared internally for future use
|
- the public route still exposes weekly/monthly only; yearly is prepared internally for future use
|
||||||
- runtime fallback remains necessary until production refresh automation is in place
|
- the runner refreshes all supported public scopes and periods on each cycle even when a manual runner execution is limited with `--server`
|
||||||
|
- runtime fallback remains necessary as a safety net even with periodic automation in place
|
||||||
- canonical historical truth remains in materialized RCON tables, not in the read model
|
- canonical historical truth remains in materialized RCON tables, not in the read model
|
||||||
|
|||||||
@@ -71,6 +71,19 @@ Refresh policy:
|
|||||||
- keep the latest current-year player name
|
- keep the latest current-year player name
|
||||||
- store accent-insensitive normalized names in Python
|
- store accent-insensitive normalized names in Python
|
||||||
|
|
||||||
|
Automatic runner refresh:
|
||||||
|
|
||||||
|
- `backend/app/historical_runner.py` refreshes `player_search_index` automatically
|
||||||
|
- it inherits the periodic cadence of the historical runner via `HLL_HISTORICAL_REFRESH_INTERVAL_SECONDS`
|
||||||
|
- the runner executes this step after the existing RCON ingestion/materialization cycle
|
||||||
|
- the runner keeps runtime fallback preserved for the public endpoint
|
||||||
|
|
||||||
|
Emergency manual command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.rcon_historical_player_stats refresh-player-search-index
|
||||||
|
```
|
||||||
|
|
||||||
## Public Read Path
|
## Public Read Path
|
||||||
|
|
||||||
Priority for `/api/stats/players/search`:
|
Priority for `/api/stats/players/search`:
|
||||||
@@ -113,13 +126,15 @@ Search tolerance is implemented with:
|
|||||||
- this read model is focused on player search only, not personal profile totals
|
- this read model is focused on player search only, not personal profile totals
|
||||||
- counts are current-year only by design
|
- counts are current-year only by design
|
||||||
- historical players with no activity in the current UTC year are not intentionally prioritized in this first model
|
- historical players with no activity in the current UTC year are not intentionally prioritized in this first model
|
||||||
- profile and personal stats still use runtime aggregation over materialized tables
|
- the runner refreshes all supported public scopes on each cycle even when a manual runner execution is limited with `--server`
|
||||||
|
- profile and personal stats use their own dedicated read model and still preserve runtime fallback when needed
|
||||||
|
|
||||||
## Production Validation
|
## Production Validation
|
||||||
|
|
||||||
Recommended checks after refresh:
|
Recommended checks after refresh:
|
||||||
|
|
||||||
- run `python -m app.rcon_historical_player_stats refresh-player-search-index`
|
- confirm the historical runner output reports `player_search_index_result`
|
||||||
|
- if an emergency rebuild is needed, run `python -m app.rcon_historical_player_stats refresh-player-search-index`
|
||||||
- confirm the command reports rows for `all-servers`, `comunidad-hispana-01` and `comunidad-hispana-02`
|
- confirm the command reports rows for `all-servers`, `comunidad-hispana-01` and `comunidad-hispana-02`
|
||||||
- call `/api/stats/players/search?q=<known-player>&limit=5`
|
- call `/api/stats/players/search?q=<known-player>&limit=5`
|
||||||
- verify response metadata reports `read_model=player-search-index`
|
- verify response metadata reports `read_model=player-search-index`
|
||||||
|
|||||||
@@ -292,7 +292,12 @@ Supported manual parameters:
|
|||||||
Current implementation note:
|
Current implementation note:
|
||||||
- `generate-ranking-snapshot` remains unitary per command invocation for explicit manual control
|
- `generate-ranking-snapshot` remains unitary per command invocation for explicit manual control
|
||||||
- `refresh-ranking-snapshots --limit 30` generates the full weekly/monthly public matrix in one run
|
- `refresh-ranking-snapshots --limit 30` generates the full weekly/monthly public matrix in one run
|
||||||
- the periodic historical runner in `backend/app/historical_runner.py` should invoke that bulk refresh as part of the normal backend refresh cycle
|
- the periodic historical runner in `backend/app/historical_runner.py` invokes that bulk refresh as part of the normal backend refresh cycle
|
||||||
|
- current cycle order is:
|
||||||
|
- existing RCON ingestion/materialization cycle
|
||||||
|
- `player_search_index`
|
||||||
|
- `player_period_stats`
|
||||||
|
- `ranking_snapshots`
|
||||||
- per-combination failures should be reported without aborting the entire matrix refresh
|
- per-combination failures should be reported without aborting the entire matrix refresh
|
||||||
|
|
||||||
## Recommended Combinations
|
## Recommended Combinations
|
||||||
@@ -323,6 +328,8 @@ Operational guidance:
|
|||||||
- regenerate after manual backfill
|
- regenerate after manual backfill
|
||||||
- regenerate after metric SQL changes that affect ranking totals or ordering
|
- regenerate after metric SQL changes that affect ranking totals or ordering
|
||||||
- when using the periodic backend runner, keep ranking refresh attached to the same recurring cycle rather than a separate scheduler unless operational load proves otherwise
|
- when using the periodic backend runner, keep ranking refresh attached to the same recurring cycle rather than a separate scheduler unless operational load proves otherwise
|
||||||
|
- the runner inherits its cadence from `HLL_HISTORICAL_REFRESH_INTERVAL_SECONDS`
|
||||||
|
- fallback runtime remains preserved for `/api/ranking` if a requested snapshot is missing or unavailable
|
||||||
|
|
||||||
## Ready Vs Fallback
|
## Ready Vs Fallback
|
||||||
|
|
||||||
|
|||||||
@@ -127,6 +127,7 @@ sys.path.insert(0, "backend")
|
|||||||
from app.routes import resolve_get_payload
|
from app.routes import resolve_get_payload
|
||||||
from app.config import use_postgres_rcon_storage
|
from app.config import use_postgres_rcon_storage
|
||||||
import app.postgres_rcon_storage as postgres_rcon_storage
|
import app.postgres_rcon_storage as postgres_rcon_storage
|
||||||
|
import app.historical_runner as historical_runner
|
||||||
import app.rcon_historical_leaderboards as ranking_leaderboards
|
import app.rcon_historical_leaderboards as ranking_leaderboards
|
||||||
import app.rcon_historical_player_stats as player_search_stats
|
import app.rcon_historical_player_stats as player_search_stats
|
||||||
|
|
||||||
@@ -1217,6 +1218,186 @@ def validate_player_period_stats_fallbacks():
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def validate_historical_runner_read_model_refresh_cycle():
|
||||||
|
original_backend_writer_lock = historical_runner.backend_writer_lock
|
||||||
|
original_build_writer_lock_holder = historical_runner.build_writer_lock_holder
|
||||||
|
original_run_primary_rcon_capture = historical_runner._run_primary_rcon_capture
|
||||||
|
original_resolve_classic_fallback_policy = historical_runner._resolve_classic_fallback_policy
|
||||||
|
original_rcon_capture_has_new_useful_data = historical_runner._rcon_capture_has_new_useful_data
|
||||||
|
original_generate_historical_snapshots = historical_runner.generate_historical_snapshots
|
||||||
|
original_build_elo_mmr_rebuild_policy = historical_runner._build_elo_mmr_rebuild_policy
|
||||||
|
original_rebuild_elo_mmr_models = historical_runner.rebuild_elo_mmr_models
|
||||||
|
original_refresh_player_search_index = historical_runner.refresh_player_search_index
|
||||||
|
original_refresh_player_period_stats = historical_runner.refresh_player_period_stats
|
||||||
|
original_refresh_ranking_snapshots = historical_runner.refresh_ranking_snapshots
|
||||||
|
original_maybe_run_database_maintenance = historical_runner._maybe_run_database_maintenance
|
||||||
|
original_emit_json_log = historical_runner._emit_json_log
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def fake_backend_writer_lock(holder):
|
||||||
|
yield
|
||||||
|
|
||||||
|
captured_logs = []
|
||||||
|
call_order = []
|
||||||
|
|
||||||
|
historical_runner.backend_writer_lock = fake_backend_writer_lock
|
||||||
|
historical_runner.build_writer_lock_holder = lambda label: label
|
||||||
|
historical_runner._run_primary_rcon_capture = lambda: {
|
||||||
|
"status": "ok",
|
||||||
|
"targets": [{"target_key": "comunidad-hispana-01", "sample_inserted": True}],
|
||||||
|
"totals": {
|
||||||
|
"samples_inserted": 1,
|
||||||
|
"admin_log_events_inserted": 0,
|
||||||
|
"materialized_matches_inserted": 0,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
historical_runner._resolve_classic_fallback_policy = lambda **kwargs: (
|
||||||
|
False,
|
||||||
|
"validation-rcon-primary-cycle",
|
||||||
|
)
|
||||||
|
historical_runner._rcon_capture_has_new_useful_data = lambda payload: True
|
||||||
|
historical_runner.generate_historical_snapshots = lambda **kwargs: {
|
||||||
|
"status": "ok",
|
||||||
|
"generated_at": "2026-06-09T08:00:00Z",
|
||||||
|
}
|
||||||
|
historical_runner._build_elo_mmr_rebuild_policy = lambda **kwargs: {
|
||||||
|
"due": False,
|
||||||
|
"policy": "validation-policy",
|
||||||
|
"last_generated_at": None,
|
||||||
|
"samples_since_last_rebuild": 1,
|
||||||
|
"minutes_since_last_rebuild": None,
|
||||||
|
"rebuild_interval_minutes": 60,
|
||||||
|
"min_new_samples": 10,
|
||||||
|
}
|
||||||
|
historical_runner.rebuild_elo_mmr_models = lambda: {"status": "ok"}
|
||||||
|
historical_runner._maybe_run_database_maintenance = lambda: {
|
||||||
|
"status": "skipped",
|
||||||
|
"reason": "validation-disabled",
|
||||||
|
}
|
||||||
|
historical_runner._emit_json_log = lambda payload: captured_logs.append(payload)
|
||||||
|
|
||||||
|
def fake_refresh_player_search_index(**kwargs):
|
||||||
|
call_order.append("player-search-index")
|
||||||
|
return {"status": "ok", "total_rows": 3}
|
||||||
|
|
||||||
|
def fake_refresh_player_period_stats(**kwargs):
|
||||||
|
call_order.append("player-period-stats")
|
||||||
|
return {"status": "ok", "total_rows": 9}
|
||||||
|
|
||||||
|
def fake_refresh_ranking_snapshots(**kwargs):
|
||||||
|
call_order.append("ranking-snapshots")
|
||||||
|
return {
|
||||||
|
"status": "ok",
|
||||||
|
"totals": {"succeeded": 36, "failed": 0, "skipped_regeneration": 0},
|
||||||
|
}
|
||||||
|
|
||||||
|
historical_runner.refresh_player_search_index = fake_refresh_player_search_index
|
||||||
|
historical_runner.refresh_player_period_stats = fake_refresh_player_period_stats
|
||||||
|
historical_runner.refresh_ranking_snapshots = fake_refresh_ranking_snapshots
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = historical_runner._run_refresh_with_retries(
|
||||||
|
max_retries=0,
|
||||||
|
retry_delay_seconds=0,
|
||||||
|
server_slug=None,
|
||||||
|
max_pages=None,
|
||||||
|
page_size=None,
|
||||||
|
run_number=1,
|
||||||
|
)
|
||||||
|
require(result.get("status") == "ok", "Historical runner should return ok when all periodic refresh steps succeed.")
|
||||||
|
require(
|
||||||
|
call_order == ["player-search-index", "player-period-stats", "ranking-snapshots"],
|
||||||
|
"Historical runner should refresh player_search_index, then player_period_stats, then ranking_snapshots.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
(result.get("player_search_index_result") or {}).get("status") == "ok",
|
||||||
|
"Historical runner should report player_search_index_result separately.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
(result.get("player_period_stats_result") or {}).get("status") == "ok",
|
||||||
|
"Historical runner should report player_period_stats_result separately.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
(result.get("ranking_snapshot_result") or {}).get("status") == "ok",
|
||||||
|
"Historical runner should keep reporting ranking_snapshot_result separately.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
any(
|
||||||
|
log.get("event") == "player-search-index-refresh-started"
|
||||||
|
for log in captured_logs
|
||||||
|
if isinstance(log, dict)
|
||||||
|
),
|
||||||
|
"Historical runner should log the player_search_index refresh start event.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
any(
|
||||||
|
log.get("event") == "player-period-stats-refresh-started"
|
||||||
|
for log in captured_logs
|
||||||
|
if isinstance(log, dict)
|
||||||
|
),
|
||||||
|
"Historical runner should log the player_period_stats refresh start event.",
|
||||||
|
)
|
||||||
|
|
||||||
|
call_order.clear()
|
||||||
|
captured_logs.clear()
|
||||||
|
|
||||||
|
def fake_refresh_player_search_index_failure(**kwargs):
|
||||||
|
call_order.append("player-search-index")
|
||||||
|
raise RuntimeError("forced player search refresh failure")
|
||||||
|
|
||||||
|
historical_runner.refresh_player_search_index = fake_refresh_player_search_index_failure
|
||||||
|
result = historical_runner._run_refresh_with_retries(
|
||||||
|
max_retries=0,
|
||||||
|
retry_delay_seconds=0,
|
||||||
|
server_slug=None,
|
||||||
|
max_pages=None,
|
||||||
|
page_size=None,
|
||||||
|
run_number=1,
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
result.get("status") == "partial",
|
||||||
|
"Historical runner should return partial when one read-model refresh fails but the cycle continues.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
call_order == ["player-search-index", "player-period-stats", "ranking-snapshots"],
|
||||||
|
"Historical runner should still attempt the remaining refresh steps after a player search read-model failure.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
(result.get("player_search_index_result") or {}).get("status") == "error",
|
||||||
|
"Historical runner should expose the player_search_index refresh failure explicitly.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
(result.get("player_period_stats_result") or {}).get("status") == "ok",
|
||||||
|
"Historical runner should continue refreshing player_period_stats after a player_search_index failure.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
(result.get("ranking_snapshot_result") or {}).get("status") == "ok",
|
||||||
|
"Historical runner should continue refreshing ranking snapshots after a player_search_index failure.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
any(
|
||||||
|
log.get("event") == "player-search-index-refresh-failed"
|
||||||
|
for log in captured_logs
|
||||||
|
if isinstance(log, dict)
|
||||||
|
),
|
||||||
|
"Historical runner should emit a dedicated failure log for player_search_index refresh failures.",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
historical_runner.backend_writer_lock = original_backend_writer_lock
|
||||||
|
historical_runner.build_writer_lock_holder = original_build_writer_lock_holder
|
||||||
|
historical_runner._run_primary_rcon_capture = original_run_primary_rcon_capture
|
||||||
|
historical_runner._resolve_classic_fallback_policy = original_resolve_classic_fallback_policy
|
||||||
|
historical_runner._rcon_capture_has_new_useful_data = original_rcon_capture_has_new_useful_data
|
||||||
|
historical_runner.generate_historical_snapshots = original_generate_historical_snapshots
|
||||||
|
historical_runner._build_elo_mmr_rebuild_policy = original_build_elo_mmr_rebuild_policy
|
||||||
|
historical_runner.rebuild_elo_mmr_models = original_rebuild_elo_mmr_models
|
||||||
|
historical_runner.refresh_player_search_index = original_refresh_player_search_index
|
||||||
|
historical_runner.refresh_player_period_stats = original_refresh_player_period_stats
|
||||||
|
historical_runner.refresh_ranking_snapshots = original_refresh_ranking_snapshots
|
||||||
|
historical_runner._maybe_run_database_maintenance = original_maybe_run_database_maintenance
|
||||||
|
historical_runner._emit_json_log = original_emit_json_log
|
||||||
|
|
||||||
|
|
||||||
def cleanup_snapshot_fixture(db_path):
|
def cleanup_snapshot_fixture(db_path):
|
||||||
connection = sqlite3.connect(db_path)
|
connection = sqlite3.connect(db_path)
|
||||||
with connection:
|
with connection:
|
||||||
@@ -1271,6 +1452,7 @@ validate_player_period_stats_cli_defaults()
|
|||||||
validate_fetch_player_stats_sql_contract()
|
validate_fetch_player_stats_sql_contract()
|
||||||
validate_player_period_stats_refresh_and_read_model()
|
validate_player_period_stats_refresh_and_read_model()
|
||||||
validate_player_period_stats_fallbacks()
|
validate_player_period_stats_fallbacks()
|
||||||
|
validate_historical_runner_read_model_refresh_cycle()
|
||||||
|
|
||||||
kd_metric_sql, _, _ = ranking_leaderboards._resolve_metric_sql("kd_ratio")
|
kd_metric_sql, _, _ = ranking_leaderboards._resolve_metric_sql("kd_ratio")
|
||||||
require(
|
require(
|
||||||
@@ -1654,6 +1836,7 @@ print(json.dumps({
|
|||||||
"ranking-snapshot-missing",
|
"ranking-snapshot-missing",
|
||||||
"player-search-index",
|
"player-search-index",
|
||||||
"player-period-stats",
|
"player-period-stats",
|
||||||
|
"historical-runner-player-read-model-refresh",
|
||||||
],
|
],
|
||||||
"annual_snapshot_status": annual_data.get("snapshot_status"),
|
"annual_snapshot_status": annual_data.get("snapshot_status"),
|
||||||
"global_ranking_annual_snapshot_status": annual_ranking_data.get("snapshot_status"),
|
"global_ranking_annual_snapshot_status": annual_ranking_data.get("snapshot_status"),
|
||||||
|
|||||||
Reference in New Issue
Block a user