Add weekly monthly ranking snapshot generator
This commit is contained in:
@@ -0,0 +1,149 @@
|
|||||||
|
---
|
||||||
|
id: TASK-194-add-weekly-monthly-ranking-snapshot-generator
|
||||||
|
title: Add weekly monthly ranking snapshot generator
|
||||||
|
status: done
|
||||||
|
type: backend
|
||||||
|
team: Backend Senior
|
||||||
|
supporting_teams:
|
||||||
|
- Arquitecto de Base de Datos
|
||||||
|
- Arquitecto Python
|
||||||
|
roadmap_item: foundation
|
||||||
|
priority: high
|
||||||
|
---
|
||||||
|
|
||||||
|
# TASK-194 - Add weekly monthly ranking snapshot generator
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Implement a manual weekly/monthly ranking snapshot generator plus CLI so PostgreSQL-backed `/api/ranking` can serve persisted global ranking snapshots instead of runtime fallback when operators generate the required combinations.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
`TASK-191` introduced the snapshot-first read path for weekly/monthly global ranking, and `TASK-192` / `TASK-193` stabilized the PostgreSQL path and runtime fallback metrics. The missing piece is still out-of-band generation for `ranking_snapshots` and `ranking_snapshot_items`, so production ranking requests continue to return `snapshot_status=missing`, `freshness=runtime` and `fallback_used=true`.
|
||||||
|
|
||||||
|
Preserve the current product identity: Spanish-speaking HLL Vietnam community, military/Vietnam/tactical/sober visual direction and controlled repository evolution.
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
1. Read the listed files first.
|
||||||
|
2. Add a manual generator for weekly/monthly ranking snapshots using the existing materialized RCON leaderboard logic and snapshot tables.
|
||||||
|
3. Expose a CLI command for one explicit combination; only add matrix generation if it remains small and justified.
|
||||||
|
4. Validate generated snapshot reads, preserved runtime fallback behavior and production-oriented run commands.
|
||||||
|
5. Document the manual generation workflow and recommended combinations.
|
||||||
|
|
||||||
|
## Files to Read First
|
||||||
|
|
||||||
|
- `AGENTS.md`
|
||||||
|
- `ai/repo-context.md`
|
||||||
|
- `ai/architecture-index.md`
|
||||||
|
- `backend/app/rcon_historical_leaderboards.py`
|
||||||
|
- `backend/app/rcon_annual_rankings.py`
|
||||||
|
- `backend/app/routes.py`
|
||||||
|
- `backend/app/payloads.py`
|
||||||
|
- `backend/app/postgres_rcon_storage.py`
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- `docs/ranking-stats-performance-audit.md`
|
||||||
|
- `ai/tasks/done/TASK-191-serve-ranking-from-snapshots-with-runtime-fallback.md`
|
||||||
|
- `ai/tasks/done/TASK-192-fix-postgres-ranking-snapshot-schema.md`
|
||||||
|
- `ai/tasks/done/TASK-193-fix-postgres-ranking-derived-metrics.md`
|
||||||
|
|
||||||
|
## Expected Files to Modify
|
||||||
|
|
||||||
|
- `backend/app/rcon_historical_leaderboards.py`
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- `ai/tasks/done/TASK-194-add-weekly-monthly-ranking-snapshot-generator.md`
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
- Keep the change minimal.
|
||||||
|
- Do not modify frontend, assets or design.
|
||||||
|
- Do not reactivate Elo/MMR.
|
||||||
|
- Do not reintroduce Comunidad Hispana #03.
|
||||||
|
- Keep annual ranking behavior unchanged unless a helper can be safely shared without changing annual contracts.
|
||||||
|
- Prefer the existing module pattern already used by `backend/app/rcon_annual_rankings.py`.
|
||||||
|
- Do not change `/api/ranking` unless strictly necessary for validation-safe compatibility.
|
||||||
|
- Keep PostgreSQL as the operational target and preserve SQLite compatibility already present in the shared backend path.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
Before completing the task ensure:
|
||||||
|
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||||
|
- local import validation proves:
|
||||||
|
- `generate_ranking_snapshot(timeframe="weekly", server_key="all", metric="kills", limit=20)` creates a snapshot
|
||||||
|
- `get_latest_ranking_snapshot(...)` can read that snapshot
|
||||||
|
- `/api/ranking?timeframe=weekly&server_id=all&metric=kills&limit=20` returns `snapshot_status=ready`
|
||||||
|
- the same response returns `fallback_used=false`
|
||||||
|
- requesting a combination without a snapshot still preserves runtime fallback when enabled
|
||||||
|
- document the manual CLI command for local and Docker execution
|
||||||
|
- `git diff --name-only` matches the expected scope
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Implemented:
|
||||||
|
|
||||||
|
- `backend/app/rcon_historical_leaderboards.py`
|
||||||
|
- `generate_ranking_snapshot(...)`
|
||||||
|
- persistence helpers for `ranking_snapshots` and `ranking_snapshot_items`
|
||||||
|
- manual CLI entrypoint:
|
||||||
|
- `python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20`
|
||||||
|
- `scripts/run-stats-validation.ps1`
|
||||||
|
- now validates a real generated weekly snapshot through:
|
||||||
|
- direct function import
|
||||||
|
- `get_latest_ranking_snapshot(...)`
|
||||||
|
- `/api/ranking`
|
||||||
|
- confirms generated snapshot reads return:
|
||||||
|
- `snapshot_status=ready`
|
||||||
|
- `fallback_used=false`
|
||||||
|
- confirms runtime fallback still works for a missing combination
|
||||||
|
- `docs/ranking-snapshot-read-model-plan.md`
|
||||||
|
- documents the manual command
|
||||||
|
- documents the recommended weekly/monthly/server/metric matrix
|
||||||
|
- documents suggested refresh cadence
|
||||||
|
- documents the expected `ready` vs fallback response states
|
||||||
|
|
||||||
|
Supported generation scope:
|
||||||
|
|
||||||
|
- `timeframe`
|
||||||
|
- `weekly`
|
||||||
|
- `monthly`
|
||||||
|
- `server-key`
|
||||||
|
- `all`
|
||||||
|
- `all-servers`
|
||||||
|
- `comunidad-hispana-01`
|
||||||
|
- `comunidad-hispana-02`
|
||||||
|
- `metric`
|
||||||
|
- `kills`
|
||||||
|
- `deaths`
|
||||||
|
- `teamkills`
|
||||||
|
- `matches_considered`
|
||||||
|
- `kd_ratio`
|
||||||
|
- `kills_per_match`
|
||||||
|
|
||||||
|
Validations executed:
|
||||||
|
|
||||||
|
- `python -m compileall backend/app/rcon_historical_leaderboards.py backend/app/payloads.py backend/app/routes.py backend/app/postgres_rcon_storage.py`
|
||||||
|
- direct local import probe for `generate_ranking_snapshot(...)`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||||
|
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||||
|
- local CLI probe:
|
||||||
|
- `python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20`
|
||||||
|
|
||||||
|
Validation notes:
|
||||||
|
|
||||||
|
- Live backend HTTP at `http://127.0.0.1:8000` was not available, so route validation completed through local imports and repository scripts.
|
||||||
|
- The manual CLI is unitary per `(timeframe, server-key, metric)` combination.
|
||||||
|
- A broader matrix helper was intentionally left out to keep the task small and verifiable.
|
||||||
|
|
||||||
|
Recommended Docker command for production:
|
||||||
|
|
||||||
|
- `docker compose exec backend python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20`
|
||||||
|
|
||||||
|
## Change Budget
|
||||||
|
|
||||||
|
- Prefer fewer than 5 modified files.
|
||||||
|
- Prefer changes under 200 lines when feasible.
|
||||||
|
- Split the work into follow-up tasks if limits are exceeded.
|
||||||
@@ -2,6 +2,8 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
import os
|
import os
|
||||||
from contextlib import closing
|
from contextlib import closing
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
@@ -29,6 +31,21 @@ LeaderboardMetric = Literal[
|
|||||||
"support",
|
"support",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
SNAPSHOT_GENERATOR_TIMEFRAMES = ("weekly", "monthly")
|
||||||
|
SNAPSHOT_GENERATOR_SERVER_KEYS = (
|
||||||
|
ALL_SERVERS_SLUG,
|
||||||
|
"comunidad-hispana-01",
|
||||||
|
"comunidad-hispana-02",
|
||||||
|
)
|
||||||
|
SNAPSHOT_GENERATOR_METRICS = (
|
||||||
|
"kills",
|
||||||
|
"deaths",
|
||||||
|
"teamkills",
|
||||||
|
"matches_considered",
|
||||||
|
"kd_ratio",
|
||||||
|
"kills_per_match",
|
||||||
|
)
|
||||||
|
|
||||||
RANKING_SNAPSHOT_SQLITE_SCHEMA_SQL = """
|
RANKING_SNAPSHOT_SQLITE_SCHEMA_SQL = """
|
||||||
CREATE TABLE IF NOT EXISTS ranking_snapshots (
|
CREATE TABLE IF NOT EXISTS ranking_snapshots (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
@@ -102,6 +119,107 @@ def is_ranking_runtime_fallback_enabled() -> bool:
|
|||||||
return normalized in {"1", "true", "yes", "on"}
|
return normalized in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
def generate_ranking_snapshot(
|
||||||
|
*,
|
||||||
|
timeframe: str,
|
||||||
|
server_key: str | None,
|
||||||
|
metric: str,
|
||||||
|
limit: int,
|
||||||
|
replace_existing: bool = True,
|
||||||
|
now: datetime | None = None,
|
||||||
|
db_path: Path | None = None,
|
||||||
|
) -> dict[str, object]:
|
||||||
|
"""Generate and persist one weekly/monthly ranking snapshot."""
|
||||||
|
normalized_timeframe = _normalize_generator_timeframe(timeframe)
|
||||||
|
normalized_server_key = _normalize_generator_server_key(server_key)
|
||||||
|
normalized_metric = _normalize_generator_metric(metric)
|
||||||
|
normalized_limit = _normalize_limit(limit)
|
||||||
|
anchor = _as_utc(now or datetime.now(timezone.utc))
|
||||||
|
|
||||||
|
resolved_path = initialize_ranking_snapshot_storage(db_path=db_path)
|
||||||
|
connection_scope = _connect_write_scope(resolved_path, db_path=db_path)
|
||||||
|
with connection_scope as connection:
|
||||||
|
window = select_leaderboard_window(
|
||||||
|
connection=connection,
|
||||||
|
server_key=normalized_server_key,
|
||||||
|
timeframe=normalized_timeframe,
|
||||||
|
now=anchor,
|
||||||
|
)
|
||||||
|
window_start = _to_iso(window["start"])
|
||||||
|
window_end = _to_iso(window["end"])
|
||||||
|
existing_snapshot_id = _find_existing_snapshot_for_window(
|
||||||
|
connection=connection,
|
||||||
|
timeframe=normalized_timeframe,
|
||||||
|
server_key=normalized_server_key,
|
||||||
|
metric=normalized_metric,
|
||||||
|
window_start=window_start,
|
||||||
|
window_end=window_end,
|
||||||
|
)
|
||||||
|
if existing_snapshot_id is not None and not replace_existing:
|
||||||
|
snapshot = _get_snapshot_record(connection=connection, snapshot_id=existing_snapshot_id)
|
||||||
|
items = _list_snapshot_items(connection=connection, snapshot_id=existing_snapshot_id)
|
||||||
|
return {
|
||||||
|
"status": "ok",
|
||||||
|
"snapshot": snapshot,
|
||||||
|
"items": items,
|
||||||
|
"source_matches_count": int(snapshot.get("source_matches_count") or 0)
|
||||||
|
if snapshot
|
||||||
|
else 0,
|
||||||
|
"ranked_players": len(items),
|
||||||
|
"skipped_regeneration": True,
|
||||||
|
}
|
||||||
|
|
||||||
|
ranking_rows = _fetch_leaderboard_rows(
|
||||||
|
connection,
|
||||||
|
server_key=normalized_server_key,
|
||||||
|
metric=normalized_metric,
|
||||||
|
limit=normalized_limit,
|
||||||
|
window_start=window["start"],
|
||||||
|
window_end=window["end"],
|
||||||
|
)
|
||||||
|
source_matches_count = _count_matches(
|
||||||
|
connection,
|
||||||
|
server_key=normalized_server_key,
|
||||||
|
start=window["start"],
|
||||||
|
end=window["end"],
|
||||||
|
)
|
||||||
|
|
||||||
|
if existing_snapshot_id is not None:
|
||||||
|
_delete_snapshot(connection=connection, snapshot_id=existing_snapshot_id)
|
||||||
|
|
||||||
|
snapshot_id = _insert_snapshot_record(
|
||||||
|
connection=connection,
|
||||||
|
timeframe=normalized_timeframe,
|
||||||
|
server_key=normalized_server_key,
|
||||||
|
metric=normalized_metric,
|
||||||
|
limit=normalized_limit,
|
||||||
|
source_matches_count=source_matches_count,
|
||||||
|
window_start=window_start,
|
||||||
|
window_end=window_end,
|
||||||
|
generated_at=_to_iso(datetime.now(timezone.utc)),
|
||||||
|
window_kind=str(window["kind"]),
|
||||||
|
window_label=str(window["label"]),
|
||||||
|
item_count=len(ranking_rows),
|
||||||
|
)
|
||||||
|
_insert_snapshot_items(
|
||||||
|
connection=connection,
|
||||||
|
snapshot_id=snapshot_id,
|
||||||
|
rows=ranking_rows,
|
||||||
|
limit=normalized_limit,
|
||||||
|
)
|
||||||
|
snapshot = _get_snapshot_record(connection=connection, snapshot_id=snapshot_id)
|
||||||
|
items = _list_snapshot_items(connection=connection, snapshot_id=snapshot_id)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"status": "ok",
|
||||||
|
"snapshot": snapshot,
|
||||||
|
"items": items,
|
||||||
|
"source_matches_count": source_matches_count,
|
||||||
|
"ranked_players": len(items),
|
||||||
|
"skipped_regeneration": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def get_latest_ranking_snapshot(
|
def get_latest_ranking_snapshot(
|
||||||
*,
|
*,
|
||||||
server_key: str | None = None,
|
server_key: str | None = None,
|
||||||
@@ -383,6 +501,64 @@ def _find_latest_snapshot(
|
|||||||
return dict(row) if row else None
|
return dict(row) if row else None
|
||||||
|
|
||||||
|
|
||||||
|
def _find_existing_snapshot_for_window(
|
||||||
|
*,
|
||||||
|
connection: object,
|
||||||
|
timeframe: str,
|
||||||
|
server_key: str,
|
||||||
|
metric: str,
|
||||||
|
window_start: str,
|
||||||
|
window_end: str,
|
||||||
|
) -> int | None:
|
||||||
|
row = connection.execute(
|
||||||
|
"""
|
||||||
|
SELECT id
|
||||||
|
FROM ranking_snapshots
|
||||||
|
WHERE timeframe = ?
|
||||||
|
AND server_id = ?
|
||||||
|
AND metric = ?
|
||||||
|
AND window_start = ?
|
||||||
|
AND window_end = ?
|
||||||
|
LIMIT 1
|
||||||
|
""",
|
||||||
|
[timeframe, server_key, metric, window_start, window_end],
|
||||||
|
).fetchone()
|
||||||
|
return int(row["id"]) if row else None
|
||||||
|
|
||||||
|
|
||||||
|
def _get_snapshot_record(
|
||||||
|
*,
|
||||||
|
connection: object,
|
||||||
|
snapshot_id: int,
|
||||||
|
) -> dict[str, object] | None:
|
||||||
|
row = connection.execute(
|
||||||
|
"""
|
||||||
|
SELECT
|
||||||
|
id,
|
||||||
|
timeframe,
|
||||||
|
server_id,
|
||||||
|
metric,
|
||||||
|
window_start,
|
||||||
|
window_end,
|
||||||
|
generated_at,
|
||||||
|
source,
|
||||||
|
snapshot_status,
|
||||||
|
item_count,
|
||||||
|
limit_size,
|
||||||
|
source_matches_count,
|
||||||
|
freshness,
|
||||||
|
window_kind,
|
||||||
|
window_label,
|
||||||
|
error_message
|
||||||
|
FROM ranking_snapshots
|
||||||
|
WHERE id = ?
|
||||||
|
LIMIT 1
|
||||||
|
""",
|
||||||
|
[snapshot_id],
|
||||||
|
).fetchone()
|
||||||
|
return dict(row) if row else None
|
||||||
|
|
||||||
|
|
||||||
def _list_snapshot_items(
|
def _list_snapshot_items(
|
||||||
*,
|
*,
|
||||||
connection: object,
|
connection: object,
|
||||||
@@ -413,6 +589,115 @@ def _list_snapshot_items(
|
|||||||
return [dict(row) for row in rows]
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
|
||||||
|
def _delete_snapshot(*, connection: object, snapshot_id: int) -> None:
|
||||||
|
connection.execute(
|
||||||
|
"DELETE FROM ranking_snapshot_items WHERE snapshot_id = ?",
|
||||||
|
[snapshot_id],
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"DELETE FROM ranking_snapshots WHERE id = ?",
|
||||||
|
[snapshot_id],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _insert_snapshot_record(
|
||||||
|
*,
|
||||||
|
connection: object,
|
||||||
|
timeframe: str,
|
||||||
|
server_key: str,
|
||||||
|
metric: str,
|
||||||
|
limit: int,
|
||||||
|
source_matches_count: int,
|
||||||
|
window_start: str,
|
||||||
|
window_end: str,
|
||||||
|
generated_at: str,
|
||||||
|
window_kind: str,
|
||||||
|
window_label: str,
|
||||||
|
item_count: int,
|
||||||
|
) -> int:
|
||||||
|
cursor = connection.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO ranking_snapshots (
|
||||||
|
timeframe,
|
||||||
|
server_id,
|
||||||
|
metric,
|
||||||
|
window_start,
|
||||||
|
window_end,
|
||||||
|
generated_at,
|
||||||
|
source,
|
||||||
|
snapshot_status,
|
||||||
|
item_count,
|
||||||
|
limit_size,
|
||||||
|
source_matches_count,
|
||||||
|
freshness,
|
||||||
|
window_kind,
|
||||||
|
window_label,
|
||||||
|
error_message
|
||||||
|
) VALUES (?, ?, ?, ?, ?, ?, ?, 'ready', ?, ?, ?, 'fresh', ?, ?, NULL)
|
||||||
|
RETURNING id
|
||||||
|
""",
|
||||||
|
[
|
||||||
|
timeframe,
|
||||||
|
server_key,
|
||||||
|
metric,
|
||||||
|
window_start,
|
||||||
|
window_end,
|
||||||
|
generated_at,
|
||||||
|
"rcon-materialized-admin-log",
|
||||||
|
item_count,
|
||||||
|
limit,
|
||||||
|
source_matches_count,
|
||||||
|
window_kind,
|
||||||
|
window_label,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
row = cursor.fetchone()
|
||||||
|
if row is None or row["id"] is None:
|
||||||
|
raise RuntimeError("Unable to resolve ranking snapshot id after insert.")
|
||||||
|
return int(row["id"])
|
||||||
|
|
||||||
|
|
||||||
|
def _insert_snapshot_items(
|
||||||
|
*,
|
||||||
|
connection: object,
|
||||||
|
snapshot_id: int,
|
||||||
|
rows: list[dict[str, object]],
|
||||||
|
limit: int,
|
||||||
|
) -> None:
|
||||||
|
for index, row in enumerate(rows[:limit], start=1):
|
||||||
|
item = _build_item(row, index=index)
|
||||||
|
connection.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO ranking_snapshot_items (
|
||||||
|
snapshot_id,
|
||||||
|
ranking_position,
|
||||||
|
player_id,
|
||||||
|
player_name,
|
||||||
|
metric_value,
|
||||||
|
matches_considered,
|
||||||
|
kills,
|
||||||
|
deaths,
|
||||||
|
teamkills,
|
||||||
|
kd_ratio,
|
||||||
|
kills_per_match
|
||||||
|
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
""",
|
||||||
|
[
|
||||||
|
snapshot_id,
|
||||||
|
int(item["ranking_position"]),
|
||||||
|
str(item["player_id"]),
|
||||||
|
str(item["player_name"]),
|
||||||
|
float(item["metric_value"]),
|
||||||
|
int(item["matches_considered"]),
|
||||||
|
int(item["kills"]),
|
||||||
|
int(item["deaths"]),
|
||||||
|
int(item["teamkills"]),
|
||||||
|
float(item["kd_ratio"]),
|
||||||
|
float(item["kills_per_match"]),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _fetch_leaderboard_rows(
|
def _fetch_leaderboard_rows(
|
||||||
connection: object,
|
connection: object,
|
||||||
*,
|
*,
|
||||||
@@ -702,6 +987,14 @@ def _connect_scope(resolved_path: Path, *, db_path: Path | None):
|
|||||||
return closing(connect_sqlite_readonly(resolved_path))
|
return closing(connect_sqlite_readonly(resolved_path))
|
||||||
|
|
||||||
|
|
||||||
|
def _connect_write_scope(resolved_path: Path, *, db_path: Path | None):
|
||||||
|
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
|
||||||
|
from .postgres_rcon_storage import connect_postgres_compat
|
||||||
|
|
||||||
|
return connect_postgres_compat()
|
||||||
|
return connect_sqlite_writer(resolved_path)
|
||||||
|
|
||||||
|
|
||||||
def _empty_payload(
|
def _empty_payload(
|
||||||
*,
|
*,
|
||||||
server_key: str | None,
|
server_key: str | None,
|
||||||
@@ -785,6 +1078,40 @@ def _normalize_timeframe(value: str) -> LeaderboardTimeframe:
|
|||||||
return "monthly" if str(value or "").strip().lower() == "monthly" else "weekly"
|
return "monthly" if str(value or "").strip().lower() == "monthly" else "weekly"
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_generator_timeframe(value: str) -> str:
|
||||||
|
normalized = str(value or "").strip().lower()
|
||||||
|
if normalized not in SNAPSHOT_GENERATOR_TIMEFRAMES:
|
||||||
|
raise ValueError(
|
||||||
|
f"timeframe must be one of: {', '.join(SNAPSHOT_GENERATOR_TIMEFRAMES)}"
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_generator_server_key(server_key: str | None) -> str:
|
||||||
|
normalized = _normalize_snapshot_server_key(server_key)
|
||||||
|
if normalized not in SNAPSHOT_GENERATOR_SERVER_KEYS:
|
||||||
|
raise ValueError(
|
||||||
|
"server_key must be one of: all, all-servers, comunidad-hispana-01, comunidad-hispana-02"
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_generator_metric(metric: str) -> str:
|
||||||
|
normalized = str(metric or "").strip().lower()
|
||||||
|
if normalized not in SNAPSHOT_GENERATOR_METRICS:
|
||||||
|
raise ValueError(
|
||||||
|
f"metric must be one of: {', '.join(SNAPSHOT_GENERATOR_METRICS)}"
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_limit(limit: object, *, maximum: int = 100) -> int:
|
||||||
|
normalized_limit = int(limit or 1)
|
||||||
|
if normalized_limit < 1:
|
||||||
|
raise ValueError("limit must be greater than zero")
|
||||||
|
return min(normalized_limit, maximum)
|
||||||
|
|
||||||
|
|
||||||
def _normalize_metric(value: str) -> LeaderboardMetric:
|
def _normalize_metric(value: str) -> LeaderboardMetric:
|
||||||
normalized = str(value or "kills").strip().lower()
|
normalized = str(value or "kills").strip().lower()
|
||||||
if normalized in {
|
if normalized in {
|
||||||
@@ -909,3 +1236,50 @@ def _to_iso(value: object) -> str:
|
|||||||
if parsed is None:
|
if parsed is None:
|
||||||
parsed = datetime.now(timezone.utc)
|
parsed = datetime.now(timezone.utc)
|
||||||
return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
|
return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
|
|
||||||
|
def _main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Read and generate weekly/monthly ranking snapshots.",
|
||||||
|
)
|
||||||
|
subparsers = parser.add_subparsers(dest="command")
|
||||||
|
generate_parser = subparsers.add_parser("generate-ranking-snapshot")
|
||||||
|
generate_parser.add_argument(
|
||||||
|
"--timeframe",
|
||||||
|
required=True,
|
||||||
|
choices=SNAPSHOT_GENERATOR_TIMEFRAMES,
|
||||||
|
)
|
||||||
|
generate_parser.add_argument("--server-key", default=None)
|
||||||
|
generate_parser.add_argument(
|
||||||
|
"--metric",
|
||||||
|
required=True,
|
||||||
|
choices=SNAPSHOT_GENERATOR_METRICS,
|
||||||
|
)
|
||||||
|
generate_parser.add_argument("--limit", type=int, default=20)
|
||||||
|
generate_parser.add_argument(
|
||||||
|
"--no-replace-existing",
|
||||||
|
action="store_false",
|
||||||
|
dest="replace_existing",
|
||||||
|
help="keep an existing snapshot for the selected exact window and metric scope",
|
||||||
|
)
|
||||||
|
parser.set_defaults(command="generate-ranking-snapshot", replace_existing=True)
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
if args.command == "generate-ranking-snapshot":
|
||||||
|
payload = generate_ranking_snapshot(
|
||||||
|
timeframe=args.timeframe,
|
||||||
|
server_key=args.server_key,
|
||||||
|
metric=args.metric,
|
||||||
|
limit=args.limit,
|
||||||
|
replace_existing=args.replace_existing,
|
||||||
|
db_path=get_storage_path(),
|
||||||
|
)
|
||||||
|
print(json.dumps({"status": "ok", "data": payload}, ensure_ascii=True, indent=2))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
parser.print_help()
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(_main())
|
||||||
|
|||||||
@@ -245,6 +245,72 @@ Operational note after `TASK-191`:
|
|||||||
- runtime fallback remains enabled by default for transition through `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED=true`
|
- runtime fallback remains enabled by default for transition through `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED=true`
|
||||||
- operators can force controlled missing behavior by setting `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED=false`
|
- operators can force controlled missing behavior by setting `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED=false`
|
||||||
|
|
||||||
|
## Manual Generation Workflow
|
||||||
|
|
||||||
|
Manual generator entrypoint:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20
|
||||||
|
```
|
||||||
|
|
||||||
|
Docker form:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose exec backend python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported manual parameters:
|
||||||
|
- `timeframe`: `weekly`, `monthly`
|
||||||
|
- `server-key`: `all`, `all-servers`, `comunidad-hispana-01`, `comunidad-hispana-02`
|
||||||
|
- `metric`: `kills`, `deaths`, `teamkills`, `matches_considered`, `kd_ratio`, `kills_per_match`
|
||||||
|
- `limit`: positive integer, normally `20`
|
||||||
|
|
||||||
|
Current implementation note:
|
||||||
|
- V1 generator is unitary per command invocation
|
||||||
|
- operators should run one command per required `(timeframe, server-key, metric)` combination
|
||||||
|
- broad matrix generation can remain a later operational helper if scheduling needs justify it
|
||||||
|
|
||||||
|
## Recommended Combinations
|
||||||
|
|
||||||
|
Minimum production matrix for parity with the public Ranking filters:
|
||||||
|
- weekly + `all-servers` + all six supported metrics
|
||||||
|
- weekly + `comunidad-hispana-01` + all six supported metrics
|
||||||
|
- weekly + `comunidad-hispana-02` + all six supported metrics
|
||||||
|
- monthly + `all-servers` + all six supported metrics
|
||||||
|
- monthly + `comunidad-hispana-01` + all six supported metrics
|
||||||
|
- monthly + `comunidad-hispana-02` + all six supported metrics
|
||||||
|
|
||||||
|
That matrix requires `36` snapshot generations for each refresh cycle.
|
||||||
|
|
||||||
|
## Suggested Frequency
|
||||||
|
|
||||||
|
Suggested operator cadence:
|
||||||
|
- weekly current window: regenerate every `5` to `15` minutes while the active week is changing
|
||||||
|
- monthly current window: regenerate every `15` to `30` minutes while the active month is changing
|
||||||
|
- previous week and previous month windows: regenerate once after closure or after any historical backfill that changes source coverage
|
||||||
|
|
||||||
|
Operational guidance:
|
||||||
|
- regenerate after materialized RCON/AdminLog data grows
|
||||||
|
- regenerate after manual backfill
|
||||||
|
- regenerate after metric SQL changes that affect ranking totals or ordering
|
||||||
|
|
||||||
|
## Ready Vs Fallback
|
||||||
|
|
||||||
|
Expected API result after successful generation for an exact requested combination:
|
||||||
|
- `snapshot_status=ready`
|
||||||
|
- `fallback_used=false`
|
||||||
|
- `source.read_model=ranking-snapshot`
|
||||||
|
|
||||||
|
Expected API result when the requested combination has not been generated yet and fallback remains enabled:
|
||||||
|
- `snapshot_status=missing`
|
||||||
|
- `fallback_used=true`
|
||||||
|
- `freshness=runtime`
|
||||||
|
|
||||||
|
Expected API result when the requested combination has not been generated and fallback is disabled:
|
||||||
|
- `snapshot_status=missing`
|
||||||
|
- `fallback_used=false`
|
||||||
|
- `items=[]`
|
||||||
|
|
||||||
## Out Of Scope
|
## Out Of Scope
|
||||||
|
|
||||||
- backend implementation of the snapshot generator
|
- backend implementation of the snapshot generator
|
||||||
|
|||||||
@@ -127,6 +127,8 @@ import app.postgres_rcon_storage as postgres_rcon_storage
|
|||||||
import app.rcon_historical_leaderboards as ranking_leaderboards
|
import app.rcon_historical_leaderboards as ranking_leaderboards
|
||||||
|
|
||||||
initialize_ranking_snapshot_storage = ranking_leaderboards.initialize_ranking_snapshot_storage
|
initialize_ranking_snapshot_storage = ranking_leaderboards.initialize_ranking_snapshot_storage
|
||||||
|
generate_ranking_snapshot = ranking_leaderboards.generate_ranking_snapshot
|
||||||
|
get_latest_ranking_snapshot = ranking_leaderboards.get_latest_ranking_snapshot
|
||||||
|
|
||||||
|
|
||||||
def require(condition, message):
|
def require(condition, message):
|
||||||
@@ -301,6 +303,23 @@ def cleanup_snapshot_fixture(db_path):
|
|||||||
connection.close()
|
connection.close()
|
||||||
|
|
||||||
|
|
||||||
|
def cleanup_generated_snapshot(snapshot_id):
|
||||||
|
if not snapshot_id:
|
||||||
|
return
|
||||||
|
db_path = Path("backend/data/hll_vietnam_dev.sqlite3")
|
||||||
|
connection = sqlite3.connect(db_path)
|
||||||
|
with connection:
|
||||||
|
connection.execute(
|
||||||
|
"DELETE FROM ranking_snapshot_items WHERE snapshot_id = ?",
|
||||||
|
(snapshot_id,),
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"DELETE FROM ranking_snapshots WHERE id = ?",
|
||||||
|
(snapshot_id,),
|
||||||
|
)
|
||||||
|
connection.close()
|
||||||
|
|
||||||
|
|
||||||
health_status, health_payload = read_payload("/health")
|
health_status, health_payload = read_payload("/health")
|
||||||
require(health_status == 200, "Route resolver /health should return 200.")
|
require(health_status == 200, "Route resolver /health should return 200.")
|
||||||
require(health_payload.get("status") == "ok", "/health payload should be ok.")
|
require(health_payload.get("status") == "ok", "/health payload should be ok.")
|
||||||
@@ -567,6 +586,73 @@ missing_year_ranking_status, _ = read_payload(
|
|||||||
)
|
)
|
||||||
require(missing_year_ranking_status == 400, "Global ranking annual requests without year should return 400.")
|
require(missing_year_ranking_status == 400, "Global ranking annual requests without year should return 400.")
|
||||||
|
|
||||||
|
generated_snapshot_id = None
|
||||||
|
try:
|
||||||
|
generated_snapshot = generate_ranking_snapshot(
|
||||||
|
timeframe="weekly",
|
||||||
|
server_key="all",
|
||||||
|
metric="kills",
|
||||||
|
limit=20,
|
||||||
|
now=datetime(2026, 5, 20, 12, 0, 0, tzinfo=timezone.utc),
|
||||||
|
db_path=Path("backend/data/hll_vietnam_dev.sqlite3"),
|
||||||
|
)
|
||||||
|
generated_snapshot_record = generated_snapshot.get("snapshot") or {}
|
||||||
|
generated_snapshot_id = generated_snapshot_record.get("id")
|
||||||
|
require(generated_snapshot.get("status") == "ok", "Weekly ranking snapshot generation should return ok.")
|
||||||
|
require(generated_snapshot_id, "Generated weekly ranking snapshot should expose snapshot id.")
|
||||||
|
require(
|
||||||
|
generated_snapshot_record.get("timeframe") == "weekly",
|
||||||
|
"Generated weekly ranking snapshot should preserve timeframe.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
generated_snapshot_record.get("metric") == "kills",
|
||||||
|
"Generated weekly ranking snapshot should preserve metric.",
|
||||||
|
)
|
||||||
|
latest_generated_snapshot = get_latest_ranking_snapshot(
|
||||||
|
server_key="all",
|
||||||
|
timeframe="weekly",
|
||||||
|
metric="kills",
|
||||||
|
limit=20,
|
||||||
|
db_path=Path("backend/data/hll_vietnam_dev.sqlite3"),
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
latest_generated_snapshot.get("snapshot_status") == "ready",
|
||||||
|
"Generated weekly ranking snapshot should be readable as ready.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
latest_generated_snapshot.get("generated_at"),
|
||||||
|
"Generated weekly ranking snapshot should expose generated_at.",
|
||||||
|
)
|
||||||
|
generated_route_status, generated_route_payload = read_payload(
|
||||||
|
"/api/ranking?timeframe=weekly&server_id=all&metric=kills&limit=20"
|
||||||
|
)
|
||||||
|
require(generated_route_status == 200, "Generated weekly ranking route should return 200.")
|
||||||
|
generated_route_data = generated_route_payload.get("data") or {}
|
||||||
|
require(
|
||||||
|
generated_route_data.get("snapshot_status") == "ready",
|
||||||
|
"Generated weekly ranking route should return snapshot_status=ready.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
generated_route_data.get("fallback_used") is False,
|
||||||
|
"Generated weekly ranking route should not use runtime fallback.",
|
||||||
|
)
|
||||||
|
require(
|
||||||
|
isinstance(generated_route_data.get("items"), list),
|
||||||
|
"Generated weekly ranking route should return items list.",
|
||||||
|
)
|
||||||
|
|
||||||
|
generated_fallback_status, generated_fallback_payload = read_payload(
|
||||||
|
"/api/ranking?timeframe=weekly&server_id=all&metric=deaths&limit=20"
|
||||||
|
)
|
||||||
|
require(generated_fallback_status == 200, "Generated validation fallback route should return 200.")
|
||||||
|
generated_fallback_data = generated_fallback_payload.get("data") or {}
|
||||||
|
require(
|
||||||
|
generated_fallback_data.get("fallback_used") is True,
|
||||||
|
"Missing weekly deaths snapshot should still use runtime fallback when enabled.",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
cleanup_generated_snapshot(generated_snapshot_id)
|
||||||
|
|
||||||
fixture_db_path = build_snapshot_fixture()
|
fixture_db_path = build_snapshot_fixture()
|
||||||
try:
|
try:
|
||||||
fixture_weekly_status, fixture_weekly_payload = read_payload(
|
fixture_weekly_status, fixture_weekly_payload = read_payload(
|
||||||
@@ -610,6 +696,7 @@ print(json.dumps({
|
|||||||
"global-ranking",
|
"global-ranking",
|
||||||
"postgres-ranking-derived-metric-sql",
|
"postgres-ranking-derived-metric-sql",
|
||||||
"postgres-ranking-schema-path",
|
"postgres-ranking-schema-path",
|
||||||
|
"ranking-snapshot-generator",
|
||||||
"ranking-snapshot-ready",
|
"ranking-snapshot-ready",
|
||||||
"ranking-snapshot-missing",
|
"ranking-snapshot-missing",
|
||||||
],
|
],
|
||||||
|
|||||||
Reference in New Issue
Block a user