Add weekly monthly ranking snapshot generator
This commit is contained in:
@@ -0,0 +1,149 @@
|
||||
---
|
||||
id: TASK-194-add-weekly-monthly-ranking-snapshot-generator
|
||||
title: Add weekly monthly ranking snapshot generator
|
||||
status: done
|
||||
type: backend
|
||||
team: Backend Senior
|
||||
supporting_teams:
|
||||
- Arquitecto de Base de Datos
|
||||
- Arquitecto Python
|
||||
roadmap_item: foundation
|
||||
priority: high
|
||||
---
|
||||
|
||||
# TASK-194 - Add weekly monthly ranking snapshot generator
|
||||
|
||||
## Goal
|
||||
|
||||
Implement a manual weekly/monthly ranking snapshot generator plus CLI so PostgreSQL-backed `/api/ranking` can serve persisted global ranking snapshots instead of runtime fallback when operators generate the required combinations.
|
||||
|
||||
## Context
|
||||
|
||||
`TASK-191` introduced the snapshot-first read path for weekly/monthly global ranking, and `TASK-192` / `TASK-193` stabilized the PostgreSQL path and runtime fallback metrics. The missing piece is still out-of-band generation for `ranking_snapshots` and `ranking_snapshot_items`, so production ranking requests continue to return `snapshot_status=missing`, `freshness=runtime` and `fallback_used=true`.
|
||||
|
||||
Preserve the current product identity: Spanish-speaking HLL Vietnam community, military/Vietnam/tactical/sober visual direction and controlled repository evolution.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read the listed files first.
|
||||
2. Add a manual generator for weekly/monthly ranking snapshots using the existing materialized RCON leaderboard logic and snapshot tables.
|
||||
3. Expose a CLI command for one explicit combination; only add matrix generation if it remains small and justified.
|
||||
4. Validate generated snapshot reads, preserved runtime fallback behavior and production-oriented run commands.
|
||||
5. Document the manual generation workflow and recommended combinations.
|
||||
|
||||
## Files to Read First
|
||||
|
||||
- `AGENTS.md`
|
||||
- `ai/repo-context.md`
|
||||
- `ai/architecture-index.md`
|
||||
- `backend/app/rcon_historical_leaderboards.py`
|
||||
- `backend/app/rcon_annual_rankings.py`
|
||||
- `backend/app/routes.py`
|
||||
- `backend/app/payloads.py`
|
||||
- `backend/app/postgres_rcon_storage.py`
|
||||
- `scripts/run-stats-validation.ps1`
|
||||
- `docs/ranking-snapshot-read-model-plan.md`
|
||||
- `docs/ranking-stats-performance-audit.md`
|
||||
- `ai/tasks/done/TASK-191-serve-ranking-from-snapshots-with-runtime-fallback.md`
|
||||
- `ai/tasks/done/TASK-192-fix-postgres-ranking-snapshot-schema.md`
|
||||
- `ai/tasks/done/TASK-193-fix-postgres-ranking-derived-metrics.md`
|
||||
|
||||
## Expected Files to Modify
|
||||
|
||||
- `backend/app/rcon_historical_leaderboards.py`
|
||||
- `scripts/run-stats-validation.ps1`
|
||||
- `docs/ranking-snapshot-read-model-plan.md`
|
||||
- `ai/tasks/done/TASK-194-add-weekly-monthly-ranking-snapshot-generator.md`
|
||||
|
||||
## Constraints
|
||||
|
||||
- Keep the change minimal.
|
||||
- Do not modify frontend, assets or design.
|
||||
- Do not reactivate Elo/MMR.
|
||||
- Do not reintroduce Comunidad Hispana #03.
|
||||
- Keep annual ranking behavior unchanged unless a helper can be safely shared without changing annual contracts.
|
||||
- Prefer the existing module pattern already used by `backend/app/rcon_annual_rankings.py`.
|
||||
- Do not change `/api/ranking` unless strictly necessary for validation-safe compatibility.
|
||||
- Keep PostgreSQL as the operational target and preserve SQLite compatibility already present in the shared backend path.
|
||||
|
||||
## Validation
|
||||
|
||||
Before completing the task ensure:
|
||||
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||
- local import validation proves:
|
||||
- `generate_ranking_snapshot(timeframe="weekly", server_key="all", metric="kills", limit=20)` creates a snapshot
|
||||
- `get_latest_ranking_snapshot(...)` can read that snapshot
|
||||
- `/api/ranking?timeframe=weekly&server_id=all&metric=kills&limit=20` returns `snapshot_status=ready`
|
||||
- the same response returns `fallback_used=false`
|
||||
- requesting a combination without a snapshot still preserves runtime fallback when enabled
|
||||
- document the manual CLI command for local and Docker execution
|
||||
- `git diff --name-only` matches the expected scope
|
||||
|
||||
## Outcome
|
||||
|
||||
Implemented:
|
||||
|
||||
- `backend/app/rcon_historical_leaderboards.py`
|
||||
- `generate_ranking_snapshot(...)`
|
||||
- persistence helpers for `ranking_snapshots` and `ranking_snapshot_items`
|
||||
- manual CLI entrypoint:
|
||||
- `python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20`
|
||||
- `scripts/run-stats-validation.ps1`
|
||||
- now validates a real generated weekly snapshot through:
|
||||
- direct function import
|
||||
- `get_latest_ranking_snapshot(...)`
|
||||
- `/api/ranking`
|
||||
- confirms generated snapshot reads return:
|
||||
- `snapshot_status=ready`
|
||||
- `fallback_used=false`
|
||||
- confirms runtime fallback still works for a missing combination
|
||||
- `docs/ranking-snapshot-read-model-plan.md`
|
||||
- documents the manual command
|
||||
- documents the recommended weekly/monthly/server/metric matrix
|
||||
- documents suggested refresh cadence
|
||||
- documents the expected `ready` vs fallback response states
|
||||
|
||||
Supported generation scope:
|
||||
|
||||
- `timeframe`
|
||||
- `weekly`
|
||||
- `monthly`
|
||||
- `server-key`
|
||||
- `all`
|
||||
- `all-servers`
|
||||
- `comunidad-hispana-01`
|
||||
- `comunidad-hispana-02`
|
||||
- `metric`
|
||||
- `kills`
|
||||
- `deaths`
|
||||
- `teamkills`
|
||||
- `matches_considered`
|
||||
- `kd_ratio`
|
||||
- `kills_per_match`
|
||||
|
||||
Validations executed:
|
||||
|
||||
- `python -m compileall backend/app/rcon_historical_leaderboards.py backend/app/payloads.py backend/app/routes.py backend/app/postgres_rcon_storage.py`
|
||||
- direct local import probe for `generate_ranking_snapshot(...)`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||
- local CLI probe:
|
||||
- `python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20`
|
||||
|
||||
Validation notes:
|
||||
|
||||
- Live backend HTTP at `http://127.0.0.1:8000` was not available, so route validation completed through local imports and repository scripts.
|
||||
- The manual CLI is unitary per `(timeframe, server-key, metric)` combination.
|
||||
- A broader matrix helper was intentionally left out to keep the task small and verifiable.
|
||||
|
||||
Recommended Docker command for production:
|
||||
|
||||
- `docker compose exec backend python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20`
|
||||
|
||||
## Change Budget
|
||||
|
||||
- Prefer fewer than 5 modified files.
|
||||
- Prefer changes under 200 lines when feasible.
|
||||
- Split the work into follow-up tasks if limits are exceeded.
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from contextlib import closing
|
||||
from datetime import datetime, timedelta, timezone
|
||||
@@ -29,6 +31,21 @@ LeaderboardMetric = Literal[
|
||||
"support",
|
||||
]
|
||||
|
||||
SNAPSHOT_GENERATOR_TIMEFRAMES = ("weekly", "monthly")
|
||||
SNAPSHOT_GENERATOR_SERVER_KEYS = (
|
||||
ALL_SERVERS_SLUG,
|
||||
"comunidad-hispana-01",
|
||||
"comunidad-hispana-02",
|
||||
)
|
||||
SNAPSHOT_GENERATOR_METRICS = (
|
||||
"kills",
|
||||
"deaths",
|
||||
"teamkills",
|
||||
"matches_considered",
|
||||
"kd_ratio",
|
||||
"kills_per_match",
|
||||
)
|
||||
|
||||
RANKING_SNAPSHOT_SQLITE_SCHEMA_SQL = """
|
||||
CREATE TABLE IF NOT EXISTS ranking_snapshots (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -102,6 +119,107 @@ def is_ranking_runtime_fallback_enabled() -> bool:
|
||||
return normalized in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def generate_ranking_snapshot(
|
||||
*,
|
||||
timeframe: str,
|
||||
server_key: str | None,
|
||||
metric: str,
|
||||
limit: int,
|
||||
replace_existing: bool = True,
|
||||
now: datetime | None = None,
|
||||
db_path: Path | None = None,
|
||||
) -> dict[str, object]:
|
||||
"""Generate and persist one weekly/monthly ranking snapshot."""
|
||||
normalized_timeframe = _normalize_generator_timeframe(timeframe)
|
||||
normalized_server_key = _normalize_generator_server_key(server_key)
|
||||
normalized_metric = _normalize_generator_metric(metric)
|
||||
normalized_limit = _normalize_limit(limit)
|
||||
anchor = _as_utc(now or datetime.now(timezone.utc))
|
||||
|
||||
resolved_path = initialize_ranking_snapshot_storage(db_path=db_path)
|
||||
connection_scope = _connect_write_scope(resolved_path, db_path=db_path)
|
||||
with connection_scope as connection:
|
||||
window = select_leaderboard_window(
|
||||
connection=connection,
|
||||
server_key=normalized_server_key,
|
||||
timeframe=normalized_timeframe,
|
||||
now=anchor,
|
||||
)
|
||||
window_start = _to_iso(window["start"])
|
||||
window_end = _to_iso(window["end"])
|
||||
existing_snapshot_id = _find_existing_snapshot_for_window(
|
||||
connection=connection,
|
||||
timeframe=normalized_timeframe,
|
||||
server_key=normalized_server_key,
|
||||
metric=normalized_metric,
|
||||
window_start=window_start,
|
||||
window_end=window_end,
|
||||
)
|
||||
if existing_snapshot_id is not None and not replace_existing:
|
||||
snapshot = _get_snapshot_record(connection=connection, snapshot_id=existing_snapshot_id)
|
||||
items = _list_snapshot_items(connection=connection, snapshot_id=existing_snapshot_id)
|
||||
return {
|
||||
"status": "ok",
|
||||
"snapshot": snapshot,
|
||||
"items": items,
|
||||
"source_matches_count": int(snapshot.get("source_matches_count") or 0)
|
||||
if snapshot
|
||||
else 0,
|
||||
"ranked_players": len(items),
|
||||
"skipped_regeneration": True,
|
||||
}
|
||||
|
||||
ranking_rows = _fetch_leaderboard_rows(
|
||||
connection,
|
||||
server_key=normalized_server_key,
|
||||
metric=normalized_metric,
|
||||
limit=normalized_limit,
|
||||
window_start=window["start"],
|
||||
window_end=window["end"],
|
||||
)
|
||||
source_matches_count = _count_matches(
|
||||
connection,
|
||||
server_key=normalized_server_key,
|
||||
start=window["start"],
|
||||
end=window["end"],
|
||||
)
|
||||
|
||||
if existing_snapshot_id is not None:
|
||||
_delete_snapshot(connection=connection, snapshot_id=existing_snapshot_id)
|
||||
|
||||
snapshot_id = _insert_snapshot_record(
|
||||
connection=connection,
|
||||
timeframe=normalized_timeframe,
|
||||
server_key=normalized_server_key,
|
||||
metric=normalized_metric,
|
||||
limit=normalized_limit,
|
||||
source_matches_count=source_matches_count,
|
||||
window_start=window_start,
|
||||
window_end=window_end,
|
||||
generated_at=_to_iso(datetime.now(timezone.utc)),
|
||||
window_kind=str(window["kind"]),
|
||||
window_label=str(window["label"]),
|
||||
item_count=len(ranking_rows),
|
||||
)
|
||||
_insert_snapshot_items(
|
||||
connection=connection,
|
||||
snapshot_id=snapshot_id,
|
||||
rows=ranking_rows,
|
||||
limit=normalized_limit,
|
||||
)
|
||||
snapshot = _get_snapshot_record(connection=connection, snapshot_id=snapshot_id)
|
||||
items = _list_snapshot_items(connection=connection, snapshot_id=snapshot_id)
|
||||
|
||||
return {
|
||||
"status": "ok",
|
||||
"snapshot": snapshot,
|
||||
"items": items,
|
||||
"source_matches_count": source_matches_count,
|
||||
"ranked_players": len(items),
|
||||
"skipped_regeneration": False,
|
||||
}
|
||||
|
||||
|
||||
def get_latest_ranking_snapshot(
|
||||
*,
|
||||
server_key: str | None = None,
|
||||
@@ -383,6 +501,64 @@ def _find_latest_snapshot(
|
||||
return dict(row) if row else None
|
||||
|
||||
|
||||
def _find_existing_snapshot_for_window(
|
||||
*,
|
||||
connection: object,
|
||||
timeframe: str,
|
||||
server_key: str,
|
||||
metric: str,
|
||||
window_start: str,
|
||||
window_end: str,
|
||||
) -> int | None:
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT id
|
||||
FROM ranking_snapshots
|
||||
WHERE timeframe = ?
|
||||
AND server_id = ?
|
||||
AND metric = ?
|
||||
AND window_start = ?
|
||||
AND window_end = ?
|
||||
LIMIT 1
|
||||
""",
|
||||
[timeframe, server_key, metric, window_start, window_end],
|
||||
).fetchone()
|
||||
return int(row["id"]) if row else None
|
||||
|
||||
|
||||
def _get_snapshot_record(
|
||||
*,
|
||||
connection: object,
|
||||
snapshot_id: int,
|
||||
) -> dict[str, object] | None:
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT
|
||||
id,
|
||||
timeframe,
|
||||
server_id,
|
||||
metric,
|
||||
window_start,
|
||||
window_end,
|
||||
generated_at,
|
||||
source,
|
||||
snapshot_status,
|
||||
item_count,
|
||||
limit_size,
|
||||
source_matches_count,
|
||||
freshness,
|
||||
window_kind,
|
||||
window_label,
|
||||
error_message
|
||||
FROM ranking_snapshots
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
""",
|
||||
[snapshot_id],
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
|
||||
def _list_snapshot_items(
|
||||
*,
|
||||
connection: object,
|
||||
@@ -413,6 +589,115 @@ def _list_snapshot_items(
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
|
||||
def _delete_snapshot(*, connection: object, snapshot_id: int) -> None:
|
||||
connection.execute(
|
||||
"DELETE FROM ranking_snapshot_items WHERE snapshot_id = ?",
|
||||
[snapshot_id],
|
||||
)
|
||||
connection.execute(
|
||||
"DELETE FROM ranking_snapshots WHERE id = ?",
|
||||
[snapshot_id],
|
||||
)
|
||||
|
||||
|
||||
def _insert_snapshot_record(
|
||||
*,
|
||||
connection: object,
|
||||
timeframe: str,
|
||||
server_key: str,
|
||||
metric: str,
|
||||
limit: int,
|
||||
source_matches_count: int,
|
||||
window_start: str,
|
||||
window_end: str,
|
||||
generated_at: str,
|
||||
window_kind: str,
|
||||
window_label: str,
|
||||
item_count: int,
|
||||
) -> int:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO ranking_snapshots (
|
||||
timeframe,
|
||||
server_id,
|
||||
metric,
|
||||
window_start,
|
||||
window_end,
|
||||
generated_at,
|
||||
source,
|
||||
snapshot_status,
|
||||
item_count,
|
||||
limit_size,
|
||||
source_matches_count,
|
||||
freshness,
|
||||
window_kind,
|
||||
window_label,
|
||||
error_message
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, 'ready', ?, ?, ?, 'fresh', ?, ?, NULL)
|
||||
RETURNING id
|
||||
""",
|
||||
[
|
||||
timeframe,
|
||||
server_key,
|
||||
metric,
|
||||
window_start,
|
||||
window_end,
|
||||
generated_at,
|
||||
"rcon-materialized-admin-log",
|
||||
item_count,
|
||||
limit,
|
||||
source_matches_count,
|
||||
window_kind,
|
||||
window_label,
|
||||
],
|
||||
)
|
||||
row = cursor.fetchone()
|
||||
if row is None or row["id"] is None:
|
||||
raise RuntimeError("Unable to resolve ranking snapshot id after insert.")
|
||||
return int(row["id"])
|
||||
|
||||
|
||||
def _insert_snapshot_items(
|
||||
*,
|
||||
connection: object,
|
||||
snapshot_id: int,
|
||||
rows: list[dict[str, object]],
|
||||
limit: int,
|
||||
) -> None:
|
||||
for index, row in enumerate(rows[:limit], start=1):
|
||||
item = _build_item(row, index=index)
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO ranking_snapshot_items (
|
||||
snapshot_id,
|
||||
ranking_position,
|
||||
player_id,
|
||||
player_name,
|
||||
metric_value,
|
||||
matches_considered,
|
||||
kills,
|
||||
deaths,
|
||||
teamkills,
|
||||
kd_ratio,
|
||||
kills_per_match
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
snapshot_id,
|
||||
int(item["ranking_position"]),
|
||||
str(item["player_id"]),
|
||||
str(item["player_name"]),
|
||||
float(item["metric_value"]),
|
||||
int(item["matches_considered"]),
|
||||
int(item["kills"]),
|
||||
int(item["deaths"]),
|
||||
int(item["teamkills"]),
|
||||
float(item["kd_ratio"]),
|
||||
float(item["kills_per_match"]),
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def _fetch_leaderboard_rows(
|
||||
connection: object,
|
||||
*,
|
||||
@@ -702,6 +987,14 @@ def _connect_scope(resolved_path: Path, *, db_path: Path | None):
|
||||
return closing(connect_sqlite_readonly(resolved_path))
|
||||
|
||||
|
||||
def _connect_write_scope(resolved_path: Path, *, db_path: Path | None):
|
||||
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
|
||||
from .postgres_rcon_storage import connect_postgres_compat
|
||||
|
||||
return connect_postgres_compat()
|
||||
return connect_sqlite_writer(resolved_path)
|
||||
|
||||
|
||||
def _empty_payload(
|
||||
*,
|
||||
server_key: str | None,
|
||||
@@ -785,6 +1078,40 @@ def _normalize_timeframe(value: str) -> LeaderboardTimeframe:
|
||||
return "monthly" if str(value or "").strip().lower() == "monthly" else "weekly"
|
||||
|
||||
|
||||
def _normalize_generator_timeframe(value: str) -> str:
|
||||
normalized = str(value or "").strip().lower()
|
||||
if normalized not in SNAPSHOT_GENERATOR_TIMEFRAMES:
|
||||
raise ValueError(
|
||||
f"timeframe must be one of: {', '.join(SNAPSHOT_GENERATOR_TIMEFRAMES)}"
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def _normalize_generator_server_key(server_key: str | None) -> str:
|
||||
normalized = _normalize_snapshot_server_key(server_key)
|
||||
if normalized not in SNAPSHOT_GENERATOR_SERVER_KEYS:
|
||||
raise ValueError(
|
||||
"server_key must be one of: all, all-servers, comunidad-hispana-01, comunidad-hispana-02"
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def _normalize_generator_metric(metric: str) -> str:
|
||||
normalized = str(metric or "").strip().lower()
|
||||
if normalized not in SNAPSHOT_GENERATOR_METRICS:
|
||||
raise ValueError(
|
||||
f"metric must be one of: {', '.join(SNAPSHOT_GENERATOR_METRICS)}"
|
||||
)
|
||||
return normalized
|
||||
|
||||
|
||||
def _normalize_limit(limit: object, *, maximum: int = 100) -> int:
|
||||
normalized_limit = int(limit or 1)
|
||||
if normalized_limit < 1:
|
||||
raise ValueError("limit must be greater than zero")
|
||||
return min(normalized_limit, maximum)
|
||||
|
||||
|
||||
def _normalize_metric(value: str) -> LeaderboardMetric:
|
||||
normalized = str(value or "kills").strip().lower()
|
||||
if normalized in {
|
||||
@@ -909,3 +1236,50 @@ def _to_iso(value: object) -> str:
|
||||
if parsed is None:
|
||||
parsed = datetime.now(timezone.utc)
|
||||
return parsed.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Read and generate weekly/monthly ranking snapshots.",
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command")
|
||||
generate_parser = subparsers.add_parser("generate-ranking-snapshot")
|
||||
generate_parser.add_argument(
|
||||
"--timeframe",
|
||||
required=True,
|
||||
choices=SNAPSHOT_GENERATOR_TIMEFRAMES,
|
||||
)
|
||||
generate_parser.add_argument("--server-key", default=None)
|
||||
generate_parser.add_argument(
|
||||
"--metric",
|
||||
required=True,
|
||||
choices=SNAPSHOT_GENERATOR_METRICS,
|
||||
)
|
||||
generate_parser.add_argument("--limit", type=int, default=20)
|
||||
generate_parser.add_argument(
|
||||
"--no-replace-existing",
|
||||
action="store_false",
|
||||
dest="replace_existing",
|
||||
help="keep an existing snapshot for the selected exact window and metric scope",
|
||||
)
|
||||
parser.set_defaults(command="generate-ranking-snapshot", replace_existing=True)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.command == "generate-ranking-snapshot":
|
||||
payload = generate_ranking_snapshot(
|
||||
timeframe=args.timeframe,
|
||||
server_key=args.server_key,
|
||||
metric=args.metric,
|
||||
limit=args.limit,
|
||||
replace_existing=args.replace_existing,
|
||||
db_path=get_storage_path(),
|
||||
)
|
||||
print(json.dumps({"status": "ok", "data": payload}, ensure_ascii=True, indent=2))
|
||||
return 0
|
||||
|
||||
parser.print_help()
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(_main())
|
||||
|
||||
@@ -245,6 +245,72 @@ Operational note after `TASK-191`:
|
||||
- runtime fallback remains enabled by default for transition through `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED=true`
|
||||
- operators can force controlled missing behavior by setting `HLL_BACKEND_RANKING_RUNTIME_FALLBACK_ENABLED=false`
|
||||
|
||||
## Manual Generation Workflow
|
||||
|
||||
Manual generator entrypoint:
|
||||
|
||||
```bash
|
||||
python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20
|
||||
```
|
||||
|
||||
Docker form:
|
||||
|
||||
```bash
|
||||
docker compose exec backend python -m app.rcon_historical_leaderboards generate-ranking-snapshot --timeframe weekly --server-key all --metric kills --limit 20
|
||||
```
|
||||
|
||||
Supported manual parameters:
|
||||
- `timeframe`: `weekly`, `monthly`
|
||||
- `server-key`: `all`, `all-servers`, `comunidad-hispana-01`, `comunidad-hispana-02`
|
||||
- `metric`: `kills`, `deaths`, `teamkills`, `matches_considered`, `kd_ratio`, `kills_per_match`
|
||||
- `limit`: positive integer, normally `20`
|
||||
|
||||
Current implementation note:
|
||||
- V1 generator is unitary per command invocation
|
||||
- operators should run one command per required `(timeframe, server-key, metric)` combination
|
||||
- broad matrix generation can remain a later operational helper if scheduling needs justify it
|
||||
|
||||
## Recommended Combinations
|
||||
|
||||
Minimum production matrix for parity with the public Ranking filters:
|
||||
- weekly + `all-servers` + all six supported metrics
|
||||
- weekly + `comunidad-hispana-01` + all six supported metrics
|
||||
- weekly + `comunidad-hispana-02` + all six supported metrics
|
||||
- monthly + `all-servers` + all six supported metrics
|
||||
- monthly + `comunidad-hispana-01` + all six supported metrics
|
||||
- monthly + `comunidad-hispana-02` + all six supported metrics
|
||||
|
||||
That matrix requires `36` snapshot generations for each refresh cycle.
|
||||
|
||||
## Suggested Frequency
|
||||
|
||||
Suggested operator cadence:
|
||||
- weekly current window: regenerate every `5` to `15` minutes while the active week is changing
|
||||
- monthly current window: regenerate every `15` to `30` minutes while the active month is changing
|
||||
- previous week and previous month windows: regenerate once after closure or after any historical backfill that changes source coverage
|
||||
|
||||
Operational guidance:
|
||||
- regenerate after materialized RCON/AdminLog data grows
|
||||
- regenerate after manual backfill
|
||||
- regenerate after metric SQL changes that affect ranking totals or ordering
|
||||
|
||||
## Ready Vs Fallback
|
||||
|
||||
Expected API result after successful generation for an exact requested combination:
|
||||
- `snapshot_status=ready`
|
||||
- `fallback_used=false`
|
||||
- `source.read_model=ranking-snapshot`
|
||||
|
||||
Expected API result when the requested combination has not been generated yet and fallback remains enabled:
|
||||
- `snapshot_status=missing`
|
||||
- `fallback_used=true`
|
||||
- `freshness=runtime`
|
||||
|
||||
Expected API result when the requested combination has not been generated and fallback is disabled:
|
||||
- `snapshot_status=missing`
|
||||
- `fallback_used=false`
|
||||
- `items=[]`
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- backend implementation of the snapshot generator
|
||||
|
||||
@@ -127,6 +127,8 @@ import app.postgres_rcon_storage as postgres_rcon_storage
|
||||
import app.rcon_historical_leaderboards as ranking_leaderboards
|
||||
|
||||
initialize_ranking_snapshot_storage = ranking_leaderboards.initialize_ranking_snapshot_storage
|
||||
generate_ranking_snapshot = ranking_leaderboards.generate_ranking_snapshot
|
||||
get_latest_ranking_snapshot = ranking_leaderboards.get_latest_ranking_snapshot
|
||||
|
||||
|
||||
def require(condition, message):
|
||||
@@ -301,6 +303,23 @@ def cleanup_snapshot_fixture(db_path):
|
||||
connection.close()
|
||||
|
||||
|
||||
def cleanup_generated_snapshot(snapshot_id):
|
||||
if not snapshot_id:
|
||||
return
|
||||
db_path = Path("backend/data/hll_vietnam_dev.sqlite3")
|
||||
connection = sqlite3.connect(db_path)
|
||||
with connection:
|
||||
connection.execute(
|
||||
"DELETE FROM ranking_snapshot_items WHERE snapshot_id = ?",
|
||||
(snapshot_id,),
|
||||
)
|
||||
connection.execute(
|
||||
"DELETE FROM ranking_snapshots WHERE id = ?",
|
||||
(snapshot_id,),
|
||||
)
|
||||
connection.close()
|
||||
|
||||
|
||||
health_status, health_payload = read_payload("/health")
|
||||
require(health_status == 200, "Route resolver /health should return 200.")
|
||||
require(health_payload.get("status") == "ok", "/health payload should be ok.")
|
||||
@@ -567,6 +586,73 @@ missing_year_ranking_status, _ = read_payload(
|
||||
)
|
||||
require(missing_year_ranking_status == 400, "Global ranking annual requests without year should return 400.")
|
||||
|
||||
generated_snapshot_id = None
|
||||
try:
|
||||
generated_snapshot = generate_ranking_snapshot(
|
||||
timeframe="weekly",
|
||||
server_key="all",
|
||||
metric="kills",
|
||||
limit=20,
|
||||
now=datetime(2026, 5, 20, 12, 0, 0, tzinfo=timezone.utc),
|
||||
db_path=Path("backend/data/hll_vietnam_dev.sqlite3"),
|
||||
)
|
||||
generated_snapshot_record = generated_snapshot.get("snapshot") or {}
|
||||
generated_snapshot_id = generated_snapshot_record.get("id")
|
||||
require(generated_snapshot.get("status") == "ok", "Weekly ranking snapshot generation should return ok.")
|
||||
require(generated_snapshot_id, "Generated weekly ranking snapshot should expose snapshot id.")
|
||||
require(
|
||||
generated_snapshot_record.get("timeframe") == "weekly",
|
||||
"Generated weekly ranking snapshot should preserve timeframe.",
|
||||
)
|
||||
require(
|
||||
generated_snapshot_record.get("metric") == "kills",
|
||||
"Generated weekly ranking snapshot should preserve metric.",
|
||||
)
|
||||
latest_generated_snapshot = get_latest_ranking_snapshot(
|
||||
server_key="all",
|
||||
timeframe="weekly",
|
||||
metric="kills",
|
||||
limit=20,
|
||||
db_path=Path("backend/data/hll_vietnam_dev.sqlite3"),
|
||||
)
|
||||
require(
|
||||
latest_generated_snapshot.get("snapshot_status") == "ready",
|
||||
"Generated weekly ranking snapshot should be readable as ready.",
|
||||
)
|
||||
require(
|
||||
latest_generated_snapshot.get("generated_at"),
|
||||
"Generated weekly ranking snapshot should expose generated_at.",
|
||||
)
|
||||
generated_route_status, generated_route_payload = read_payload(
|
||||
"/api/ranking?timeframe=weekly&server_id=all&metric=kills&limit=20"
|
||||
)
|
||||
require(generated_route_status == 200, "Generated weekly ranking route should return 200.")
|
||||
generated_route_data = generated_route_payload.get("data") or {}
|
||||
require(
|
||||
generated_route_data.get("snapshot_status") == "ready",
|
||||
"Generated weekly ranking route should return snapshot_status=ready.",
|
||||
)
|
||||
require(
|
||||
generated_route_data.get("fallback_used") is False,
|
||||
"Generated weekly ranking route should not use runtime fallback.",
|
||||
)
|
||||
require(
|
||||
isinstance(generated_route_data.get("items"), list),
|
||||
"Generated weekly ranking route should return items list.",
|
||||
)
|
||||
|
||||
generated_fallback_status, generated_fallback_payload = read_payload(
|
||||
"/api/ranking?timeframe=weekly&server_id=all&metric=deaths&limit=20"
|
||||
)
|
||||
require(generated_fallback_status == 200, "Generated validation fallback route should return 200.")
|
||||
generated_fallback_data = generated_fallback_payload.get("data") or {}
|
||||
require(
|
||||
generated_fallback_data.get("fallback_used") is True,
|
||||
"Missing weekly deaths snapshot should still use runtime fallback when enabled.",
|
||||
)
|
||||
finally:
|
||||
cleanup_generated_snapshot(generated_snapshot_id)
|
||||
|
||||
fixture_db_path = build_snapshot_fixture()
|
||||
try:
|
||||
fixture_weekly_status, fixture_weekly_payload = read_payload(
|
||||
@@ -610,9 +696,10 @@ print(json.dumps({
|
||||
"global-ranking",
|
||||
"postgres-ranking-derived-metric-sql",
|
||||
"postgres-ranking-schema-path",
|
||||
"ranking-snapshot-generator",
|
||||
"ranking-snapshot-ready",
|
||||
"ranking-snapshot-missing",
|
||||
],
|
||||
],
|
||||
"annual_snapshot_status": annual_data.get("snapshot_status"),
|
||||
"global_ranking_annual_snapshot_status": annual_ranking_data.get("snapshot_status"),
|
||||
"search_items_count": len(search_data.get("items") or []),
|
||||
|
||||
Reference in New Issue
Block a user