rgonsal/comunidadhll
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add player period stats read model

Browse Source
This commit is contained in:
devRaGonSa
2026-06-09 10:49:00 +02:00
parent 73a6c1afe1
commit 5a52889cfb
5 changed files with 1164 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

172
ai/tasks/done/TASK-199-add-player-period-stats-read-model.md Normal file
View File

@@ -0,0 +1,172 @@
---
id: TASK-199-add-player-period-stats-read-model
title: Add player period stats read model
status: done
type: backend
team: Backend Senior
supporting_teams:
- Arquitecto de Base de Datos
- Arquitecto Python
roadmap_item: foundation
priority: high
---
# TASK-199 - Add player period stats read model
## Goal
Add a dedicated PostgreSQL read model for player personal stats by period so `/api/stats/players/{player_id}` can avoid aggregating large RCON historical tables on every public request when a regenerated read model is available.
## Context
The repository already has PostgreSQL-backed ranking snapshots and a player search read model with controlled runtime fallback. The next expensive public path is the player profile/stats flow, which still aggregates directly from `rcon_materialized_matches` and `rcon_match_player_stats` at request time.
This task adds a regenerable `player_period_stats` read model for weekly, monthly and yearly player totals across the supported public scopes. The profile read path must prefer the read model when it exists and contains rows for the requested player, scope and period, while preserving the current runtime fallback path if the table is missing, empty, incomplete or temporarily unreadable.
Preserve the current product identity: Spanish-speaking HLL Vietnam community, military/Vietnam/tactical/sober visual direction and controlled repository evolution.
## Steps
1. Read the listed files first.
2. Add a dedicated PostgreSQL read-model table for player personal stats by period.
3. Implement a refresh function that rebuilds the table from materialized RCON PostgreSQL tables for the supported scopes and windows.
4. Add a manual CLI command to rebuild the read model on demand.
5. Make `/api/stats/players/{player_id}` prefer the read model and keep controlled runtime fallback when needed.
6. Preserve the current frontend contract consumed by `frontend/assets/js/stats.js`.
7. Document the operational command, fallback behavior and current limitations.
## Files to Read First
- `AGENTS.md`
- `ai/repo-context.md`
- `ai/architecture-index.md`
- `backend/app/routes.py`
- `backend/app/payloads.py`
- `backend/app/rcon_historical_player_stats.py`
- `backend/app/postgres_rcon_storage.py`
- `backend/app/rcon_admin_log_materialization.py`
- `docs/stats-section-functional-plan.md`
- `docs/player-search-read-model-plan.md`
- `ai/tasks/done/TASK-198-add-player-search-read-model.md`
## Expected Files to Modify
- `backend/app/rcon_historical_player_stats.py`
- `backend/app/postgres_rcon_storage.py`
- `scripts/run-stats-validation.ps1`
- `docs/player-period-stats-read-model-plan.md`
- `ai/tasks/done/TASK-199-add-player-period-stats-read-model.md`
## Constraints
- Keep the change minimal and backend-only.
- Do not execute `ai-platform run`.
- Do not modify frontend.
- Do not change design.
- Do not touch images or assets.
- Do not reactivate Elo/MMR.
- Do not reintroduce Comunidad Hispana `#03`.
- Do not mix visual corrections into this task.
- PostgreSQL is the operational storage target.
- SQLite may remain only as explicit local compatibility where the current backend architecture already supports it.
- The new table is a regenerable read model, not a canonical source of truth.
- Do not require PostgreSQL extensions that are not already installed.
- Keep the current frontend contract for player profile data compatible with `frontend/assets/js/stats.js`.
- Minimum supported period types are `weekly`, `monthly` and `yearly`.
- Minimum supported public scopes are `all-servers`, `comunidad-hispana-01` and `comunidad-hispana-02`.
## Validation
Before completing the task ensure:
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
- local validation proves the `player_period_stats` table can be initialized
- local validation proves `refresh_player_period_stats(...)` generates rows from materialized test data
- local validation proves player profile/stats uses `player_period_stats` when data exists
- local validation proves profile/stats falls back when the read model is empty, missing or unavailable
- the public player profile payload remains compatible with `frontend/assets/js/stats.js`
- player search refresh and read path remain functional
- `git diff --name-only` matches the expected scope
## Outcome
Implemented:
- `backend/app/rcon_historical_player_stats.py`
- added `player_period_stats` initialization for SQLite compatibility and PostgreSQL-first operational mode
- added `refresh_player_period_stats(...)`
- added snapshot-first player profile reads over `player_period_stats`
- preserved runtime fallback when the read model is empty, incomplete or unavailable
- added manual CLI:
- `python -m app.rcon_historical_player_stats refresh-player-period-stats`
- `backend/app/postgres_rcon_storage.py`
- added PostgreSQL schema for `player_period_stats`
- added indexes for:
- `(player_id, period_type, server_id)`
- `(server_id, period_type)`
- `last_seen_at`
- `updated_at`
- `scripts/run-stats-validation.ps1`
- validates PostgreSQL schema wiring
- validates CLI defaults
- validates fixture-driven refresh generation
- validates read-model-first profile reads
- validates fallback when the read model is empty, incomplete or unavailable
- `docs/player-period-stats-read-model-plan.md`
- documents objective, fields, windows, CLI, PostgreSQL role, fallback and current limitations
Created table:
- `player_period_stats`
Table behavior:
- regenerable read model only
- one row per `(period_type, server_id, player_id)` public scope
- supported periods:
- `weekly`
- `monthly`
- `yearly`
- supported scopes:
- `all-servers`
- `comunidad-hispana-01`
- `comunidad-hispana-02`
- stores ranking position by kills for the selected window
- preserves latest player name inside each generated window
CLI added:
- `python -m app.rcon_historical_player_stats refresh-player-period-stats`
Production validation path:
- refresh manually with the CLI above
- confirm row counts are reported for the supported scopes and periods
- call `/api/stats/players/<known-player>?timeframe=weekly`
- verify `data.source.read_model=player-period-stats`
- verify `data.source.fallback_used=false` when the read model is populated
- verify fallback metadata appears only when the read model is empty, incomplete or unavailable
Validations executed:
- `python -m py_compile backend/app/rcon_historical_player_stats.py backend/app/postgres_rcon_storage.py`
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
Validation notes:
- live backend HTTP at `http://127.0.0.1:8000` was not available in this environment
- route-contract validation still passed through local Python imports
- no frontend, design, asset, Elo/MMR or Comunidad Hispana `#03` changes were made
Pending follow-up kept out of scope:
- automate periodic refresh scheduling for `player_period_stats`
- decide whether production refresh should be chained with ranking snapshot/search index refresh jobs
## Change Budget
- Prefer fewer than 5 modified files.
- Prefer changes under 200 lines when feasible.
- Split the work into follow-up tasks if limits are exceeded.

30
backend/app/postgres_rcon_storage.py
View File

@@ -275,6 +275,28 @@ CREATE TABLE IF NOT EXISTS player_search_index (
UNIQUE(server_id, player_id)
);
CREATE TABLE IF NOT EXISTS player_period_stats (
id BIGSERIAL PRIMARY KEY,
period_type TEXT NOT NULL,
window_kind TEXT NOT NULL,
period_start TEXT NOT NULL,
period_end TEXT NOT NULL,
server_id TEXT NOT NULL,
player_id TEXT NOT NULL,
player_name TEXT NOT NULL,
matches_considered INTEGER NOT NULL DEFAULT 0,
kills INTEGER NOT NULL DEFAULT 0,
deaths INTEGER NOT NULL DEFAULT 0,
teamkills INTEGER NOT NULL DEFAULT 0,
ranking_position INTEGER,
kd_ratio DOUBLE PRECISION NOT NULL DEFAULT 0.0,
kills_per_match DOUBLE PRECISION NOT NULL DEFAULT 0.0,
first_seen_at TEXT,
last_seen_at TEXT,
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(period_type, server_id, player_id)
);
CREATE TABLE IF NOT EXISTS rcon_scoreboard_match_candidates (
id BIGSERIAL PRIMARY KEY,
server_slug TEXT NOT NULL,
@@ -345,6 +367,14 @@ CREATE INDEX IF NOT EXISTS idx_player_search_index_last_seen
ON player_search_index(server_id, last_seen_at DESC);
CREATE INDEX IF NOT EXISTS idx_player_search_index_player
ON player_search_index(server_id, player_id);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_player_period_server
ON player_period_stats(player_id, period_type, server_id);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_server_period
ON player_period_stats(server_id, period_type);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_last_seen
ON player_period_stats(last_seen_at DESC);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_updated
ON player_period_stats(updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_rcon_scoreboard_candidates_server_end
ON rcon_scoreboard_match_candidates(server_slug, ended_at DESC, started_at DESC);
"""

447
backend/app/rcon_historical_player_stats.py
View File

@@ -21,6 +21,7 @@ PLAYER_SEARCH_INDEX_SERVER_KEYS = (
"comunidad-hispana-01",
"comunidad-hispana-02",
)
PLAYER_PERIOD_STATS_PERIOD_TYPES = ("weekly", "monthly", "yearly")
PLAYER_SEARCH_INDEX_SQLITE_SCHEMA_SQL = """
CREATE TABLE IF NOT EXISTS player_search_index (
@@ -50,6 +51,42 @@ CREATE INDEX IF NOT EXISTS idx_player_search_index_player
ON player_search_index(server_id, player_id);
"""
PLAYER_PERIOD_STATS_SQLITE_SCHEMA_SQL = """
CREATE TABLE IF NOT EXISTS player_period_stats (
id INTEGER PRIMARY KEY AUTOINCREMENT,
period_type TEXT NOT NULL,
window_kind TEXT NOT NULL,
period_start TEXT NOT NULL,
period_end TEXT NOT NULL,
server_id TEXT NOT NULL,
player_id TEXT NOT NULL,
player_name TEXT NOT NULL,
matches_considered INTEGER NOT NULL DEFAULT 0,
kills INTEGER NOT NULL DEFAULT 0,
deaths INTEGER NOT NULL DEFAULT 0,
teamkills INTEGER NOT NULL DEFAULT 0,
ranking_position INTEGER,
kd_ratio REAL NOT NULL DEFAULT 0.0,
kills_per_match REAL NOT NULL DEFAULT 0.0,
first_seen_at TEXT,
last_seen_at TEXT,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE(period_type, server_id, player_id)
);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_player_period_server
ON player_period_stats(player_id, period_type, server_id);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_server_period
ON player_period_stats(server_id, period_type);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_last_seen
ON player_period_stats(last_seen_at DESC);
CREATE INDEX IF NOT EXISTS idx_player_period_stats_updated
ON player_period_stats(updated_at DESC);
"""
def initialize_player_search_index_storage(*, db_path: Path | None = None) -> Path:
"""Create the player search read model storage if it does not exist."""
@@ -66,6 +103,21 @@ def initialize_player_search_index_storage(*, db_path: Path | None = None) -> Pa
return resolved_path
def initialize_player_period_stats_storage(*, db_path: Path | None = None) -> Path:
"""Create the player period stats read model storage if it does not exist."""
resolved_path = initialize_rcon_materialized_storage(db_path=db_path)
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
from .postgres_rcon_storage import initialize_postgres_rcon_storage
initialize_postgres_rcon_storage()
return resolved_path
with closing(connect_sqlite_writer(resolved_path)) as connection:
with connection:
connection.executescript(PLAYER_PERIOD_STATS_SQLITE_SCHEMA_SQL)
return resolved_path
def refresh_player_search_index(
*,
db_path: Path | None = None,
@@ -112,6 +164,62 @@ def refresh_player_search_index(
}
def refresh_player_period_stats(
*,
db_path: Path | None = None,
now: datetime | None = None,
) -> dict[str, object]:
"""Rebuild the personal player stats read model for supported periods and scopes."""
anchor = _as_utc(now or datetime.now(timezone.utc))
resolved_path = initialize_player_period_stats_storage(db_path=db_path)
results: list[dict[str, object]] = []
total_rows = 0
with _connect_write_scope(resolved_path, db_path=db_path) as connection:
for server_id in PLAYER_SEARCH_INDEX_SERVER_KEYS:
for period_type in PLAYER_PERIOD_STATS_PERIOD_TYPES:
window = _select_player_period_window(
connection=connection,
server_id=server_id,
period_type=period_type,
now=anchor,
)
rows = _build_player_period_stats_rows(
connection=connection,
server_id=server_id,
period_type=period_type,
window=window,
)
_replace_player_period_stats_scope(
connection=connection,
server_id=server_id,
period_type=period_type,
rows=rows,
updated_at=_to_iso(anchor),
)
results.append(
{
"server_id": server_id,
"period_type": period_type,
"window_kind": window["kind"],
"period_start": _to_iso(window["start"]),
"period_end": _to_iso(window["end"]),
"row_count": len(rows),
}
)
total_rows += len(rows)
return {
"status": "ok",
"generated_at": _to_iso(anchor),
"source": "rcon-materialized-admin-log",
"server_ids": list(PLAYER_SEARCH_INDEX_SERVER_KEYS),
"period_types": list(PLAYER_PERIOD_STATS_PERIOD_TYPES),
"total_rows": total_rows,
"results": results,
}
def search_rcon_materialized_players(
*,
query: str,
@@ -252,6 +360,52 @@ def get_rcon_materialized_player_stats(
if resolved_timeframe not in {"weekly", "monthly", "all"}:
raise ValueError("Invalid timeframe parameter")
if resolved_timeframe != "all":
read_model_result, fallback_reason = _get_player_period_stats_read_model(
player_id=normalized_player_id,
server_id=server_id,
timeframe=resolved_timeframe,
db_path=db_path,
)
if read_model_result is not None:
return read_model_result
runtime_result = _get_rcon_materialized_player_stats_runtime(
player_id=normalized_player_id,
server_id=server_id,
timeframe=resolved_timeframe,
db_path=db_path,
)
runtime_source = dict(runtime_result.get("source") or {})
runtime_source.update(
{
"read_model": "player-period-stats",
"fallback_used": True,
"fallback_reason": fallback_reason or "player-period-stats-unavailable",
}
)
runtime_result["source"] = runtime_source
return runtime_result
return _get_rcon_materialized_player_stats_runtime(
player_id=normalized_player_id,
server_id=server_id,
timeframe=resolved_timeframe,
db_path=db_path,
)
def _get_rcon_materialized_player_stats_runtime(
*,
player_id: str,
server_id: str | None = None,
timeframe: str = "weekly",
db_path: Path | None = None,
) -> dict[str, object]:
"""Return runtime player stats directly from materialized matches and player rows."""
normalized_player_id = player_id.strip()
resolved_timeframe = timeframe.strip().lower() if isinstance(timeframe, str) else "weekly"
resolved_path = initialize_rcon_materialized_storage(db_path=db_path)
resolved_server_id = _normalize_server_id(server_id)
now = datetime.now(timezone.utc)
@@ -319,7 +473,6 @@ def get_rcon_materialized_player_stats(
"monthly_ranking": monthly_ranking,
"source": {
"primary_source": "rcon",
"read_model": "rcon-materialized-admin-log-player-stats",
"generated_at": _to_iso(now),
"source_range_start": _to_iso(source_range[0]) if source_range[0] else None,
"source_range_end": _to_iso(source_range[1]) if source_range[1] else None,
@@ -328,6 +481,280 @@ def get_rcon_materialized_player_stats(
}
def _get_player_period_stats_read_model(
*,
player_id: str,
server_id: str | None,
timeframe: str,
db_path: Path | None = None,
) -> tuple[dict[str, object] | None, str | None]:
resolved_path = initialize_player_period_stats_storage(db_path=db_path)
resolved_server_id = _normalize_server_id(server_id)
required_periods = sorted({timeframe, "weekly", "monthly"})
placeholders = ", ".join(["?"] * len(required_periods))
try:
with _connect_scope(resolved_path, db_path=db_path) as connection:
scope_rows = connection.execute(
f"""
SELECT period_type, COUNT(*) AS row_count
FROM player_period_stats
WHERE server_id = ?
AND period_type IN ({placeholders})
GROUP BY period_type
""",
[resolved_server_id, *required_periods],
).fetchall()
counts_by_period = {
str(row["period_type"] or "").strip(): int(row["row_count"] or 0)
for row in scope_rows
}
if not counts_by_period:
return None, "player-period-stats-empty"
if any(counts_by_period.get(period_type, 0) <= 0 for period_type in required_periods):
return None, "player-period-stats-empty"
rows = connection.execute(
f"""
SELECT *
FROM player_period_stats
WHERE server_id = ?
AND player_id = ?
AND period_type IN ({placeholders})
""",
[resolved_server_id, player_id, *required_periods],
).fetchall()
except Exception:
return None, "player-period-stats-unavailable"
rows_by_period = {
str(row["period_type"] or "").strip(): dict(row)
for row in rows
}
if any(period_type not in rows_by_period for period_type in required_periods):
return None, "player-period-stats-player-missing"
selected_row = rows_by_period[timeframe]
weekly_row = rows_by_period["weekly"]
monthly_row = rows_by_period["monthly"]
return (
{
"player_id": player_id,
"server_id": resolved_server_id,
"timeframe": timeframe,
"player_name": selected_row.get("player_name"),
"window_start": _to_datetime_or_none(selected_row.get("period_start")),
"window_end": _to_datetime_or_none(selected_row.get("period_end")),
"window_kind": selected_row.get("window_kind"),
"matches_considered": int(selected_row.get("matches_considered") or 0),
"kills": int(selected_row.get("kills") or 0),
"deaths": int(selected_row.get("deaths") or 0),
"teamkills": int(selected_row.get("teamkills") or 0),
"weekly_ranking": _build_player_period_ranking_payload(weekly_row),
"monthly_ranking": _build_player_period_ranking_payload(monthly_row),
"source": {
"primary_source": "rcon",
"read_model": "player-period-stats",
"fallback_used": False,
"fallback_reason": None,
"generated_at": selected_row.get("updated_at"),
"source_range_start": selected_row.get("first_seen_at"),
"source_range_end": selected_row.get("last_seen_at"),
"freshness": "read-model",
},
},
None,
)
def _build_player_period_ranking_payload(row: dict[str, object]) -> dict[str, object]:
ranking_position = row.get("ranking_position")
return {
"metric": "kills",
"ranking_position": int(ranking_position) if ranking_position is not None else None,
"window_kind": row.get("window_kind"),
"window_start": row.get("period_start"),
"window_end": row.get("period_end"),
}
def _select_player_period_window(
*,
connection: object,
server_id: str,
period_type: str,
now: datetime,
) -> dict[str, object]:
if period_type == "yearly":
start = now.replace(month=1, day=1, hour=0, minute=0, second=0, microsecond=0)
return {
"start": start,
"end": now,
"kind": "current-year",
"label": "Ano actual",
}
return select_leaderboard_window(
connection=connection,
server_key=server_id,
timeframe=period_type,
now=now,
)
def _build_player_period_stats_rows(
*,
connection: object,
server_id: str,
period_type: str,
window: dict[str, object],
) -> list[dict[str, object]]:
scope_sql, scope_params = _build_scope_sql(server_id)
rows = connection.execute(
f"""
SELECT
stats.player_id,
MIN(COALESCE(CAST(matches.ended_at AS TEXT), CAST(matches.started_at AS TEXT))) AS first_seen_at,
MAX(COALESCE(CAST(matches.ended_at AS TEXT), CAST(matches.started_at AS TEXT))) AS last_seen_at,
COUNT(DISTINCT stats.match_key) AS matches_considered,
SUM(COALESCE(stats.kills, 0)) AS kills,
SUM(COALESCE(stats.deaths, 0)) AS deaths,
SUM(COALESCE(stats.teamkills, 0)) AS teamkills
FROM rcon_match_player_stats AS stats
INNER JOIN rcon_materialized_matches AS matches
ON matches.target_key = stats.target_key
AND matches.match_key = stats.match_key
WHERE matches.source_basis = ?
AND COALESCE(CAST(matches.ended_at AS TEXT), CAST(matches.started_at AS TEXT)) >= ?
AND COALESCE(CAST(matches.ended_at AS TEXT), CAST(matches.started_at AS TEXT)) <= ?
{scope_sql}
GROUP BY stats.player_id
HAVING COUNT(DISTINCT stats.match_key) > 0
""",
[
MATCH_RESULT_SOURCE,
_to_iso(window["start"]),
_to_iso(window["end"]),
*scope_params,
],
).fetchall()
if not rows:
return []
player_ids = [
str(row["player_id"]).strip()
for row in rows
if str(row["player_id"] or "").strip()
]
latest_names = _lookup_latest_player_names_with_connection(
connection=connection,
player_ids=player_ids,
server_id=server_id,
start=window["start"],
end=window["end"],
)
period_rows: list[dict[str, object]] = []
for row in rows:
player_id = str(row["player_id"] or "").strip()
if not player_id:
continue
matches_considered = int(row["matches_considered"] or 0)
kills = int(row["kills"] or 0)
deaths = int(row["deaths"] or 0)
teamkills = int(row["teamkills"] or 0)
player_name = latest_names.get(player_id) or player_id
period_rows.append(
{
"period_type": period_type,
"window_kind": str(window["kind"]),
"period_start": _to_iso(window["start"]),
"period_end": _to_iso(window["end"]),
"server_id": server_id,
"player_id": player_id,
"player_name": player_name,
"matches_considered": matches_considered,
"kills": kills,
"deaths": deaths,
"teamkills": teamkills,
"kd_ratio": round(kills / deaths, 2) if deaths else float(kills),
"kills_per_match": round(kills / matches_considered, 2) if matches_considered else 0.0,
"first_seen_at": row["first_seen_at"],
"last_seen_at": row["last_seen_at"],
}
)
period_rows.sort(
key=lambda item: (
-int(item["kills"]),
-int(item["matches_considered"]),
str(item["player_name"]).casefold(),
)
)
for position, row in enumerate(period_rows, start=1):
row["ranking_position"] = position
return period_rows
def _replace_player_period_stats_scope(
*,
connection: object,
server_id: str,
period_type: str,
rows: list[dict[str, object]],
updated_at: str,
) -> None:
connection.execute(
"""
DELETE FROM player_period_stats
WHERE server_id = ? AND period_type = ?
""",
[server_id, period_type],
)
for row in rows:
connection.execute(
"""
INSERT INTO player_period_stats (
period_type,
window_kind,
period_start,
period_end,
server_id,
player_id,
player_name,
matches_considered,
kills,
deaths,
teamkills,
ranking_position,
kd_ratio,
kills_per_match,
first_seen_at,
last_seen_at,
updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
row["period_type"],
row["window_kind"],
row["period_start"],
row["period_end"],
row["server_id"],
row["player_id"],
row["player_name"],
row["matches_considered"],
row["kills"],
row["deaths"],
row["teamkills"],
row["ranking_position"],
row["kd_ratio"],
row["kills_per_match"],
row["first_seen_at"],
row["last_seen_at"],
updated_at,
],
)
def _build_player_timeframe_window(
*,
connection: object,
@@ -1031,6 +1458,13 @@ def _main(argv: list[str] | None = None) -> int:
default=None,
help="explicit local SQLite override; default operational mode uses PostgreSQL when configured",
)
refresh_period_stats_parser = subparsers.add_parser("refresh-player-period-stats")
refresh_period_stats_parser.add_argument(
"--sqlite-path",
type=Path,
default=None,
help="explicit local SQLite override; default operational mode uses PostgreSQL when configured",
)
args = parser.parse_args(argv)
if args.command == "refresh-player-search-index":
@@ -1044,6 +1478,17 @@ def _main(argv: list[str] | None = None) -> int:
)
)
return 0
if args.command == "refresh-player-period-stats":
payload = refresh_player_period_stats(db_path=args.sqlite_path)
print(
json.dumps(
{"status": "ok", "data": payload},
ensure_ascii=True,
indent=2,
default=_json_default,
)
)
return 0
parser.print_help()
return 2

168
docs/player-period-stats-read-model-plan.md Normal file
View File

@@ -0,0 +1,168 @@
# Player Period Stats Read Model
## Objective
Define the PostgreSQL read model that backs personal player stats by period so `/api/stats/players/{player_id}` does not aggregate large RCON historical tables on every public request when a regenerated read model is available.
## Table
Primary table:
- `player_period_stats`
Operational scope:
- PostgreSQL is the default operational store
- SQLite remains available only as explicit local compatibility for validation and isolated maintenance runs
Read-model nature:
- this table is regenerable
- it is not the canonical source of truth
- canonical historical data remains in `rcon_materialized_matches` and `rcon_match_player_stats`
## Stored Fields
Each row stores one player projection for one public scope and one active period window:
- `period_type`
- `window_kind`
- `period_start`
- `period_end`
- `server_id`
- `player_id`
- `player_name`
- `matches_considered`
- `kills`
- `deaths`
- `teamkills`
- `ranking_position`
- `kd_ratio`
- `kills_per_match`
- `first_seen_at`
- `last_seen_at`
- `updated_at`
Current period types:
- `weekly`
- `monthly`
- `yearly`
Current scope rows:
- `all-servers`
- `comunidad-hispana-01`
- `comunidad-hispana-02`
Why `server_id` exists:
- the public profile endpoint already supports `server_id`
- keeping one row per scope preserves the current frontend contract without recalculating large runtime aggregates for server-filtered profiles
Why `window_kind` exists:
- the profile payload already exposes weekly and monthly ranking context
- persisting `current-week`, `previous-week`, `current-month` or `previous-month` keeps the public contract stable without recomputing that selection on each request
## Period Windows
Window selection follows the same policy already used by ranking/stats reads where applicable:
- `weekly`
- uses `select_leaderboard_window(... timeframe="weekly")`
- current week is used only when the closed-match threshold is sufficient
- otherwise the read model stores the previous week window
- `monthly`
- uses `select_leaderboard_window(... timeframe="monthly")`
- from day 1 to day 7 it stores the previous month
- from day 8 onward it stores the current month
- `yearly`
- stores the current UTC year from January 1 until refresh time
- it is prepared internally even if the current public profile contract still requests weekly/monthly only
## Refresh Command
Manual command:
```bash
python -m app.rcon_historical_player_stats refresh-player-period-stats
```
SQLite-only local override:
```bash
python -m app.rcon_historical_player_stats refresh-player-period-stats --sqlite-path backend/data/hll_vietnam_dev.sqlite3
```
Refresh policy:
- rebuild from materialized RCON/AdminLog tables
- replace rows scope by scope and period by period
- keep the latest player name seen inside the selected period window
- persist ranking position by kills inside each generated period window
## Public Read Path
Priority for `/api/stats/players/{player_id}`:
1. use `player_period_stats` when the requested scope has generated rows for the required periods
2. serve the requested weekly/monthly totals and ranking context directly from the read model
3. fall back to runtime aggregation only when:
- the read model table is unavailable
- the requested scope has no generated rows yet
- the player has no generated row for one required period
- a controlled read error occurs
Returned compatibility:
- the payload still returns `player_id`
- the payload still returns `player_name`
- the payload still returns `matches_considered`
- the payload still returns `kills`
- the payload still returns `deaths`
- the payload still returns `teamkills`
- the payload still returns `kd_ratio`
- the payload still returns `kills_per_match`
- the payload still returns weekly/monthly ranking blocks
Source metadata:
- read-model path reports `source.read_model = "player-period-stats"`
- read-model path reports `source.fallback_used = false`
- runtime fallback keeps the same public contract and reports `source.fallback_used = true`
## PostgreSQL Notes
No extra PostgreSQL extensions are required.
Indexes kept for the public profile flow:
- `(player_id, period_type, server_id)`
- `(server_id, period_type)`
- `last_seen_at`
- `updated_at`
## Production Validation
Recommended checks after refresh:
- run `python -m app.rcon_historical_player_stats refresh-player-period-stats`
- confirm the command reports rows for:
- `all-servers`
- `comunidad-hispana-01`
- `comunidad-hispana-02`
- `weekly`
- `monthly`
- `yearly`
- call `/api/stats/players/<known-player>?timeframe=weekly`
- verify response metadata reports `read_model=player-period-stats`
- verify response metadata reports `fallback_used=false` when the read model is populated
- verify fallback metadata appears only when the read model is empty, incomplete or unavailable
## Current Limitations
- periodic refresh is still manual; there is no scheduled operational refresh yet
- the public route still exposes weekly/monthly only; yearly is prepared internally for future use
- runtime fallback remains necessary until production refresh automation is in place
- canonical historical truth remains in materialized RCON tables, not in the read model

359
scripts/run-stats-validation.ps1
View File

@@ -136,7 +136,10 @@ get_latest_ranking_snapshot = ranking_leaderboards.get_latest_ranking_snapshot
refresh_ranking_snapshots = ranking_leaderboards.refresh_ranking_snapshots
initialize_player_search_index_storage = player_search_stats.initialize_player_search_index_storage
refresh_player_search_index = player_search_stats.refresh_player_search_index
initialize_player_period_stats_storage = player_search_stats.initialize_player_period_stats_storage
refresh_player_period_stats = player_search_stats.refresh_player_period_stats
search_rcon_materialized_players = player_search_stats.search_rcon_materialized_players
get_rcon_materialized_player_stats = player_search_stats.get_rcon_materialized_player_stats
MATCH_RESULT_SOURCE = player_search_stats.MATCH_RESULT_SOURCE
@@ -821,6 +824,333 @@ def validate_player_search_index_fallbacks():
pass
def validate_postgres_player_period_stats_schema_path():
require(
"CREATE TABLE IF NOT EXISTS player_period_stats" in postgres_rcon_storage.RCON_SCHEMA_SQL,
"PostgreSQL schema should define player_period_stats.",
)
require(
"CREATE INDEX IF NOT EXISTS idx_player_period_stats_player_period_server" in postgres_rcon_storage.RCON_SCHEMA_SQL,
"PostgreSQL schema should define the player_period_stats player/period/server index.",
)
original_initialize_materialized = player_search_stats.initialize_rcon_materialized_storage
original_use_postgres_storage = player_search_stats.use_postgres_rcon_storage
original_initialize_postgres_storage = postgres_rcon_storage.initialize_postgres_rcon_storage
calls = {"postgres_init": 0}
player_search_stats.initialize_rcon_materialized_storage = (
lambda db_path=None: Path("backend/data/hll_vietnam_dev.sqlite3")
)
player_search_stats.use_postgres_rcon_storage = lambda explicit_sqlite_path=None: True
def fake_initialize_postgres_storage():
calls["postgres_init"] += 1
postgres_rcon_storage.initialize_postgres_rcon_storage = fake_initialize_postgres_storage
try:
initialize_player_period_stats_storage()
finally:
player_search_stats.initialize_rcon_materialized_storage = original_initialize_materialized
player_search_stats.use_postgres_rcon_storage = original_use_postgres_storage
postgres_rcon_storage.initialize_postgres_rcon_storage = original_initialize_postgres_storage
require(
calls["postgres_init"] == 1,
"Player period stats initialization should delegate to initialize_postgres_rcon_storage exactly once.",
)
def validate_player_period_stats_cli_defaults():
original_refresh_player_period_stats = player_search_stats.refresh_player_period_stats
captured = {}
def fake_refresh_player_period_stats(**kwargs):
captured.update(kwargs)
return {
"status": "ok",
"generated_at": datetime(2026, 6, 10, 12, 0, 0, tzinfo=timezone.utc),
"source": "rcon-materialized-admin-log",
"server_ids": ["all-servers", "comunidad-hispana-01", "comunidad-hispana-02"],
"period_types": ["weekly", "monthly", "yearly"],
"total_rows": 9,
"results": [{"server_id": "all-servers", "period_type": "weekly", "row_count": 2}],
}
player_search_stats.refresh_player_period_stats = fake_refresh_player_period_stats
try:
stdout_buffer = StringIO()
with redirect_stdout(stdout_buffer):
exit_code = player_search_stats._main(["refresh-player-period-stats"])
require(exit_code == 0, "Player period stats CLI should exit 0 for a valid command.")
require(
captured.get("db_path") is None,
"Player period stats CLI should use PostgreSQL-compatible db_path=None by default.",
)
serialized_default_payload = json.loads(stdout_buffer.getvalue())
require(
serialized_default_payload.get("data", {}).get("generated_at") == "2026-06-10T12:00:00Z",
"Player period stats CLI should serialize datetime values as ISO strings.",
)
captured.clear()
stdout_buffer = StringIO()
with redirect_stdout(stdout_buffer):
exit_code = player_search_stats._main([
"refresh-player-period-stats",
"--sqlite-path", "backend/data/hll_vietnam_dev.sqlite3",
])
require(exit_code == 0, "Player period stats CLI with --sqlite-path should exit 0.")
require(
captured.get("db_path") == Path("backend/data/hll_vietnam_dev.sqlite3"),
"Player period stats CLI should pass the explicit --sqlite-path override through to refresh_player_period_stats.",
)
finally:
player_search_stats.refresh_player_period_stats = original_refresh_player_period_stats
def build_player_period_stats_fixture():
stamp = datetime.now(timezone.utc).strftime("%Y%m%d%H%M%S%f")
db_path = Path(f"backend/data/hll_vietnam_player_period_stats_validation_{stamp}.sqlite3")
if db_path.exists():
try:
db_path.unlink()
except PermissionError:
pass
initialize_player_period_stats_storage(db_path=db_path)
connection = sqlite3.connect(db_path)
connection.row_factory = sqlite3.Row
with connection:
connection.execute(
"""
INSERT INTO rcon_materialized_matches (
target_key, external_server_id, match_key, map_name, map_pretty_name, game_mode,
started_server_time, ended_server_time, started_at, ended_at,
allied_score, axis_score, winner, confidence_mode, source_basis
) VALUES
('comunidad-hispana-01', 'comunidad-hispana-01', 's1w1', 'stmarie', 'St. Marie Du Mont', 'warfare', 101, 102, '2026-06-08T18:00:00Z', '2026-06-08T19:00:00Z', 5, 3, 'allied', 'exact', ?),
('comunidad-hispana-01', 'comunidad-hispana-01', 's1w2', 'foy', 'Foy', 'warfare', 103, 104, '2026-06-09T18:00:00Z', '2026-06-09T19:00:00Z', 2, 5, 'axis', 'exact', ?),
('comunidad-hispana-01', 'comunidad-hispana-01', 's1w3', 'kursk', 'Kursk', 'warfare', 105, 106, '2026-06-10T18:00:00Z', '2026-06-10T19:00:00Z', 4, 1, 'allied', 'exact', ?),
('comunidad-hispana-02', 'comunidad-hispana-02', 's2w1', 'hurtgen', 'Hurtgen Forest', 'warfare', 201, 202, '2026-06-08T20:00:00Z', '2026-06-08T21:00:00Z', 5, 0, 'allied', 'exact', ?),
('comunidad-hispana-02', 'comunidad-hispana-02', 's2w2', 'kharkov', 'Kharkov', 'warfare', 203, 204, '2026-06-09T20:00:00Z', '2026-06-09T21:00:00Z', 3, 2, 'allied', 'exact', ?),
('comunidad-hispana-02', 'comunidad-hispana-02', 's2w3', 'driel', 'Driel', 'warfare', 205, 206, '2026-06-10T20:00:00Z', '2026-06-10T21:00:00Z', 2, 1, 'allied', 'exact', ?),
('comunidad-hispana-01', 'comunidad-hispana-01', 'old-year', 'omaha', 'Omaha Beach', 'warfare', 301, 302, '2025-12-15T18:00:00Z', '2025-12-15T19:00:00Z', 1, 0, 'allied', 'exact', ?)
""",
(
MATCH_RESULT_SOURCE,
MATCH_RESULT_SOURCE,
MATCH_RESULT_SOURCE,
MATCH_RESULT_SOURCE,
MATCH_RESULT_SOURCE,
MATCH_RESULT_SOURCE,
MATCH_RESULT_SOURCE,
),
)
connection.execute(
"""
INSERT INTO rcon_match_player_stats (
target_key, match_key, player_id, player_name, team,
kills, deaths, teamkills, deaths_by_teamkill,
weapons_json, death_by_weapons_json, most_killed_json, death_by_json,
first_seen_server_time, last_seen_server_time
) VALUES
('comunidad-hispana-01', 's1w1', 'regression-player', 'Old Regression', 'allied', 10, 4, 1, 0, '{}', '{}', '{}', '{}', 101, 102),
('comunidad-hispana-01', 's1w2', 'regression-player', 'Regression Hero', 'axis', 9, 5, 0, 0, '{}', '{}', '{}', '{}', 103, 104),
('comunidad-hispana-01', 's1w3', 'regression-player', 'Regression Hero', 'allied', 11, 3, 0, 0, '{}', '{}', '{}', '{}', 105, 106),
('comunidad-hispana-02', 's2w1', 'regression-player', 'Regression Hero', 'allied', 4, 2, 0, 0, '{}', '{}', '{}', '{}', 201, 202),
('comunidad-hispana-02', 's2w2', 'regression-player', 'Regression Hero', 'allied', 5, 4, 0, 0, '{}', '{}', '{}', '{}', 203, 204),
('comunidad-hispana-02', 's2w3', 'regression-player', 'Regression Hero', 'allied', 3, 2, 0, 0, '{}', '{}', '{}', '{}', 205, 206),
('comunidad-hispana-01', 'old-year', 'regression-player', 'Historic Regression', 'allied', 99, 1, 0, 0, '{}', '{}', '{}', '{}', 301, 302),
('comunidad-hispana-01', 's1w1', 'other-player', 'Other Player', 'axis', 5, 7, 0, 0, '{}', '{}', '{}', '{}', 101, 102),
('comunidad-hispana-01', 's1w2', 'other-player', 'Other Player', 'allied', 3, 6, 0, 0, '{}', '{}', '{}', '{}', 103, 104),
('comunidad-hispana-01', 's1w3', 'other-player', 'Other Player', 'axis', 4, 8, 0, 0, '{}', '{}', '{}', '{}', 105, 106),
('comunidad-hispana-02', 's2w1', 'other-player', 'Other Player', 'axis', 1, 4, 0, 0, '{}', '{}', '{}', '{}', 201, 202),
('comunidad-hispana-02', 's2w2', 'other-player', 'Other Player', 'axis', 2, 3, 0, 0, '{}', '{}', '{}', '{}', 203, 204),
('comunidad-hispana-02', 's2w3', 'other-player', 'Other Player', 'axis', 1, 2, 0, 0, '{}', '{}', '{}', '{}', 205, 206)
"""
)
connection.close()
return db_path
def validate_player_period_stats_refresh_and_read_model():
db_path = build_player_period_stats_fixture()
anchor = datetime(2026, 6, 10, 22, 0, 0, tzinfo=timezone.utc)
try:
payload = refresh_player_period_stats(
db_path=db_path,
now=anchor,
)
require(payload.get("status") == "ok", "Player period stats refresh should return ok.")
require(payload.get("period_types") == ["weekly", "monthly", "yearly"], "Player period stats refresh should report the supported periods.")
require_int(payload.get("total_rows"), "Player period stats refresh should report numeric total_rows.")
connection = sqlite3.connect(db_path)
connection.row_factory = sqlite3.Row
weekly_row = connection.execute(
"""
SELECT *
FROM player_period_stats
WHERE period_type = 'weekly'
AND server_id = 'all-servers'
AND player_id = 'regression-player'
LIMIT 1
"""
).fetchone()
monthly_row = connection.execute(
"""
SELECT *
FROM player_period_stats
WHERE period_type = 'monthly'
AND server_id = 'all-servers'
AND player_id = 'regression-player'
LIMIT 1
"""
).fetchone()
yearly_row = connection.execute(
"""
SELECT *
FROM player_period_stats
WHERE period_type = 'yearly'
AND server_id = 'all-servers'
AND player_id = 'regression-player'
LIMIT 1
"""
).fetchone()
server_row = connection.execute(
"""
SELECT *
FROM player_period_stats
WHERE period_type = 'weekly'
AND server_id = 'comunidad-hispana-01'
AND player_id = 'regression-player'
LIMIT 1
"""
).fetchone()
connection.close()
require(weekly_row is not None, "Player period stats should materialize a weekly all-servers row.")
require(monthly_row is not None, "Player period stats should materialize a monthly all-servers row.")
require(yearly_row is not None, "Player period stats should materialize a yearly all-servers row.")
require(server_row is not None, "Player period stats should materialize server-scoped rows.")
require(int(weekly_row["matches_considered"] or 0) == 6, "Weekly all-servers row should aggregate six closed matches.")
require(int(weekly_row["kills"] or 0) == 42, "Weekly all-servers row should aggregate weekly kills.")
require(int(weekly_row["deaths"] or 0) == 20, "Weekly all-servers row should aggregate weekly deaths.")
require(int(weekly_row["teamkills"] or 0) == 1, "Weekly all-servers row should aggregate weekly teamkills.")
require(int(weekly_row["ranking_position"] or 0) == 1, "Weekly all-servers row should persist ranking_position.")
require(str(weekly_row["window_kind"]) == "current-week", "Weekly all-servers row should preserve the selected window kind.")
require(str(monthly_row["window_kind"]) == "current-month", "Monthly all-servers row should preserve the selected window kind.")
require(int(yearly_row["kills"] or 0) == 42, "Yearly row should exclude prior-year matches.")
require(int(server_row["matches_considered"] or 0) == 3, "Server-scoped weekly row should aggregate only server-local matches.")
read_model_result = get_rcon_materialized_player_stats(
player_id="regression-player",
server_id="all",
timeframe="weekly",
db_path=db_path,
)
require(
(read_model_result.get("source") or {}).get("read_model") == "player-period-stats",
"Player profile should prefer player_period_stats when populated.",
)
require(
(read_model_result.get("source") or {}).get("fallback_used") is False,
"Player profile should not report fallback when player_period_stats is used.",
)
require(read_model_result.get("matches_considered") == 6, "Player period stats read path should preserve weekly matches_considered.")
require(read_model_result.get("kills") == 42, "Player period stats read path should preserve weekly kills.")
require((read_model_result.get("weekly_ranking") or {}).get("ranking_position") == 1, "Player profile weekly ranking should come from the read model.")
require((read_model_result.get("monthly_ranking") or {}).get("ranking_position") == 1, "Player profile monthly ranking should come from the read model.")
finally:
if db_path.exists():
try:
db_path.unlink()
except PermissionError:
pass
def validate_player_period_stats_fallbacks():
db_path = build_player_period_stats_fixture()
try:
empty_result = get_rcon_materialized_player_stats(
player_id="regression-player",
server_id="all",
timeframe="weekly",
db_path=db_path,
)
require(
(empty_result.get("source") or {}).get("fallback_used") is True,
"Player profile should fall back to runtime when player_period_stats is empty.",
)
require(
(empty_result.get("source") or {}).get("fallback_reason") == "player-period-stats-empty",
"Player profile should expose an explicit empty read-model fallback reason.",
)
refresh_player_period_stats(
db_path=db_path,
now=datetime(2026, 6, 10, 22, 0, 0, tzinfo=timezone.utc),
)
connection = sqlite3.connect(db_path)
with connection:
connection.execute(
"""
DELETE FROM player_period_stats
WHERE period_type = 'monthly'
AND server_id = 'all-servers'
AND player_id = 'regression-player'
"""
)
connection.close()
missing_row_result = get_rcon_materialized_player_stats(
player_id="regression-player",
server_id="all",
timeframe="weekly",
db_path=db_path,
)
require(
(missing_row_result.get("source") or {}).get("fallback_used") is True,
"Player profile should fall back when one required player period row is missing.",
)
require(
(missing_row_result.get("source") or {}).get("fallback_reason") == "player-period-stats-player-missing",
"Player profile should expose an explicit missing-player-row fallback reason.",
)
original_read_model = player_search_stats._get_player_period_stats_read_model
def fake_get_player_period_stats_read_model(**kwargs):
return None, "player-period-stats-unavailable"
player_search_stats._get_player_period_stats_read_model = fake_get_player_period_stats_read_model
try:
unavailable_result = get_rcon_materialized_player_stats(
player_id="regression-player",
server_id="all",
timeframe="weekly",
db_path=db_path,
)
finally:
player_search_stats._get_player_period_stats_read_model = original_read_model
require(
(unavailable_result.get("source") or {}).get("fallback_used") is True,
"Player profile should preserve runtime fallback when the read model is unavailable.",
)
require(
(unavailable_result.get("source") or {}).get("fallback_reason") == "player-period-stats-unavailable",
"Player profile should expose an explicit unavailable read-model fallback reason.",
)
finally:
if db_path.exists():
try:
db_path.unlink()
except PermissionError:
pass
def cleanup_snapshot_fixture(db_path):
connection = sqlite3.connect(db_path)
with connection:
@@ -870,6 +1200,10 @@ validate_postgres_player_search_index_schema_path()
validate_player_search_index_cli_defaults()
validate_player_search_index_refresh_and_search()
validate_player_search_index_fallbacks()
validate_postgres_player_period_stats_schema_path()
validate_player_period_stats_cli_defaults()
validate_player_period_stats_refresh_and_read_model()
validate_player_period_stats_fallbacks()
kd_metric_sql, _, _ = ranking_leaderboards._resolve_metric_sql("kd_ratio")
require(
@@ -1241,21 +1575,24 @@ print(json.dumps({
"stats-player-profile",
"stats-annual-ranking",
"global-ranking",
"postgres-ranking-derived-metric-sql",
"postgres-ranking-schema-path",
"ranking-snapshot-cli-postgres-default",
"ranking-snapshot-bulk-cli",
"ranking-snapshot-postgres-selection",
"ranking-snapshot-bulk-refresh",
"ranking-snapshot-bulk-partial-failure",
"ranking-snapshot-generator",
"ranking-snapshot-ready",
"ranking-snapshot-missing",
],
"postgres-ranking-derived-metric-sql",
"postgres-ranking-schema-path",
"ranking-snapshot-cli-postgres-default",
"ranking-snapshot-bulk-cli",
"ranking-snapshot-postgres-selection",
"ranking-snapshot-bulk-refresh",
"ranking-snapshot-bulk-partial-failure",
"ranking-snapshot-generator",
"ranking-snapshot-ready",
"ranking-snapshot-missing",
"player-search-index",
"player-period-stats",
],
"annual_snapshot_status": annual_data.get("snapshot_status"),
"global_ranking_annual_snapshot_status": annual_ranking_data.get("snapshot_status"),
"search_items_count": len(search_data.get("items") or []),
"profile_matches_considered": profile_data.get("matches_considered"),
"profile_read_model_source": (profile_data.get("source") or {}).get("read_model"),
}))
'@