Fix PostgreSQL ranking snapshot schema
This commit is contained in:
devRaGonSa
126
ai/tasks/done/TASK-192-fix-postgres-ranking-snapshot-schema.md
Normal file
126
ai/tasks/done/TASK-192-fix-postgres-ranking-snapshot-schema.md
Normal file
@@ -0,0 +1,126 @@
|
||||
---
|
||||
id: TASK-192-fix-postgres-ranking-snapshot-schema
|
||||
title: Fix PostgreSQL ranking snapshot schema
|
||||
status: done
|
||||
type: backend
|
||||
team: Backend Senior
|
||||
supporting_teams:
|
||||
- Arquitecto de Base de Datos
|
||||
- Arquitecto Python
|
||||
roadmap_item: foundation
|
||||
priority: high
|
||||
---
|
||||
|
||||
# TASK-192 - Fix PostgreSQL ranking snapshot schema
|
||||
|
||||
## Goal
|
||||
|
||||
Apply a backend hotfix so weekly and monthly ranking snapshot storage initializes correctly in PostgreSQL without executing SQLite-only schema syntax.
|
||||
|
||||
## Context
|
||||
|
||||
Production `GET /api/ranking?timeframe=weekly&metric=kills&limit=20` currently fails because `backend/app/rcon_historical_leaderboards.py` executes SQLite DDL with `AUTOINCREMENT` through `psycopg`. HLL Vietnam operational storage must stay in PostgreSQL, including ranking snapshots, and this fix must preserve the current annual snapshot path and avoid any SQLite expansion for this functionality.
|
||||
|
||||
Preserve the current product identity: Spanish-speaking HLL Vietnam community, military/Vietnam/tactical/sober visual direction and controlled repository evolution.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Inspect the listed files first.
|
||||
2. Create the PostgreSQL-compatible ranking snapshot schema following the existing backend storage pattern.
|
||||
3. Keep annual snapshot storage intact and scoped to `rcon_annual_ranking_snapshots`.
|
||||
4. Validate the backend path and document the root cause, fix and production verification steps.
|
||||
|
||||
## Files to Read First
|
||||
|
||||
- `AGENTS.md`
|
||||
- `ai/repo-context.md`
|
||||
- `ai/architecture-index.md`
|
||||
- `backend/app/rcon_historical_leaderboards.py`
|
||||
- `backend/app/postgres_rcon_storage.py`
|
||||
- `backend/app/rcon_admin_log_materialization.py`
|
||||
- `backend/app/rcon_annual_rankings.py`
|
||||
- `backend/app/payloads.py`
|
||||
- `backend/app/routes.py`
|
||||
- `scripts/run-stats-validation.ps1`
|
||||
- `ai/tasks/done/TASK-191-serve-ranking-from-snapshots-with-runtime-fallback.md`
|
||||
|
||||
## Expected Files to Modify
|
||||
|
||||
- `backend/app/rcon_historical_leaderboards.py`
|
||||
- `backend/app/postgres_rcon_storage.py`, solo si el patrón del proyecto lo requiere
|
||||
- `scripts/run-stats-validation.ps1`, solo si se añade validación de regresión
|
||||
- `ai/tasks/done/TASK-192-fix-postgres-ranking-snapshot-schema.md`
|
||||
|
||||
## Constraints
|
||||
|
||||
- Keep the change minimal.
|
||||
- Preserve HLL Vietnam project identity.
|
||||
- Do not introduce unnecessary frameworks or dependencies.
|
||||
- Do not create or depend on a separate SQLite database for ranking snapshots.
|
||||
- Do not expand legacy SQLite compatibility routes for this functionality beyond what already exists.
|
||||
- Do not modify frontend, visual logic, Elo/MMR or Comunidad Hispana #03 handling.
|
||||
- Keep annual ranking on `rcon_annual_ranking_snapshots`.
|
||||
- Do not overwrite repository-specific context with generic platform template text.
|
||||
|
||||
## Validation
|
||||
|
||||
Before completing the task ensure:
|
||||
|
||||
- `node --check frontend/assets/js/ranking.js`
|
||||
- `node --check frontend/assets/js/stats.js`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||
- local validation proves `initialize_ranking_snapshot_storage` does not contain or execute `AUTOINCREMENT` on the PostgreSQL path
|
||||
- local validation proves PostgreSQL schema defines `ranking_snapshots` and `ranking_snapshot_items` with PostgreSQL-compatible syntax
|
||||
- `/api/ranking?timeframe=weekly&metric=kills&limit=20` no longer fails because of table creation
|
||||
- `/api/ranking?timeframe=annual&metric=kills&limit=20&year=2026` keeps working
|
||||
- `git diff --name-only` matches the expected scope
|
||||
|
||||
If local PostgreSQL is unavailable, document that in Outcome and validate by code inspection plus existing scripts.
|
||||
|
||||
## Outcome
|
||||
|
||||
- Root cause:
|
||||
- `backend/app/rcon_historical_leaderboards.py` defined `ranking_snapshots` / `ranking_snapshot_items` with SQLite syntax and executed that same DDL through `psycopg` on the PostgreSQL path.
|
||||
- Production failure matched the SQLite-only token exactly: `AUTOINCREMENT`.
|
||||
- Confirmation:
|
||||
- The failing path was `build_global_ranking_payload` -> `get_latest_ranking_snapshot` -> `initialize_ranking_snapshot_storage`.
|
||||
- The hotfix removes PostgreSQL execution of the SQLite snapshot DDL and routes PostgreSQL initialization through the central PostgreSQL schema initializer.
|
||||
- Applied change:
|
||||
- Kept the SQLite snapshot schema local to the SQLite path in `backend/app/rcon_historical_leaderboards.py`.
|
||||
- Added PostgreSQL-compatible `ranking_snapshots` and `ranking_snapshot_items` tables plus indexes to `backend/app/postgres_rcon_storage.py`.
|
||||
- Used `BIGSERIAL` primary keys and PostgreSQL-compatible `BIGINT`, `DOUBLE PRECISION`, `TIMESTAMPTZ`, foreign keys and index syntax.
|
||||
- Added regression validation in `scripts/run-stats-validation.ps1` to prove the PostgreSQL path does not depend on `AUTOINCREMENT` and that it delegates to `initialize_postgres_rcon_storage`.
|
||||
- Expected PostgreSQL tables:
|
||||
- `ranking_snapshots`
|
||||
- `ranking_snapshot_items`
|
||||
- Annual ranking remains on:
|
||||
- `rcon_annual_ranking_snapshots`
|
||||
- `rcon_annual_ranking_snapshot_items`
|
||||
- Validations executed:
|
||||
- `node --check frontend/assets/js/ranking.js`
|
||||
- `node --check frontend/assets/js/stats.js`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-stats-validation.ps1`
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
|
||||
- Local regression validation now proves:
|
||||
- PostgreSQL schema text contains `ranking_snapshots` and `ranking_snapshot_items`
|
||||
- PostgreSQL schema text does not contain `AUTOINCREMENT`
|
||||
- `initialize_ranking_snapshot_storage()` delegates to PostgreSQL schema initialization on the PostgreSQL path
|
||||
- `/api/ranking?timeframe=annual&year=2026&server_id=all&metric=kills&limit=20` still resolves
|
||||
- Environment note:
|
||||
- Local backend HTTP at `http://127.0.0.1:8000` was not available during validation.
|
||||
- Validation therefore completed by local Python imports, route resolution and existing scripts, which passed.
|
||||
- How to validate in production:
|
||||
- Deploy the backend hotfix.
|
||||
- Confirm PostgreSQL contains `ranking_snapshots` and `ranking_snapshot_items`.
|
||||
- Request:
|
||||
- `/api/ranking?timeframe=weekly&metric=kills&limit=20`
|
||||
- `/api/ranking?timeframe=monthly&metric=kills&limit=20`
|
||||
- `/api/ranking?timeframe=annual&metric=kills&limit=20&year=2026`
|
||||
- Verify weekly/monthly no longer fail during table initialization and annual remains unchanged.
|
||||
|
||||
## Change Budget
|
||||
|
||||
- Prefer fewer than 5 modified files.
|
||||
- Prefer changes under 200 lines when feasible.
|
||||
- Split the work into follow-up tasks if limits are exceeded.
|
||||
@@ -221,6 +221,43 @@ CREATE TABLE IF NOT EXISTS rcon_annual_ranking_snapshot_items (
|
||||
UNIQUE(snapshot_id, player_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS ranking_snapshots (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
timeframe TEXT NOT NULL,
|
||||
server_id TEXT NOT NULL,
|
||||
metric TEXT NOT NULL,
|
||||
window_start TEXT NOT NULL,
|
||||
window_end TEXT NOT NULL,
|
||||
generated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
source TEXT NOT NULL DEFAULT 'rcon-materialized-admin-log',
|
||||
snapshot_status TEXT NOT NULL DEFAULT 'ready',
|
||||
item_count INTEGER NOT NULL DEFAULT 0,
|
||||
limit_size INTEGER NOT NULL DEFAULT 20,
|
||||
source_matches_count INTEGER NOT NULL DEFAULT 0,
|
||||
freshness TEXT NOT NULL DEFAULT 'fresh',
|
||||
window_kind TEXT,
|
||||
window_label TEXT,
|
||||
error_message TEXT,
|
||||
UNIQUE(timeframe, server_id, metric, window_start, window_end)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS ranking_snapshot_items (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
snapshot_id BIGINT NOT NULL REFERENCES ranking_snapshots(id) ON DELETE CASCADE,
|
||||
ranking_position INTEGER NOT NULL,
|
||||
player_id TEXT NOT NULL,
|
||||
player_name TEXT NOT NULL,
|
||||
metric_value DOUBLE PRECISION NOT NULL DEFAULT 0,
|
||||
matches_considered INTEGER NOT NULL DEFAULT 0,
|
||||
kills INTEGER NOT NULL DEFAULT 0,
|
||||
deaths INTEGER NOT NULL DEFAULT 0,
|
||||
teamkills INTEGER NOT NULL DEFAULT 0,
|
||||
kd_ratio DOUBLE PRECISION NOT NULL DEFAULT 0.0,
|
||||
kills_per_match DOUBLE PRECISION NOT NULL DEFAULT 0.0,
|
||||
UNIQUE(snapshot_id, ranking_position),
|
||||
UNIQUE(snapshot_id, player_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS rcon_scoreboard_match_candidates (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
server_slug TEXT NOT NULL,
|
||||
@@ -279,6 +316,12 @@ CREATE INDEX IF NOT EXISTS idx_rcon_annual_snapshot_items_snapshot
|
||||
ON rcon_annual_ranking_snapshot_items(snapshot_id, ranking_position);
|
||||
CREATE INDEX IF NOT EXISTS idx_rcon_annual_snapshot_items_player
|
||||
ON rcon_annual_ranking_snapshot_items(snapshot_id, player_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_ranking_snapshots_lookup
|
||||
ON ranking_snapshots(timeframe, server_id, metric, snapshot_status, window_end DESC, generated_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_ranking_snapshot_items_snapshot
|
||||
ON ranking_snapshot_items(snapshot_id, ranking_position);
|
||||
CREATE INDEX IF NOT EXISTS idx_ranking_snapshot_items_player
|
||||
ON ranking_snapshot_items(snapshot_id, player_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_rcon_scoreboard_candidates_server_end
|
||||
ON rcon_scoreboard_match_candidates(server_slug, ended_at DESC, started_at DESC);
|
||||
"""
|
||||
|
||||
@@ -29,7 +29,7 @@ LeaderboardMetric = Literal[
|
||||
"support",
|
||||
]
|
||||
|
||||
RANKING_SNAPSHOT_SCHEMA_SQL = """
|
||||
RANKING_SNAPSHOT_SQLITE_SCHEMA_SQL = """
|
||||
CREATE TABLE IF NOT EXISTS ranking_snapshots (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timeframe TEXT NOT NULL,
|
||||
@@ -82,16 +82,14 @@ def initialize_ranking_snapshot_storage(*, db_path: Path | None = None) -> Path:
|
||||
"""Create ranking snapshot tables used by weekly/monthly public ranking reads."""
|
||||
resolved_path = initialize_rcon_materialized_storage(db_path=db_path)
|
||||
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
|
||||
from .postgres_rcon_storage import connect_postgres
|
||||
from .postgres_rcon_storage import initialize_postgres_rcon_storage
|
||||
|
||||
with connect_postgres() as connection:
|
||||
with connection.cursor() as cursor:
|
||||
cursor.execute(RANKING_SNAPSHOT_SCHEMA_SQL)
|
||||
initialize_postgres_rcon_storage()
|
||||
return resolved_path
|
||||
|
||||
with closing(connect_sqlite_writer(resolved_path)) as connection:
|
||||
with connection:
|
||||
connection.executescript(RANKING_SNAPSHOT_SCHEMA_SQL)
|
||||
connection.executescript(RANKING_SNAPSHOT_SQLITE_SCHEMA_SQL)
|
||||
return resolved_path
|
||||
|
||||
|
||||
|
||||
@@ -123,7 +123,10 @@ from pathlib import Path
|
||||
sys.path.insert(0, "backend")
|
||||
|
||||
from app.routes import resolve_get_payload
|
||||
from app.rcon_historical_leaderboards import initialize_ranking_snapshot_storage
|
||||
import app.postgres_rcon_storage as postgres_rcon_storage
|
||||
import app.rcon_historical_leaderboards as ranking_leaderboards
|
||||
|
||||
initialize_ranking_snapshot_storage = ranking_leaderboards.initialize_ranking_snapshot_storage
|
||||
|
||||
|
||||
def require(condition, message):
|
||||
@@ -233,6 +236,52 @@ def build_snapshot_fixture():
|
||||
return db_path
|
||||
|
||||
|
||||
def validate_postgres_ranking_snapshot_schema_path():
|
||||
require(
|
||||
"AUTOINCREMENT" not in postgres_rcon_storage.RCON_SCHEMA_SQL,
|
||||
"PostgreSQL schema must not contain AUTOINCREMENT.",
|
||||
)
|
||||
require(
|
||||
"CREATE TABLE IF NOT EXISTS ranking_snapshots" in postgres_rcon_storage.RCON_SCHEMA_SQL,
|
||||
"PostgreSQL schema should define ranking_snapshots.",
|
||||
)
|
||||
require(
|
||||
"CREATE TABLE IF NOT EXISTS ranking_snapshot_items" in postgres_rcon_storage.RCON_SCHEMA_SQL,
|
||||
"PostgreSQL schema should define ranking_snapshot_items.",
|
||||
)
|
||||
require(
|
||||
"BIGSERIAL PRIMARY KEY" in postgres_rcon_storage.RCON_SCHEMA_SQL,
|
||||
"PostgreSQL snapshot schema should use BIGSERIAL primary keys.",
|
||||
)
|
||||
|
||||
original_initialize_materialized = ranking_leaderboards.initialize_rcon_materialized_storage
|
||||
original_use_postgres_storage = ranking_leaderboards.use_postgres_rcon_storage
|
||||
original_initialize_postgres_storage = postgres_rcon_storage.initialize_postgres_rcon_storage
|
||||
calls = {"postgres_init": 0}
|
||||
|
||||
ranking_leaderboards.initialize_rcon_materialized_storage = (
|
||||
lambda db_path=None: Path("backend/data/hll_vietnam_dev.sqlite3")
|
||||
)
|
||||
ranking_leaderboards.use_postgres_rcon_storage = lambda explicit_sqlite_path=None: True
|
||||
|
||||
def fake_initialize_postgres_storage():
|
||||
calls["postgres_init"] += 1
|
||||
|
||||
postgres_rcon_storage.initialize_postgres_rcon_storage = fake_initialize_postgres_storage
|
||||
|
||||
try:
|
||||
initialize_ranking_snapshot_storage()
|
||||
finally:
|
||||
ranking_leaderboards.initialize_rcon_materialized_storage = original_initialize_materialized
|
||||
ranking_leaderboards.use_postgres_rcon_storage = original_use_postgres_storage
|
||||
postgres_rcon_storage.initialize_postgres_rcon_storage = original_initialize_postgres_storage
|
||||
|
||||
require(
|
||||
calls["postgres_init"] == 1,
|
||||
"PostgreSQL ranking snapshot initialization should delegate to initialize_postgres_rcon_storage exactly once.",
|
||||
)
|
||||
|
||||
|
||||
def cleanup_snapshot_fixture(db_path):
|
||||
connection = sqlite3.connect(db_path)
|
||||
with connection:
|
||||
@@ -256,6 +305,8 @@ health_status, health_payload = read_payload("/health")
|
||||
require(health_status == 200, "Route resolver /health should return 200.")
|
||||
require(health_payload.get("status") == "ok", "/health payload should be ok.")
|
||||
|
||||
validate_postgres_ranking_snapshot_schema_path()
|
||||
|
||||
search_status, search_payload = read_payload("/api/stats/players/search?q=regression-check&limit=5")
|
||||
require(search_status == 200, "Stats player search should return 200 for a valid query.")
|
||||
search_data = search_payload.get("data") or {}
|
||||
@@ -425,6 +476,15 @@ require("generated_at" in annual_ranking_data, "Global ranking annual should exp
|
||||
require("window_start" in annual_ranking_data, "Global ranking annual should expose window_start.")
|
||||
require("window_end" in annual_ranking_data, "Global ranking annual should expose window_end.")
|
||||
|
||||
annual_2026_status, annual_2026_payload = read_payload(
|
||||
"/api/ranking?timeframe=annual&year=2026&server_id=all&metric=kills&limit=20"
|
||||
)
|
||||
require(annual_2026_status == 200, "Global ranking annual 2026 route should return 200.")
|
||||
require(
|
||||
((annual_2026_payload.get("data") or {}).get("snapshot_status") in {"ready", "missing"}),
|
||||
"Global ranking annual 2026 should expose ready/missing snapshot status.",
|
||||
)
|
||||
|
||||
for ranking_payload in [
|
||||
weekly_ranking_payload,
|
||||
weekly_deaths_payload,
|
||||
@@ -528,6 +588,7 @@ print(json.dumps({
|
||||
"stats-player-profile",
|
||||
"stats-annual-ranking",
|
||||
"global-ranking",
|
||||
"postgres-ranking-schema-path",
|
||||
"ranking-snapshot-ready",
|
||||
"ranking-snapshot-missing",
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user