Stabilize player stats and historical detail public reads
This commit is contained in:
@@ -0,0 +1,140 @@
|
||||
---
|
||||
id: TASK-225
|
||||
title: Stabilize critical public API endpoints
|
||||
status: done
|
||||
type: backend
|
||||
team: Backend Senior
|
||||
supporting_teams:
|
||||
- Analista
|
||||
- Arquitecto Python
|
||||
- Arquitecto de Base de Datos
|
||||
roadmap_item: foundation
|
||||
priority: high
|
||||
---
|
||||
|
||||
# TASK-225 - Stabilize critical public API endpoints
|
||||
|
||||
## Goal
|
||||
|
||||
Apply focused functional fixes from `docs/FULL_APPLICATION_REQUEST_AUDIT.md` to stabilize the highest-priority public API endpoints that timed out or paid storage initialization/fallback cost during public reads.
|
||||
|
||||
## Context
|
||||
|
||||
`TASK-224` found critical public request issues:
|
||||
|
||||
- `/api/stats/players/search` timed out at 30s.
|
||||
- `/api/stats/players/{player_id}` weekly/monthly timed out at 30s.
|
||||
- `/api/current-match/kills` timed out or returned 500.
|
||||
- `/api/current-match/players` timed out for `comunidad-hispana-01`.
|
||||
- `/api/servers` could spend about 4.3s refreshing RCON from a public GET.
|
||||
- `/api/historical/matches/detail` responded OK in the production probe but still had a static `initialize_*` public read path.
|
||||
|
||||
This task prioritizes safe fixes that do not change RCON connection configuration, server hosts, ports, credentials, Docker networking or frontend assets.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Inspect the listed files first.
|
||||
2. Implement Phase 1 only unless the remaining phases are clearly small and safe.
|
||||
3. Add focused tests proving public read paths do not initialize storage or fall back to heavy runtime scans.
|
||||
4. Validate backend compile/tests.
|
||||
5. Document implemented and deferred phases.
|
||||
|
||||
## Files to Read First
|
||||
|
||||
- `AGENTS.md`
|
||||
- `ai/architecture-index.md`
|
||||
- `ai/repo-context.md`
|
||||
- `docs/FULL_APPLICATION_REQUEST_AUDIT.md`
|
||||
- `backend/app/rcon_historical_player_stats.py`
|
||||
- `backend/app/rcon_admin_log_materialization.py`
|
||||
- `backend/app/rcon_historical_read_model.py`
|
||||
- `backend/app/payloads.py`
|
||||
- `backend/tests/test_rcon_materialization_pipeline.py`
|
||||
|
||||
## Expected Files to Modify
|
||||
|
||||
- `backend/app/rcon_historical_player_stats.py`
|
||||
- `backend/app/rcon_admin_log_materialization.py`
|
||||
- `backend/app/rcon_historical_read_model.py`
|
||||
- `backend/app/postgres_rcon_storage.py`
|
||||
- `backend/app/payloads.py`
|
||||
- `backend/tests/test_rcon_materialization_pipeline.py`
|
||||
- `scripts/audit_public_requests.py`
|
||||
- `docs/FULL_APPLICATION_REQUEST_AUDIT.md`
|
||||
- `docs/PERFORMANCE_PUBLIC_QUERY_AUDIT.md`
|
||||
- `ai/tasks/done/TASK-225-stabilize-critical-public-api-endpoints.md`
|
||||
|
||||
## Constraints
|
||||
|
||||
- Do not run `ai-platform run`.
|
||||
- Do not commit or push.
|
||||
- Do not touch RCON hosts, ports, credentials, environment variables, Docker networking or server connection configuration.
|
||||
- Do not change `27001` or backend/RCON `127.0.0.1`.
|
||||
- Do not touch frontend assets, weapon assets, clan assets, SVGs or physical images.
|
||||
- Do not modify `ai/system-metrics.md`.
|
||||
- Do not reactivate Elo/MMR.
|
||||
- Do not reintroduce Comunidad Hispana #03.
|
||||
|
||||
## Validation
|
||||
|
||||
Required before completion:
|
||||
|
||||
- `python -m compileall backend/app`
|
||||
- `python -m unittest tests.test_rcon_materialization_pipeline`
|
||||
- Relevant focused tests for player search, player detail and historical match detail read-only behavior
|
||||
- Partial runtime measurement if a backend is available
|
||||
|
||||
## Outcome
|
||||
|
||||
Completed Phase 1 only. Phase 2 and Phase 3 were deferred because they touch RCON-live behavior, `/api/servers`, or public frontend references and should be handled separately after the non-RCON public read paths are redeployed and measured.
|
||||
|
||||
Implemented:
|
||||
|
||||
- `/api/stats/players/search` now reads `player_search_index` strictly. It no longer initializes player search storage or falls back to runtime scans in the public helper. Missing/empty read model returns a controlled empty payload with `fallback_used: false`.
|
||||
- `/api/stats/players/{player_id}` weekly/monthly now reads `player_period_stats` strictly. Missing read model or missing player-period data returns a controlled zero-stat payload with `fallback_used: false`.
|
||||
- `/api/historical/matches/detail` now uses `get_materialized_rcon_match_detail(..., ensure_storage=False)` in the public RCON read path. The materialized detail helper can read without calling `initialize_rcon_materialized_storage()` or `initialize_postgres_rcon_storage()`. Missing storage/tables return `None`, and the RCON public payload returns `found: false`, `fallback_used: false` instead of falling through to legacy fallback.
|
||||
- `connect_postgres_compat()` now accepts `initialize=False` for read-only public paths while preserving the previous initializing default for existing write/materialization callers.
|
||||
- `scripts/audit_public_requests.py` now supports `--filter` and `--player-id` for partial endpoint measurement.
|
||||
- Documentation was updated in `docs/FULL_APPLICATION_REQUEST_AUDIT.md` and `docs/PERFORMANCE_PUBLIC_QUERY_AUDIT.md`.
|
||||
|
||||
Not implemented:
|
||||
|
||||
- Phase 2A `/api/current-match/kills` and `/api/current-match/players`: deferred. These depend on RCON-live behavior and must be hardened without changing RCON host/port/configuration.
|
||||
- Phase 2B `/api/servers`: deferred. It needs a separate read/cache change to avoid mixing server freshness decisions into the stats/read-model fix.
|
||||
- Phase 3 frontend localhost cleanup: deferred because the user requested no frontend changes in this implementation pass.
|
||||
- Phase 3B historical snapshot missing/fallback generation: deferred because it likely belongs to scheduler/generation design, not this hot-path stabilization.
|
||||
|
||||
Validation performed:
|
||||
|
||||
- `python -m compileall backend\app`: OK.
|
||||
- `python -m py_compile scripts\audit_public_requests.py`: OK.
|
||||
- `cd backend; python -m unittest tests.test_rcon_materialization_pipeline`: OK, 12 tests. The suite still emits pre-existing SQLite `ResourceWarning` messages.
|
||||
- `cd backend; python -m unittest tests.test_current_match_payload tests.test_rcon_admin_log_storage tests.test_historical_snapshot_refresh`: OK, 13 tests.
|
||||
- Partial audit script selection against `http://127.0.0.1:8000`:
|
||||
- `--filter stats-player-search`: selected 1 probe, failed because no local backend was listening.
|
||||
- `--player-id 76561198092154180 --filter stats-player-profile`: selected weekly/monthly probes, failed because no local backend was listening.
|
||||
- `--filter historical-match-detail`: selected 1 probe, failed because no local backend was listening.
|
||||
|
||||
Commands to run after redeploy:
|
||||
|
||||
```powershell
|
||||
python scripts\audit_public_requests.py --base-url https://comunidadhll.devzamode.es --timeout 30 --filter stats-player-search --output tmp\task225_prod_player_search_audit.json
|
||||
python scripts\audit_public_requests.py --base-url https://comunidadhll.devzamode.es --timeout 30 --player-id 76561198092154180 --filter stats-player-profile --output tmp\task225_prod_player_profile_audit.json
|
||||
python scripts\audit_public_requests.py --base-url https://comunidadhll.devzamode.es --timeout 30 --filter historical-match-detail --output tmp\task225_prod_match_detail_audit.json
|
||||
```
|
||||
|
||||
Restrictions confirmed:
|
||||
|
||||
- Did not run `ai-platform run`.
|
||||
- Did not commit or push.
|
||||
- Did not modify frontend files.
|
||||
- Did not touch weapon assets, clan assets, SVGs or physical images.
|
||||
- Did not modify `ai/system-metrics.md`.
|
||||
- Did not change `27001`.
|
||||
- Did not change RCON hosts, ports, credentials, environment variables, Docker networking or server configuration.
|
||||
- Did not reactivate Elo/MMR.
|
||||
- Did not reintroduce Comunidad Hispana #03.
|
||||
|
||||
## Change Budget
|
||||
|
||||
Prefer Phase 1 only if Phase 2/3 would increase production risk or require touching frontend/configuration.
|
||||
@@ -1147,6 +1147,30 @@ def build_historical_match_detail_payload(
|
||||
"item": item,
|
||||
},
|
||||
}
|
||||
return {
|
||||
"status": "ok",
|
||||
"data": {
|
||||
"title": "Detalle de partida historica",
|
||||
"context": "historical-match-detail",
|
||||
"source": "rcon-historical-competitive-read-model",
|
||||
"found": False,
|
||||
**build_source_policy(
|
||||
primary_source=SOURCE_KIND_RCON,
|
||||
selected_source=SOURCE_KIND_RCON,
|
||||
fallback_used=False,
|
||||
fallback_reason=None,
|
||||
source_attempts=[
|
||||
build_source_attempt(
|
||||
source=SOURCE_KIND_RCON,
|
||||
role="primary",
|
||||
status="empty",
|
||||
reason="historical-match-detail-read-model-missing",
|
||||
)
|
||||
],
|
||||
),
|
||||
"item": None,
|
||||
},
|
||||
}
|
||||
|
||||
item = get_historical_match_detail(server_slug=server_slug, match_id=match_id)
|
||||
return {
|
||||
|
||||
@@ -453,9 +453,10 @@ class PostgresCompatConnection:
|
||||
|
||||
|
||||
@contextmanager
|
||||
def connect_postgres_compat():
|
||||
def connect_postgres_compat(*, initialize: bool = True):
|
||||
"""Yield a query shim that accepts the phase-1 SQLite-style placeholders."""
|
||||
initialize_postgres_rcon_storage()
|
||||
if initialize:
|
||||
initialize_postgres_rcon_storage()
|
||||
with connect_postgres() as connection:
|
||||
yield PostgresCompatConnection(connection)
|
||||
|
||||
|
||||
@@ -310,66 +310,74 @@ def get_materialized_rcon_match_detail(
|
||||
server_key: str,
|
||||
match_key: str,
|
||||
db_path: Path | None = None,
|
||||
ensure_storage: bool = False,
|
||||
) -> dict[str, object] | None:
|
||||
"""Return one materialized match with player stats."""
|
||||
resolved_path = initialize_rcon_materialized_storage(db_path=db_path)
|
||||
resolved_path = (
|
||||
initialize_rcon_materialized_storage(db_path=db_path)
|
||||
if ensure_storage
|
||||
else (db_path or get_storage_path())
|
||||
)
|
||||
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
|
||||
from .postgres_rcon_storage import connect_postgres_compat
|
||||
|
||||
connection_scope = connect_postgres_compat()
|
||||
connection_scope = connect_postgres_compat(initialize=ensure_storage)
|
||||
else:
|
||||
connection_scope = closing(connect_sqlite_readonly(resolved_path))
|
||||
with connection_scope as connection:
|
||||
match = connection.execute(
|
||||
"""
|
||||
SELECT *
|
||||
FROM rcon_materialized_matches
|
||||
WHERE match_key = ?
|
||||
AND (target_key = ? OR external_server_id = ?)
|
||||
LIMIT 1
|
||||
""",
|
||||
(match_key, server_key, server_key),
|
||||
).fetchone()
|
||||
if match is None and match_key.startswith(f"{server_key}:"):
|
||||
try:
|
||||
with connection_scope as connection:
|
||||
match = connection.execute(
|
||||
"""
|
||||
SELECT *
|
||||
FROM rcon_materialized_matches
|
||||
WHERE match_key = ?
|
||||
AND (target_key = ? OR external_server_id = ?)
|
||||
LIMIT 1
|
||||
""",
|
||||
(match_key,),
|
||||
(match_key, server_key, server_key),
|
||||
).fetchone()
|
||||
if match is None:
|
||||
return None
|
||||
stat_rows = connection.execute(
|
||||
"""
|
||||
SELECT *
|
||||
FROM rcon_match_player_stats
|
||||
WHERE target_key = ? AND match_key = ?
|
||||
ORDER BY kills DESC, deaths ASC, player_name ASC
|
||||
""",
|
||||
(match["target_key"], match["match_key"]),
|
||||
).fetchall()
|
||||
timeline_rows = connection.execute(
|
||||
"""
|
||||
SELECT event_type, COUNT(*) AS event_count
|
||||
FROM rcon_admin_log_events
|
||||
WHERE target_key = ?
|
||||
AND server_time IS NOT NULL
|
||||
AND (? IS NULL OR server_time >= ?)
|
||||
AND (? IS NULL OR server_time <= ?)
|
||||
GROUP BY event_type
|
||||
ORDER BY event_count DESC, event_type ASC
|
||||
""",
|
||||
(
|
||||
match["target_key"],
|
||||
match["started_server_time"],
|
||||
match["started_server_time"],
|
||||
match["ended_server_time"],
|
||||
match["ended_server_time"],
|
||||
),
|
||||
).fetchall()
|
||||
if match is None and match_key.startswith(f"{server_key}:"):
|
||||
match = connection.execute(
|
||||
"""
|
||||
SELECT *
|
||||
FROM rcon_materialized_matches
|
||||
WHERE match_key = ?
|
||||
LIMIT 1
|
||||
""",
|
||||
(match_key,),
|
||||
).fetchone()
|
||||
if match is None:
|
||||
return None
|
||||
stat_rows = connection.execute(
|
||||
"""
|
||||
SELECT *
|
||||
FROM rcon_match_player_stats
|
||||
WHERE target_key = ? AND match_key = ?
|
||||
ORDER BY kills DESC, deaths ASC, player_name ASC
|
||||
""",
|
||||
(match["target_key"], match["match_key"]),
|
||||
).fetchall()
|
||||
timeline_rows = connection.execute(
|
||||
"""
|
||||
SELECT event_type, COUNT(*) AS event_count
|
||||
FROM rcon_admin_log_events
|
||||
WHERE target_key = ?
|
||||
AND server_time IS NOT NULL
|
||||
AND (? IS NULL OR server_time >= ?)
|
||||
AND (? IS NULL OR server_time <= ?)
|
||||
GROUP BY event_type
|
||||
ORDER BY event_count DESC, event_type ASC
|
||||
""",
|
||||
(
|
||||
match["target_key"],
|
||||
match["started_server_time"],
|
||||
match["started_server_time"],
|
||||
match["ended_server_time"],
|
||||
match["ended_server_time"],
|
||||
),
|
||||
).fetchall()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
return {
|
||||
"match": dict(match),
|
||||
|
||||
@@ -9,7 +9,7 @@ from datetime import date, datetime, timezone
|
||||
from pathlib import Path
|
||||
import unicodedata
|
||||
|
||||
from .config import use_postgres_rcon_storage
|
||||
from .config import get_storage_path, use_postgres_rcon_storage
|
||||
from .historical_storage import ALL_SERVERS_SLUG
|
||||
from .rcon_admin_log_materialization import MATCH_RESULT_SOURCE, initialize_rcon_materialized_storage
|
||||
from .rcon_historical_leaderboards import select_leaderboard_window
|
||||
@@ -227,7 +227,7 @@ def search_rcon_materialized_players(
|
||||
limit: int = 10,
|
||||
db_path: Path | None = None,
|
||||
) -> dict[str, object]:
|
||||
"""Search players from the read model first, with runtime fallback."""
|
||||
"""Search players from the public read model without runtime fallback."""
|
||||
normalized_query = query.strip()
|
||||
if not normalized_query:
|
||||
raise ValueError("Query cannot be empty.")
|
||||
@@ -241,18 +241,19 @@ def search_rcon_materialized_players(
|
||||
if read_model_result is not None:
|
||||
return read_model_result
|
||||
|
||||
runtime_result = _search_rcon_materialized_players_runtime(
|
||||
query=normalized_query,
|
||||
server_id=server_id,
|
||||
limit=limit,
|
||||
db_path=db_path,
|
||||
)
|
||||
runtime_result["source"] = {
|
||||
"read_model": "rcon-materialized-admin-log-player-stats",
|
||||
"fallback_used": True,
|
||||
"fallback_reason": fallback_reason or "player-search-index-unavailable",
|
||||
resolved_server_id = _normalize_server_id(server_id)
|
||||
return {
|
||||
"server_id": resolved_server_id,
|
||||
"query": normalized_query,
|
||||
"items": [],
|
||||
"source": {
|
||||
"read_model": "player-search-index",
|
||||
"status": "unavailable",
|
||||
"fallback_used": False,
|
||||
"fallback_reason": None,
|
||||
"missing_reason": fallback_reason or "player-search-index-unavailable",
|
||||
},
|
||||
}
|
||||
return runtime_result
|
||||
|
||||
|
||||
def _search_rcon_materialized_players_runtime(
|
||||
@@ -370,22 +371,12 @@ def get_rcon_materialized_player_stats(
|
||||
if read_model_result is not None:
|
||||
return read_model_result
|
||||
|
||||
runtime_result = _get_rcon_materialized_player_stats_runtime(
|
||||
return _build_missing_player_period_stats_result(
|
||||
player_id=normalized_player_id,
|
||||
server_id=server_id,
|
||||
timeframe=resolved_timeframe,
|
||||
db_path=db_path,
|
||||
missing_reason=fallback_reason or "player-period-stats-unavailable",
|
||||
)
|
||||
runtime_source = dict(runtime_result.get("source") or {})
|
||||
runtime_source.update(
|
||||
{
|
||||
"read_model": "player-period-stats",
|
||||
"fallback_used": True,
|
||||
"fallback_reason": fallback_reason or "player-period-stats-unavailable",
|
||||
}
|
||||
)
|
||||
runtime_result["source"] = runtime_source
|
||||
return runtime_result
|
||||
|
||||
return _get_rcon_materialized_player_stats_runtime(
|
||||
player_id=normalized_player_id,
|
||||
@@ -488,13 +479,13 @@ def _get_player_period_stats_read_model(
|
||||
timeframe: str,
|
||||
db_path: Path | None = None,
|
||||
) -> tuple[dict[str, object] | None, str | None]:
|
||||
resolved_path = initialize_player_period_stats_storage(db_path=db_path)
|
||||
resolved_path = _resolve_rcon_read_model_path(db_path=db_path)
|
||||
resolved_server_id = _normalize_server_id(server_id)
|
||||
required_periods = sorted({timeframe, "weekly", "monthly"})
|
||||
placeholders = ", ".join(["?"] * len(required_periods))
|
||||
|
||||
try:
|
||||
with _connect_scope(resolved_path, db_path=db_path) as connection:
|
||||
with _connect_read_scope(resolved_path, db_path=db_path) as connection:
|
||||
scope_rows = connection.execute(
|
||||
f"""
|
||||
SELECT period_type, COUNT(*) AS row_count
|
||||
@@ -1085,7 +1076,7 @@ def _search_player_search_index(
|
||||
limit: int = 10,
|
||||
db_path: Path | None = None,
|
||||
) -> tuple[dict[str, object] | None, str | None]:
|
||||
resolved_path = initialize_player_search_index_storage(db_path=db_path)
|
||||
resolved_path = _resolve_rcon_read_model_path(db_path=db_path)
|
||||
resolved_server_id = _normalize_server_id(server_id)
|
||||
normalized_query = query.strip()
|
||||
normalized_name_query = _normalize_player_search_text(normalized_query)
|
||||
@@ -1097,7 +1088,7 @@ def _search_player_search_index(
|
||||
escaped_id_exact = normalized_query.casefold()
|
||||
|
||||
try:
|
||||
with _connect_scope(resolved_path, db_path=db_path) as connection:
|
||||
with _connect_read_scope(resolved_path, db_path=db_path) as connection:
|
||||
has_rows = connection.execute(
|
||||
"""
|
||||
SELECT 1
|
||||
@@ -1399,6 +1390,14 @@ def _connect_scope(resolved_path: Path, *, db_path: Path | None = None):
|
||||
return closing(connect_sqlite_readonly(resolved_path))
|
||||
|
||||
|
||||
def _connect_read_scope(resolved_path: Path, *, db_path: Path | None = None):
|
||||
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
|
||||
from .postgres_rcon_storage import connect_postgres_compat
|
||||
|
||||
return connect_postgres_compat(initialize=False)
|
||||
return closing(connect_sqlite_readonly(resolved_path))
|
||||
|
||||
|
||||
def _connect_write_scope(resolved_path: Path, *, db_path: Path | None = None):
|
||||
if use_postgres_rcon_storage(explicit_sqlite_path=db_path):
|
||||
from .postgres_rcon_storage import connect_postgres_compat
|
||||
@@ -1407,6 +1406,42 @@ def _connect_write_scope(resolved_path: Path, *, db_path: Path | None = None):
|
||||
return connect_sqlite_writer(resolved_path)
|
||||
|
||||
|
||||
def _resolve_rcon_read_model_path(*, db_path: Path | None = None) -> Path:
|
||||
return db_path or get_storage_path()
|
||||
|
||||
|
||||
def _build_missing_player_period_stats_result(
|
||||
*,
|
||||
player_id: str,
|
||||
server_id: str | None,
|
||||
timeframe: str,
|
||||
missing_reason: str,
|
||||
) -> dict[str, object]:
|
||||
resolved_server_id = _normalize_server_id(server_id)
|
||||
return {
|
||||
"player_id": player_id,
|
||||
"server_id": resolved_server_id,
|
||||
"timeframe": timeframe,
|
||||
"player_name": None,
|
||||
"window_start": None,
|
||||
"window_end": None,
|
||||
"window_kind": timeframe,
|
||||
"matches_considered": 0,
|
||||
"kills": 0,
|
||||
"deaths": 0,
|
||||
"teamkills": 0,
|
||||
"weekly_ranking": None,
|
||||
"monthly_ranking": None,
|
||||
"source": {
|
||||
"read_model": "player-period-stats",
|
||||
"status": "unavailable",
|
||||
"fallback_used": False,
|
||||
"fallback_reason": None,
|
||||
"missing_reason": missing_reason,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _normalize_player_search_text(value: object) -> str:
|
||||
text = str(value or "").strip().casefold()
|
||||
if not text:
|
||||
|
||||
@@ -154,7 +154,11 @@ def get_rcon_historical_match_detail(
|
||||
"""Return one RCON competitive window as a match-detail compatible payload."""
|
||||
from .rcon_admin_log_materialization import get_materialized_rcon_match_detail
|
||||
|
||||
materialized = get_materialized_rcon_match_detail(server_key=server_key, match_key=match_id)
|
||||
materialized = get_materialized_rcon_match_detail(
|
||||
server_key=server_key,
|
||||
match_key=match_id,
|
||||
ensure_storage=False,
|
||||
)
|
||||
if materialized is not None:
|
||||
return _build_materialized_detail_item(materialized)
|
||||
|
||||
|
||||
@@ -7,9 +7,16 @@ import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from app.historical_storage import upsert_historical_match
|
||||
from app.payloads import build_recent_historical_matches_payload
|
||||
from app.rcon_historical_player_stats import (
|
||||
get_rcon_materialized_player_stats,
|
||||
initialize_player_period_stats_storage,
|
||||
initialize_player_search_index_storage,
|
||||
search_rcon_materialized_players,
|
||||
)
|
||||
from app.rcon_admin_log_materialization import (
|
||||
get_materialized_rcon_match_detail,
|
||||
materialize_rcon_admin_log,
|
||||
@@ -298,6 +305,85 @@ class RconMaterializationPipelineTests(unittest.TestCase):
|
||||
self.assertEqual(payload["data"]["items"][0]["result_source"], "public-scoreboard-fallback")
|
||||
gc.collect()
|
||||
|
||||
def test_public_player_search_uses_read_model_without_initialize_or_runtime_fallback(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db_path = Path(tmpdir) / "historical.sqlite3"
|
||||
initialize_player_search_index_storage(db_path=db_path)
|
||||
_insert_player_search_index_fixture(db_path)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"app.rcon_historical_player_stats.initialize_player_search_index_storage",
|
||||
side_effect=AssertionError("public read must not initialize player search storage"),
|
||||
),
|
||||
patch(
|
||||
"app.rcon_historical_player_stats._search_rcon_materialized_players_runtime",
|
||||
side_effect=AssertionError("public read must not use runtime player search fallback"),
|
||||
),
|
||||
):
|
||||
payload = search_rcon_materialized_players(
|
||||
query="Medu",
|
||||
server_id="all",
|
||||
limit=10,
|
||||
db_path=db_path,
|
||||
)
|
||||
|
||||
self.assertEqual(payload["source"]["read_model"], "player-search-index")
|
||||
self.assertFalse(payload["source"]["fallback_used"])
|
||||
self.assertEqual(payload["items"][0]["player_id"], "76561198092154180")
|
||||
gc.collect()
|
||||
|
||||
def test_public_player_detail_returns_controlled_empty_without_initialize_or_runtime_fallback(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db_path = Path(tmpdir) / "missing.sqlite3"
|
||||
|
||||
with (
|
||||
patch(
|
||||
"app.rcon_historical_player_stats.initialize_player_period_stats_storage",
|
||||
side_effect=AssertionError("public read must not initialize player period storage"),
|
||||
),
|
||||
patch(
|
||||
"app.rcon_historical_player_stats._get_rcon_materialized_player_stats_runtime",
|
||||
side_effect=AssertionError("public read must not use runtime player detail fallback"),
|
||||
),
|
||||
):
|
||||
payload = get_rcon_materialized_player_stats(
|
||||
player_id="76561198092154180",
|
||||
server_id="all",
|
||||
timeframe="weekly",
|
||||
db_path=db_path,
|
||||
)
|
||||
|
||||
self.assertEqual(payload["player_id"], "76561198092154180")
|
||||
self.assertEqual(payload["matches_considered"], 0)
|
||||
self.assertEqual(payload["source"]["read_model"], "player-period-stats")
|
||||
self.assertEqual(payload["source"]["status"], "unavailable")
|
||||
self.assertFalse(payload["source"]["fallback_used"])
|
||||
gc.collect()
|
||||
|
||||
def test_public_match_detail_read_does_not_initialize_materialized_storage(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
db_path = Path(tmpdir) / "historical.sqlite3"
|
||||
previous_storage_path = os.environ.get("HLL_BACKEND_STORAGE_PATH")
|
||||
os.environ["HLL_BACKEND_STORAGE_PATH"] = str(db_path)
|
||||
try:
|
||||
_persist_admin_log_fixture(db_path)
|
||||
materialize_rcon_admin_log(db_path=db_path)
|
||||
with patch(
|
||||
"app.rcon_admin_log_materialization.initialize_rcon_materialized_storage",
|
||||
side_effect=AssertionError("public match detail read must not initialize storage"),
|
||||
):
|
||||
detail = get_rcon_historical_match_detail(
|
||||
server_key="comunidad-hispana-01",
|
||||
match_id="comunidad-hispana-01:100:500:stmariedumontwarfare",
|
||||
)
|
||||
finally:
|
||||
_restore_env("HLL_BACKEND_STORAGE_PATH", previous_storage_path)
|
||||
|
||||
self.assertIsNotNone(detail)
|
||||
self.assertEqual(detail["result_source"], "admin-log-match-ended")
|
||||
gc.collect()
|
||||
|
||||
def test_safe_scoreboard_match_url_allowlist_for_active_origins(self) -> None:
|
||||
self.assertEqual(
|
||||
resolve_trusted_scoreboard_match_url(
|
||||
@@ -385,6 +471,44 @@ def _persist_scoreboard_match(db_path: Path) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _insert_player_search_index_fixture(db_path: Path) -> None:
|
||||
import sqlite3
|
||||
|
||||
with sqlite3.connect(db_path) as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO player_search_index (
|
||||
server_id,
|
||||
player_id,
|
||||
player_name,
|
||||
normalized_player_name,
|
||||
first_seen_at,
|
||||
last_seen_at,
|
||||
servers_seen,
|
||||
matches_current_year,
|
||||
kills_current_year,
|
||||
deaths_current_year,
|
||||
teamkills_current_year,
|
||||
updated_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
"all-servers",
|
||||
"76561198092154180",
|
||||
"Medu",
|
||||
"medu",
|
||||
"2026-01-01T00:00:00Z",
|
||||
"2026-06-01T00:00:00Z",
|
||||
'["comunidad-hispana-01"]',
|
||||
12,
|
||||
100,
|
||||
50,
|
||||
0,
|
||||
"2026-06-01T00:00:00Z",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _restore_env(name: str, previous_value: str | None) -> None:
|
||||
if previous_value is None:
|
||||
os.environ.pop(name, None)
|
||||
|
||||
@@ -28,6 +28,61 @@ Los fallos criticos actuales no estan en rankings/snapshots ni en el match detai
|
||||
|
||||
Siguiente fix prioritario recomendado: aislar o eliminar inicializacion/fallback runtime de `/api/stats/players/search` y `/api/stats/players/{player_id}`. Despues, corregir `/api/current-match/kills` y `/api/current-match/players`. `/api/historical/matches/detail` debe quedar en cola de hardening para hacerlo estrictamente read-only, pero no aparece como el primer incendio operativo en esta medicion.
|
||||
|
||||
## Estado post-fix TASK-225
|
||||
|
||||
Fecha: 2026-06-10
|
||||
Alcance aplicado: Fase 1, estabilizacion de lecturas publicas no dependientes de RCON live.
|
||||
|
||||
Cambios de codigo aplicados:
|
||||
|
||||
- `/api/stats/players/search`: el builder sigue llamando al mismo helper, pero `search_rcon_materialized_players()` ahora es lectura estricta de `player_search_index`. Ya no inicializa `player_search_index` ni cae a busqueda runtime sobre `rcon_match_player_stats` durante un GET publico. Si el read model no existe o esta vacio, devuelve `items: []`, `source.read_model: player-search-index`, `source.status: unavailable`, `fallback_used: false`.
|
||||
- `/api/stats/players/{player_id}` semanal/mensual: `get_rcon_materialized_player_stats()` usa `player_period_stats` como read model publico. Si falta el read model o el jugador no esta en el periodo, devuelve una respuesta controlada con contadores a cero, `source.read_model: player-period-stats`, `source.status: unavailable`, `fallback_used: false`. El modo `timeframe=all` conserva el camino runtime interno existente.
|
||||
- `/api/historical/matches/detail`: `get_materialized_rcon_match_detail()` acepta `ensure_storage=False` por defecto y la ruta publica lo usa en modo read-only. PostgreSQL puede abrirse sin `initialize_postgres_rcon_storage()` y SQLite se abre en `mode=ro`. Si falta la tabla/read model, la lectura devuelve `None` y el payload publico RCON responde `found: false`, `fallback_used: false`, sin fallback legacy pesado ni inicializacion.
|
||||
- `scripts/audit_public_requests.py`: se anadieron `--filter` y `--player-id` para medir subconjuntos afectados sin lanzar toda la matriz.
|
||||
|
||||
Validacion local de codigo:
|
||||
|
||||
```powershell
|
||||
python -m compileall backend\app
|
||||
python -m py_compile scripts\audit_public_requests.py
|
||||
cd backend
|
||||
python -m unittest tests.test_rcon_materialization_pipeline
|
||||
python -m unittest tests.test_current_match_payload tests.test_rcon_admin_log_storage tests.test_historical_snapshot_refresh
|
||||
```
|
||||
|
||||
Resultado:
|
||||
|
||||
- `compileall`: OK.
|
||||
- `py_compile`: OK.
|
||||
- `tests.test_rcon_materialization_pipeline`: OK, 12 tests. La suite mantiene `ResourceWarning` SQLite ya presentes en tests existentes.
|
||||
- Tests relacionados de current match, admin log storage y historical snapshot refresh: OK, 13 tests.
|
||||
|
||||
Medicion HTTP local:
|
||||
|
||||
- `http://127.0.0.1:8000/health` no estaba disponible desde el host.
|
||||
- Las ejecuciones parciales del script contra `http://127.0.0.1:8000` seleccionaron los probes esperados, pero fallaron por falta de backend local. No miden rendimiento del cambio.
|
||||
|
||||
Comandos de medicion recomendados tras redeploy:
|
||||
|
||||
```powershell
|
||||
python scripts\audit_public_requests.py --base-url https://comunidadhll.devzamode.es --timeout 30 --filter stats-player-search --output tmp\task225_prod_player_search_audit.json
|
||||
python scripts\audit_public_requests.py --base-url https://comunidadhll.devzamode.es --timeout 30 --player-id 76561198092154180 --filter stats-player-profile --output tmp\task225_prod_player_profile_audit.json
|
||||
python scripts\audit_public_requests.py --base-url https://comunidadhll.devzamode.es --timeout 30 --filter historical-match-detail --output tmp\task225_prod_match_detail_audit.json
|
||||
```
|
||||
|
||||
Estado de severidad esperado tras deploy:
|
||||
|
||||
| Endpoint | Severidad TASK-224 | Estado TASK-225 en codigo | Severidad esperada tras deploy |
|
||||
| --- | --- | --- | --- |
|
||||
| `/api/stats/players/search` | CRITICAL | Sin initialize ni fallback runtime pesado en GET publico | OK o WARNING si falta read model |
|
||||
| `/api/stats/players/{player_id}` weekly/monthly | CRITICAL | Sin initialize ni recalculo runtime pesado en GET publico | OK o WARNING si falta read model |
|
||||
| `/api/historical/matches/detail` | WARNING por deuda estatica | Read-only sin `initialize_*`; sin fallback legacy en modo RCON | OK o WARNING si falta read model |
|
||||
| `/api/current-match/kills` | CRITICAL | No abordado en Fase 1 | CRITICAL pendiente |
|
||||
| `/api/current-match/players` | CRITICAL | No abordado en Fase 1 | CRITICAL pendiente |
|
||||
| `/api/servers` | WARNING | No abordado en Fase 1 | WARNING pendiente |
|
||||
|
||||
Siguiente fix prioritario actualizado: Fase 2 de `TASK-225` en task separada o continuacion autorizada, centrada en hardening de `/api/current-match/kills` y `/api/current-match/players` sin cambiar configuracion RCON. La validacion final debe repetirse en el entorno donde RCON este local.
|
||||
|
||||
## Evidencia ejecutada
|
||||
|
||||
Comandos ejecutados:
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
Esta auditoria revisa las rutas publicas que alimentan `ranking`, `stats`, `historico`, `historico-partida`, `partida-actual` e `index`, con foco en latencia percibida por UI y en el cumplimiento de la regla arquitectonica de leer read models publicos propios en PostgreSQL.
|
||||
|
||||
Actualizacion TASK-225, 2026-06-10: la deuda P1 de `stats search` y `stats player profile` fue corregida en codigo para que los GET publicos usen `player_search_index` y `player_period_stats` en modo read-only estricto, sin inicializar storage ni caer a runtime fallback pesado. La medicion HTTP final requiere redeploy. `current-match` sigue pendiente porque depende de RCON live y debe tratarse como hardening/degradacion sin cambiar hosts, puertos ni configuracion RCON.
|
||||
|
||||
Conclusiones principales:
|
||||
|
||||
- El backend de `ranking` ya no muestra el cuello de botella grave del ranking anual. La evidencia mas fuerte es el test `backend/tests/test_annual_ranking_payload.py`, que confirma que la lectura anual en PostgreSQL ya no inicializa storage en request publico.
|
||||
@@ -399,4 +401,3 @@ Puntos concretos:
|
||||
7. `TASK-221-run-explain-analyze-for-public-runtime-and-read-model-queries`
|
||||
Alcance: SQL audit operativa.
|
||||
Objetivo: validar indices y predicados temporales.
|
||||
|
||||
|
||||
@@ -76,6 +76,16 @@ def main() -> int:
|
||||
default=0,
|
||||
help="Optional cap for smoke runs. 0 means all probes.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--filter",
|
||||
default=None,
|
||||
help="Run only probes whose id, kind, context, path or parameters contain this text.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--player-id",
|
||||
default=None,
|
||||
help="Add player-dependent probes for this player id before applying --filter.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
base_url = normalize_base_url(args.base_url)
|
||||
@@ -83,6 +93,10 @@ def main() -> int:
|
||||
output_path = resolve_output_path(args.output)
|
||||
|
||||
specs = build_probe_specs()
|
||||
if args.player_id:
|
||||
specs.extend(build_player_dependent_specs(args.player_id))
|
||||
if args.filter:
|
||||
specs = filter_probe_specs(specs, args.filter)
|
||||
if args.max_probes and args.max_probes > 0:
|
||||
specs = specs[: args.max_probes]
|
||||
|
||||
@@ -394,6 +408,26 @@ def build_player_dependent_specs(player_id: str) -> list[ProbeSpec]:
|
||||
]
|
||||
|
||||
|
||||
def filter_probe_specs(specs: list[ProbeSpec], raw_filter: str) -> list[ProbeSpec]:
|
||||
needle = raw_filter.strip().casefold()
|
||||
if not needle:
|
||||
return specs
|
||||
return [
|
||||
spec
|
||||
for spec in specs
|
||||
if needle
|
||||
in " ".join(
|
||||
(
|
||||
spec.id,
|
||||
spec.kind,
|
||||
spec.context,
|
||||
spec.path,
|
||||
spec.parameters,
|
||||
)
|
||||
).casefold()
|
||||
]
|
||||
|
||||
|
||||
def run_probe(*, base_url: str, spec: ProbeSpec, timeout_seconds: float) -> dict[str, Any]:
|
||||
url = f"{base_url}{spec.path}"
|
||||
started = time.perf_counter()
|
||||
|
||||
Reference in New Issue
Block a user