rgonsal/comunidadhll
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Revert "Add scoreboard-backed weekly historical stats"

Browse Source 
This reverts commit fc13715abc.
This commit is contained in:
devRaGonSa
2026-03-20 21:35:11 +01:00
parent fc13715abc
commit 687c7a926d
11 changed files with 0 additions and 1163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

71
backend/README.md
View File

@@ -109,7 +109,6 @@ normaliza espacios y barras finales para mantener la comparacion con el header
- `GET /api/servers/latest`
- `GET /api/servers/history?limit=20`
- `GET /api/servers/{id}/history?limit=20`
- `GET /api/historical/top-kills/weekly?limit=10`
`GET /api/servers` trata el ultimo snapshot persistido como cache local y lo
reutiliza solo si sigue dentro del objetivo de `120` segundos. Si ese snapshot
@@ -398,76 +397,6 @@ consulta historica:
persistido por el colector. El parametro opcional `limit` acepta valores entre
`1` y `100`.
## Proximo frente historico
El siguiente frente backend ya documentado no reutiliza A2S como fuente de
partidas. La base de estadisticas historicas por jugador y por partida parte de
los scoreboards reales de Comunidad Hispana:
- `https://scoreboard.comunidadhll.es/games`
- `https://scoreboard.comunidadhll.es:5443/games`
La definicion del modelo de dominio, identidad, deduplicacion y alcance inicial
queda en `docs/historical-stats-domain-model.md`.
El descubrimiento tecnico realizado sobre el scoreboard publico confirma dos
endpoints JSON utilizables para esta siguiente fase:
- `/api/get_public_info`
- `/api/get_live_game_stats`
`get_public_info` aporta contexto de servidor, mapa y comienzo de la partida
actual. `get_live_game_stats` aporta estadisticas por jugador de la partida en
curso, incluyendo `player_id`, `kills`, `deaths` y `time_seconds`.
## Bootstrap de ingesta historica
El backend incluye una base minima de ingesta historica orientada al primer
caso de uso de rankings semanales:
- `historical_ingestion.py` consulta los 2 scoreboards reales
- `historical_storage.py` persiste partidas, jugadores y estadisticas por
jugador en la misma SQLite local de desarrollo
Ejecucion manual desde `backend/`:
```powershell
python -m app.historical_ingestion
```
Ese comando:
- consulta `get_public_info` y `get_live_game_stats` en ambos scoreboards
- construye una identidad de partida por `server + match_start + map`
- hace upsert de jugadores por `player_id`
- hace upsert de estadisticas por `match + player`, conservando el maximo valor
observado de `kills`, `deaths` y `time_seconds`
Si se quiere validar sin escribir en SQLite:
```powershell
python -m app.historical_ingestion --no-persist
```
## Primera API historica util
Una vez exista al menos una ingesta historica persistida, el backend expone:
- `/api/historical/top-kills/weekly?limit=10`
Opcionalmente acepta:
- `server_id=comunidad-hispana-01`
- `server_id=comunidad-hispana-02`
El payload devuelve:
- rango temporal usado
- servidor
- ranking
- jugador
- kills semanales agregadas
## CORS local minimo
El backend responde con `Access-Control-Allow-Origin` solo si la peticion llega

17
backend/app/config.py
View File

@@ -19,18 +19,6 @@ DEFAULT_ALLOWED_ORIGINS = (
)
DEFAULT_A2S_TARGETS_ENV_VAR = "HLL_BACKEND_A2S_TARGETS"
DEFAULT_A2S_SOURCE_NAME = "community-hispana-a2s"
DEFAULT_HISTORICAL_SCOREBOARD_SOURCES = (
{
"external_server_id": "comunidad-hispana-01",
"display_name": "Comunidad Hispana #01",
"scoreboard_base_url": "https://scoreboard.comunidadhll.es",
},
{
"external_server_id": "comunidad-hispana-02",
"display_name": "Comunidad Hispana #02",
"scoreboard_base_url": "https://scoreboard.comunidadhll.es:5443",
},
)
def get_bind_address() -> tuple[str, int]:
@@ -87,8 +75,3 @@ def get_a2s_targets_payload() -> str | None:
normalized = raw_payload.strip()
return normalized or None
def get_historical_scoreboard_sources() -> tuple[dict[str, str], ...]:
"""Return the real scoreboard sources used for historical player stats."""
return tuple(dict(source) for source in DEFAULT_HISTORICAL_SCOREBOARD_SOURCES)

209
backend/app/historical_ingestion.py
View File

@@ -1,209 +0,0 @@
"""Historical scoreboard ingestion for the real Comunidad Hispana servers."""
from __future__ import annotations
import argparse
import json
from datetime import datetime, timezone
from pathlib import Path
from urllib.request import Request, urlopen
from .config import get_historical_scoreboard_sources
from .historical_storage import persist_historical_capture
PUBLIC_INFO_PATH = "/api/get_public_info"
LIVE_GAME_STATS_PATH = "/api/get_live_game_stats"
DEFAULT_HTTP_TIMEOUT_SECONDS = 10
def collect_historical_stats(
*,
persist: bool = False,
db_path: Path | None = None,
) -> dict[str, object]:
"""Collect the current historical-ready match state from both scoreboards."""
captures: list[dict[str, object]] = []
errors: list[dict[str, object]] = []
for source in get_historical_scoreboard_sources():
try:
capture = collect_server_historical_stats(source)
if persist:
capture["storage"] = persist_historical_capture(capture, db_path=db_path)
captures.append(capture)
except Exception as error: # noqa: BLE001 - keep ingestion failures isolated
errors.append(
{
"external_server_id": source["external_server_id"],
"scoreboard_base_url": source["scoreboard_base_url"],
"message": str(error),
}
)
return {
"status": "ok",
"captures": captures,
"capture_count": len(captures),
"errors": errors,
}
def collect_server_historical_stats(source: dict[str, str]) -> dict[str, object]:
"""Collect one scoreboard snapshot with match and player metrics."""
public_info_payload = _fetch_scoreboard_json(
source["scoreboard_base_url"],
PUBLIC_INFO_PATH,
)
live_game_stats_payload = _fetch_scoreboard_json(
source["scoreboard_base_url"],
LIVE_GAME_STATS_PATH,
)
public_info = dict(public_info_payload.get("result") or {})
live_game_stats = dict(live_game_stats_payload.get("result") or {})
captured_at = _coerce_timestamp(live_game_stats.get("snapshot_timestamp"))
return {
"external_server_id": source["external_server_id"],
"display_name": source["display_name"],
"scoreboard_base_url": source["scoreboard_base_url"],
"captured_at": captured_at,
"server_name": _extract_server_name(public_info, source["display_name"]),
"match": _build_match_record(source, public_info, live_game_stats, captured_at),
"players": _build_player_records(live_game_stats, captured_at),
}
def _fetch_scoreboard_json(base_url: str, path: str) -> dict[str, object]:
request = Request(
f"{base_url}{path}",
headers={
"Accept": "application/json",
"User-Agent": "HLLVietnamBackend/0.1",
},
)
with urlopen(request, timeout=DEFAULT_HTTP_TIMEOUT_SECONDS) as response:
payload = json.loads(response.read().decode("utf-8"))
error = payload.get("error")
if error not in (None, [], [None]):
raise RuntimeError(f"Scoreboard endpoint {path} returned error: {error}")
result = payload.get("result")
if not isinstance(result, dict):
raise RuntimeError(f"Scoreboard endpoint {path} did not return a result object.")
return payload
def _extract_server_name(public_info: dict[str, object], fallback_name: str) -> str:
server_name = public_info.get("name")
if isinstance(server_name, dict):
resolved_name = str(server_name.get("name") or "").strip()
if resolved_name:
return resolved_name
return fallback_name
def _build_match_record(
source: dict[str, str],
public_info: dict[str, object],
live_game_stats: dict[str, object],
captured_at: str,
) -> dict[str, object]:
current_map_container = dict(public_info.get("current_map") or {})
current_map = dict(current_map_container.get("map") or {})
start_epoch = current_map_container.get("start")
started_at = _coerce_timestamp(start_epoch)
match_slug = str(current_map.get("id") or "unknown-match").strip() or "unknown-match"
source_match_ref = f"{source['external_server_id']}:{int(start_epoch or 0)}:{match_slug}"
duration_seconds = None
captured_at_dt = datetime.fromisoformat(captured_at.replace("Z", "+00:00"))
started_at_dt = datetime.fromisoformat(started_at.replace("Z", "+00:00"))
if captured_at_dt >= started_at_dt:
duration_seconds = int((captured_at_dt - started_at_dt).total_seconds())
return {
"source_match_ref": source_match_ref,
"started_at": started_at,
"ended_at": None,
"duration_seconds": duration_seconds,
"map_slug": match_slug,
"map_name": str(current_map.get("pretty_name") or "Mapa no disponible"),
"mode_name": str(current_map.get("game_mode") or "unknown"),
"server_name": _extract_server_name(public_info, source["display_name"]),
}
def _build_player_records(
live_game_stats: dict[str, object],
captured_at: str,
) -> list[dict[str, object]]:
player_rows = live_game_stats.get("stats")
if not isinstance(player_rows, list):
return []
players: list[dict[str, object]] = []
for row in player_rows:
if not isinstance(row, dict):
continue
player_name = str(row.get("player") or "").strip()
player_ref = str(row.get("player_id") or "").strip()
if not player_name or not player_ref:
continue
players.append(
{
"source_player_ref": player_ref,
"canonical_name": player_name,
"last_seen_name": player_name,
"kills": _coerce_int(row.get("kills"), default=0),
"deaths": _coerce_int(row.get("deaths")),
"time_seconds": _coerce_int(row.get("time_seconds")),
"captured_at": captured_at,
}
)
return players
def _coerce_timestamp(value: object) -> str:
if isinstance(value, (int, float)):
resolved = datetime.fromtimestamp(value, tz=timezone.utc)
return resolved.isoformat().replace("+00:00", "Z")
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _coerce_int(value: object, *, default: int | None = None) -> int | None:
if value is None:
return default
try:
return max(0, int(float(value)))
except (TypeError, ValueError):
return default
def main() -> None:
"""Allow manual historical ingestion execution during development."""
parser = argparse.ArgumentParser(
description="Collect scoreboard-backed historical data for Comunidad Hispana.",
)
parser.add_argument(
"--no-persist",
action="store_true",
help="Collect data without persisting it to the local SQLite database.",
)
args = parser.parse_args()
payload = collect_historical_stats(persist=not args.no_persist)
print(json.dumps(payload, indent=2))
if __name__ == "__main__":
main()

326
backend/app/historical_storage.py
View File

@@ -1,326 +0,0 @@
"""Persistence and queries for scoreboard-backed historical player stats."""
from __future__ import annotations
import sqlite3
from datetime import datetime, timedelta, timezone
from pathlib import Path
from .config import get_storage_path
def initialize_historical_storage(*, db_path: Path | None = None) -> Path:
"""Ensure the local database contains the historical scoreboard tables."""
resolved_path = db_path or get_storage_path()
resolved_path.parent.mkdir(parents=True, exist_ok=True)
with _connect(resolved_path) as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS historical_matches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
external_server_id TEXT NOT NULL,
source_url TEXT NOT NULL,
source_match_ref TEXT NOT NULL,
server_name TEXT NOT NULL,
started_at TEXT NOT NULL,
ended_at TEXT,
duration_seconds INTEGER,
map_slug TEXT,
map_name TEXT NOT NULL,
mode_name TEXT,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE (external_server_id, source_match_ref)
);
CREATE TABLE IF NOT EXISTS historical_players (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source_player_ref TEXT NOT NULL UNIQUE,
canonical_name TEXT NOT NULL,
last_seen_name TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS historical_player_match_stats (
id INTEGER PRIMARY KEY AUTOINCREMENT,
match_id INTEGER NOT NULL,
player_id INTEGER NOT NULL,
player_name TEXT NOT NULL,
kills INTEGER NOT NULL DEFAULT 0,
deaths INTEGER,
time_seconds INTEGER,
captured_at TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE (match_id, player_id),
FOREIGN KEY (match_id) REFERENCES historical_matches(id),
FOREIGN KEY (player_id) REFERENCES historical_players(id)
);
CREATE INDEX IF NOT EXISTS idx_historical_matches_server_started_at
ON historical_matches(external_server_id, started_at);
CREATE INDEX IF NOT EXISTS idx_historical_player_match_stats_match_id
ON historical_player_match_stats(match_id);
"""
)
return resolved_path
def persist_historical_capture(
capture: dict[str, object],
*,
db_path: Path | None = None,
) -> dict[str, object]:
"""Persist one scoreboard capture without duplicating match-player rows."""
resolved_path = initialize_historical_storage(db_path=db_path)
match = dict(capture.get("match") or {})
players = list(capture.get("players") or [])
with _connect(resolved_path) as connection:
match_id = _upsert_match(connection, capture, match)
persisted_players = 0
for player in players:
player_id = _upsert_player(connection, player)
_upsert_player_match_stats(connection, match_id, player_id, player)
persisted_players += 1
return {
"db_path": str(resolved_path),
"external_server_id": capture.get("external_server_id"),
"source_match_ref": match.get("source_match_ref"),
"persisted_players": persisted_players,
}
def list_weekly_top_kills(
*,
limit: int = 10,
server_id: str | None = None,
now: datetime | None = None,
db_path: Path | None = None,
) -> dict[str, object]:
"""Return weekly aggregated kills grouped by server and player."""
resolved_path = initialize_historical_storage(db_path=db_path)
resolved_now = now or datetime.now(timezone.utc)
window_start = resolved_now - timedelta(days=7)
clauses = ["historical_matches.started_at >= ?"]
params: list[object] = [window_start.isoformat().replace("+00:00", "Z")]
if server_id:
clauses.append("historical_matches.external_server_id = ?")
params.append(server_id)
where_clause = " AND ".join(clauses)
query = f"""
SELECT
historical_matches.external_server_id,
historical_matches.server_name,
historical_players.source_player_ref AS player_ref,
historical_players.last_seen_name AS player_name,
SUM(historical_player_match_stats.kills) AS weekly_kills
FROM historical_player_match_stats
INNER JOIN historical_matches
ON historical_matches.id = historical_player_match_stats.match_id
INNER JOIN historical_players
ON historical_players.id = historical_player_match_stats.player_id
WHERE {where_clause}
GROUP BY
historical_matches.external_server_id,
historical_matches.server_name,
historical_players.id
ORDER BY
historical_matches.external_server_id ASC,
weekly_kills DESC,
player_name ASC
"""
with _connect(resolved_path) as connection:
rows = connection.execute(query, tuple(params)).fetchall()
grouped_items: list[dict[str, object]] = []
current_server_id = None
current_group: dict[str, object] | None = None
for row in rows:
external_server_id = str(row["external_server_id"])
if external_server_id != current_server_id:
current_server_id = external_server_id
current_group = {
"external_server_id": external_server_id,
"server_name": row["server_name"],
"rankings": [],
}
grouped_items.append(current_group)
assert current_group is not None
rankings = current_group["rankings"]
if len(rankings) >= limit:
continue
rankings.append(
{
"rank": len(rankings) + 1,
"player_id": row["player_ref"],
"player_name": row["player_name"],
"weekly_kills": int(row["weekly_kills"] or 0),
}
)
return {
"window_start": window_start.isoformat().replace("+00:00", "Z"),
"window_end": resolved_now.isoformat().replace("+00:00", "Z"),
"items": grouped_items,
}
def _connect(db_path: Path) -> sqlite3.Connection:
connection = sqlite3.connect(db_path)
connection.row_factory = sqlite3.Row
return connection
def _upsert_match(
connection: sqlite3.Connection,
capture: dict[str, object],
match: dict[str, object],
) -> int:
connection.execute(
"""
INSERT INTO historical_matches (
external_server_id,
source_url,
source_match_ref,
server_name,
started_at,
ended_at,
duration_seconds,
map_slug,
map_name,
mode_name
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(external_server_id, source_match_ref) DO UPDATE SET
server_name = excluded.server_name,
ended_at = excluded.ended_at,
duration_seconds = CASE
WHEN historical_matches.duration_seconds IS NULL THEN excluded.duration_seconds
WHEN excluded.duration_seconds IS NULL THEN historical_matches.duration_seconds
WHEN excluded.duration_seconds > historical_matches.duration_seconds
THEN excluded.duration_seconds
ELSE historical_matches.duration_seconds
END,
map_slug = excluded.map_slug,
map_name = excluded.map_name,
mode_name = excluded.mode_name,
updated_at = CURRENT_TIMESTAMP
""",
(
capture.get("external_server_id"),
capture.get("scoreboard_base_url"),
match.get("source_match_ref"),
match.get("server_name"),
match.get("started_at"),
match.get("ended_at"),
match.get("duration_seconds"),
match.get("map_slug"),
match.get("map_name"),
match.get("mode_name"),
),
)
row = connection.execute(
"""
SELECT id
FROM historical_matches
WHERE external_server_id = ? AND source_match_ref = ?
""",
(capture.get("external_server_id"), match.get("source_match_ref")),
).fetchone()
if row is None:
raise RuntimeError("Failed to resolve historical match id.")
return int(row["id"])
def _upsert_player(connection: sqlite3.Connection, player: dict[str, object]) -> int:
connection.execute(
"""
INSERT INTO historical_players (
source_player_ref,
canonical_name,
last_seen_name
) VALUES (?, ?, ?)
ON CONFLICT(source_player_ref) DO UPDATE SET
canonical_name = excluded.canonical_name,
last_seen_name = excluded.last_seen_name,
updated_at = CURRENT_TIMESTAMP
""",
(
player.get("source_player_ref"),
player.get("canonical_name"),
player.get("last_seen_name"),
),
)
row = connection.execute(
"SELECT id FROM historical_players WHERE source_player_ref = ?",
(player.get("source_player_ref"),),
).fetchone()
if row is None:
raise RuntimeError("Failed to resolve historical player id.")
return int(row["id"])
def _upsert_player_match_stats(
connection: sqlite3.Connection,
match_id: int,
player_id: int,
player: dict[str, object],
) -> None:
connection.execute(
"""
INSERT INTO historical_player_match_stats (
match_id,
player_id,
player_name,
kills,
deaths,
time_seconds,
captured_at
) VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(match_id, player_id) DO UPDATE SET
player_name = excluded.player_name,
kills = CASE
WHEN excluded.kills > historical_player_match_stats.kills
THEN excluded.kills
ELSE historical_player_match_stats.kills
END,
deaths = CASE
WHEN historical_player_match_stats.deaths IS NULL THEN excluded.deaths
WHEN excluded.deaths IS NULL THEN historical_player_match_stats.deaths
WHEN excluded.deaths > historical_player_match_stats.deaths
THEN excluded.deaths
ELSE historical_player_match_stats.deaths
END,
time_seconds = CASE
WHEN historical_player_match_stats.time_seconds IS NULL THEN excluded.time_seconds
WHEN excluded.time_seconds IS NULL THEN historical_player_match_stats.time_seconds
WHEN excluded.time_seconds > historical_player_match_stats.time_seconds
THEN excluded.time_seconds
ELSE historical_player_match_stats.time_seconds
END,
captured_at = excluded.captured_at,
updated_at = CURRENT_TIMESTAMP
""",
(
match_id,
player_id,
player.get("last_seen_name"),
player.get("kills"),
player.get("deaths"),
player.get("time_seconds"),
player.get("captured_at"),
),
)

8
backend/app/routes.py
View File

@@ -15,7 +15,6 @@ from .payloads import (
build_server_latest_payload,
build_servers_payload,
build_trailer_payload,
build_weekly_top_kills_payload,
)
@@ -40,13 +39,6 @@ def resolve_get_payload(path: str) -> tuple[HTTPStatus | None, dict[str, object]
return HTTPStatus.BAD_REQUEST, build_error_payload("Invalid limit parameter")
return HTTPStatus.OK, build_server_history_payload(limit=limit)
if parsed.path == "/api/historical/top-kills/weekly":
limit = _parse_limit(parsed.query)
if limit is None:
return HTTPStatus.BAD_REQUEST, build_error_payload("Invalid limit parameter")
server_id = parse_qs(parsed.query).get("server_id", [None])[0]
return HTTPStatus.OK, build_weekly_top_kills_payload(limit=limit, server_id=server_id)
builder = GET_ROUTES.get(parsed.path)
if builder is None:
if parsed.path.startswith("/api/servers/") and parsed.path.endswith("/history"):