feat: persist and diagnose scoreboard correlation candidates

This commit is contained in:
devRaGonSa
2026-05-21 14:45:49 +02:00
parent 6383d08eba
commit f0820a1da7
13 changed files with 872 additions and 20 deletions

View File

@@ -17,3 +17,4 @@ Date | Task | Duration | Result | Notes
2026-03-19T13:59:33 | worker-cycle | 641.24 sec | success | codex-runner 2026-03-19T13:59:33 | worker-cycle | 641.24 sec | success | codex-runner
2026-05-19T08:39:12 | worker-cycle | 663.71 sec | success | codex-runner 2026-05-19T08:39:12 | worker-cycle | 663.71 sec | success | codex-runner
2026-05-19T17:20:44 | worker-cycle | 2240.22 sec | success | codex-runner 2026-05-19T17:20:44 | worker-cycle | 2240.22 sec | success | codex-runner
2026-05-21T14:43:44 | worker-cycle | 5580.9 sec | success | codex-runner

View File

@@ -1,7 +1,7 @@
--- ---
id: TASK-147 id: TASK-147
title: Persist public scoreboard list matches as RCON candidates title: Persist public scoreboard list matches as RCON candidates
status: pending status: done
type: backend type: backend
team: Backend Senior team: Backend Senior
supporting_teams: supporting_teams:
@@ -157,6 +157,42 @@ outcome.
Document validation, candidate upsert decisions, counters added to the JSON Document validation, candidate upsert decisions, counters added to the JSON
report, and any follow-up task instead of widening this task. report, and any follow-up task instead of widening this task.
- Persisted list payload candidates before detail fetch through a focused
PostgreSQL upsert for `rcon_scoreboard_match_candidates`.
- Built list candidate URLs only from the trusted scoreboard origin catalog and
numeric public match IDs. List rows are accepted only when the selected
historical server slug/base URL/server number and payload `server_number`
agree with the trusted origin.
- Kept detail fetching for the existing `historical_*` match/player enrichment
path. The backfill report now exposes `list_candidates_inserted`,
`list_candidates_updated` and `list_candidates_skipped` independently from
the detail match counters.
- Added focused regression coverage for the Foy list row `1562115`; detail
fetch failure still leaves the list candidate present, and a second run
updates the same candidate key instead of creating a duplicate.
- Validation:
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"` from `backend/`
passed with pre-existing SQLite `ResourceWarning` output in the scoreboard
regression file.
- `python -m app.storage_diagnostics` from `backend/`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
passed; the script reports no product integration tests configured for its
platform-only scope.
- `docker compose --profile advanced up -d --build backend frontend postgres`
- `docker compose exec backend python -m app.scoreboard_candidate_backfill
--server comunidad-hispana-02 --from 2026-05-20T00:00:00Z --to
2026-05-21T23:59:59Z --max-pages 5 --page-size 100`
reported list candidate inserts/updates with no errors.
- Queried PostgreSQL and verified external match id `1562115` exists for
`comunidad-hispana-02` with URL
`https://scoreboard.comunidadhll.es:5443/games/1562115`.
- Invoked `/api/historical/matches/detail` for
`comunidad-hispana-02:1779310451:1779315851:foywarfare`; payload `found`
was true and `match_url` used the expected trusted URL.
- Scope review used `git diff --name-only`; changed implementation files stay
within this task plus this workflow task file.
## Change Budget ## Change Budget
- Prefer fewer than 5 modified files. - Prefer fewer than 5 modified files.

View File

@@ -1,7 +1,7 @@
--- ---
id: TASK-148 id: TASK-148
title: Relink existing RCON matches to scoreboard candidates title: Relink existing RCON matches to scoreboard candidates
status: pending status: done
type: backend type: backend
team: Backend Senior team: Backend Senior
supporting_teams: supporting_teams:
@@ -155,6 +155,47 @@ Document correlation scoring choices, time-window derivation, ambiguity
handling, relink command output, validation, and any follow-up task instead of handling, relink command output, validation, and any follow-up task instead of
expanding scope. expanding scope.
- Added `python -m app.rcon_scoreboard_relink` to scan already-materialized
ended RCON matches against the same trusted candidate resolution used by the
detail read model. The command reports `matches_scanned`,
`candidates_scanned`, `matches_linked`, `matches_skipped_no_candidate`,
`matches_skipped_ambiguous` and `errors` as JSON.
- Kept URL availability dynamic in the read model instead of adding a second
persisted URL column to RCON matches. The new correlation resolution summary
preserves the existing deterministic map/time/duration/score scoring path,
returns explicit low-confidence/no-candidate/ambiguous reasons, and exposes
the selected external candidate only to command callers.
- Reused the existing materialized window derivation for closed-at plus
server-time duration cases so relink and detail payloads evaluate the same
effective correlation window.
- Added a Foy regression for materialized match key
`comunidad-hispana-02:1779310451:1779315851:foywarfare`: relink reports a
safe link and detail returns
`https://scoreboard.comunidadhll.es:5443/games/1562115`.
- Validation:
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"` from `backend/`
passed with existing SQLite `ResourceWarning` output in the scoreboard
regression file.
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
passed; the script reports no product integration tests configured for its
platform-only scope.
- `docker compose --profile advanced up -d --build backend frontend postgres`
- Re-ran `app.scoreboard_candidate_backfill` for the requested May 20-21
window. Candidate refresh succeeded (`list_candidates_updated: 11`), while
detail enrichment returned a partial status because SQLite reported
`database is locked` for detail match `1562104`.
- `docker compose exec backend python -m app.rcon_scoreboard_relink --server
comunidad-hispana-02` reported 19 scanned matches, 13 linked matches, 6
no-candidate skips, 0 ambiguous skips and no errors.
- HTTP detail verification returned `found: true` and the expected Foy URL.
Existing Carentan detail key
`comunidad-hispana-02:1779178461:1779183861:carentanwarfare` also kept a
trusted `match_url`.
- Scope review used `git diff --name-only`; relink changes stay within the
correlation/read-model command path, focused scoreboard tests and this task
file.
## Change Budget ## Change Budget
- Prefer fewer than 5 modified files. - Prefer fewer than 5 modified files.

View File

@@ -1,7 +1,7 @@
--- ---
id: TASK-149 id: TASK-149
title: Add scoreboard correlation diagnostics title: Add scoreboard correlation diagnostics
status: pending status: done
type: backend type: backend
team: Backend Senior team: Backend Senior
supporting_teams: supporting_teams:
@@ -139,6 +139,42 @@ Document diagnostic fields, known Foy output behavior, documentation added,
validation performed, and any follow-up task instead of expanding normal UI validation performed, and any follow-up task instead of expanding normal UI
payloads. payloads.
- Added `python -m app.scoreboard_correlation_diagnostics` for one
materialized RCON match. JSON output includes match key, server, map,
timestamps, duration, score, candidate search window, safe top candidates,
selected candidate and `final_reason`.
- Candidate summaries expose trusted public `match_url` values only. Untrusted
URLs are omitted and receive `unsafe-url`; non-scoring candidates report
`map-or-window-mismatch` where applicable.
- Kept diagnostics out of normal detail and frontend payloads. The command
reuses the materialized correlation window and safe candidate scoring that
relink/detail use.
- Added `docs/scoreboard-correlation-debugging.md` with the missing-button
sequence: candidate backfill, relink scan, diagnostics command and detail
endpoint check.
- Extended focused Foy coverage so diagnostics select external match id
`1562115` with the expected Foy candidate summary.
- Validation:
- `python -m compileall backend/app`
- `python -m unittest discover -s tests -p "*scoreboard*"` from `backend/`
passed with existing SQLite `ResourceWarning` output in the scoreboard
regression file.
- `python -m app.scoreboard_correlation_diagnostics --server
comunidad-hispana-02 --match
comunidad-hispana-02:1779310451:1779315851:foywarfare` from `backend/`
returned `final_reason: linked` and selected candidate `1562115`.
- `python -m app.storage_diagnostics`
- `node --check frontend/assets/js/historico-partida.js`
- `node --check frontend/assets/js/historico.js`
- `node --check frontend/assets/js/historico-recent-live.js`
- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
exited successfully after reporting its platform checks. Its later local
route probe printed a SQLite `database disk image is malformed` traceback
from `historical_storage`; this task did not alter or repair runtime DB
files.
- Scope review used `git diff --name-only`; diagnostics changes stay in the
correlation command/test/doc path and this task file.
## Change Budget ## Change Budget
- Prefer fewer than 5 modified files. - Prefer fewer than 5 modified files.

View File

@@ -668,6 +668,61 @@ def upsert_scoreboard_candidates(
return inserted_or_updated return inserted_or_updated
def upsert_scoreboard_candidate(
*,
server_slug: str,
candidate: Mapping[str, object],
) -> str:
"""Persist one trusted scoreboard correlation candidate and report the upsert path."""
external_match_id = str(candidate.get("external_match_id") or "").strip()
match_url = str(candidate.get("match_url") or "").strip()
if not external_match_id or not match_url:
return "skipped"
initialize_postgres_rcon_storage()
with connect_postgres() as connection:
existing = connection.execute(
"""
SELECT id
FROM rcon_scoreboard_match_candidates
WHERE server_slug = %s AND external_match_id = %s
LIMIT 1
""",
(server_slug, external_match_id),
).fetchone()
connection.execute(
"""
INSERT INTO rcon_scoreboard_match_candidates (
server_slug, external_match_id, started_at, ended_at, map_name,
map_pretty_name, allied_score, axis_score, player_count, match_url
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)
ON CONFLICT(server_slug, external_match_id) DO UPDATE SET
started_at = EXCLUDED.started_at,
ended_at = EXCLUDED.ended_at,
map_name = EXCLUDED.map_name,
map_pretty_name = EXCLUDED.map_pretty_name,
allied_score = EXCLUDED.allied_score,
axis_score = EXCLUDED.axis_score,
player_count = EXCLUDED.player_count,
match_url = EXCLUDED.match_url,
updated_at = CURRENT_TIMESTAMP
""",
(
server_slug,
external_match_id,
candidate.get("started_at"),
candidate.get("ended_at"),
candidate.get("map_name"),
candidate.get("map_pretty_name"),
candidate.get("allied_score"),
candidate.get("axis_score"),
candidate.get("player_count"),
match_url,
),
)
return "updated" if existing else "inserted"
def count_migrated_tables() -> dict[str, int]: def count_migrated_tables() -> dict[str, int]:
table_names = ( table_names = (
"rcon_admin_log_events", "rcon_admin_log_events",

View File

@@ -206,10 +206,9 @@ def get_rcon_historical_match_detail(
def _build_materialized_recent_item(item: dict[str, object]) -> dict[str, object]: def _build_materialized_recent_item(item: dict[str, object]) -> dict[str, object]:
server_slug = item.get("external_server_id") or item.get("target_key")
timestamps = _build_materialized_timestamp_payload(item) timestamps = _build_materialized_timestamp_payload(item)
correlation_window = _build_materialized_scoreboard_correlation_window(item, timestamps)
player_count = _resolve_materialized_player_count(item) player_count = _resolve_materialized_player_count(item)
scoreboard_correlation = build_materialized_scoreboard_correlation_input(item)
return { return {
"server": { "server": {
"slug": item.get("target_key"), "slug": item.get("target_key"),
@@ -247,13 +246,7 @@ def _build_materialized_recent_item(item: dict[str, object]) -> dict[str, object
else SESSION_RESULT_SOURCE else SESSION_RESULT_SOURCE
), ),
"match_url": resolve_rcon_scoreboard_match_url( "match_url": resolve_rcon_scoreboard_match_url(
server_slug=server_slug, **scoreboard_correlation,
map_name=item.get("map_pretty_name") or item.get("map_name"),
started_at=correlation_window["started_at"],
ended_at=correlation_window["ended_at"],
duration_seconds=_calculate_match_duration_seconds(item),
allied_score=item.get("allied_score"),
axis_score=item.get("axis_score"),
), ),
"capabilities": describe_rcon_historical_read_model()["capabilities"], "capabilities": describe_rcon_historical_read_model()["capabilities"],
} }
@@ -387,6 +380,23 @@ def _build_materialized_scoreboard_correlation_window(
} }
def build_materialized_scoreboard_correlation_input(
item: dict[str, object],
) -> dict[str, object]:
"""Build safe candidate correlation inputs for one materialized RCON match."""
timestamps = _build_materialized_timestamp_payload(item)
correlation_window = _build_materialized_scoreboard_correlation_window(item, timestamps)
return {
"server_slug": item.get("external_server_id") or item.get("target_key"),
"map_name": item.get("map_pretty_name") or item.get("map_name"),
"started_at": correlation_window["started_at"],
"ended_at": correlation_window["ended_at"],
"duration_seconds": _calculate_match_duration_seconds(item),
"allied_score": item.get("allied_score"),
"axis_score": item.get("axis_score"),
}
def _merge_recent_items( def _merge_recent_items(
primary_items: list[dict[str, object]], primary_items: list[dict[str, object]],
fallback_items: list[dict[str, object]], fallback_items: list[dict[str, object]],

View File

@@ -30,12 +30,42 @@ def resolve_rcon_scoreboard_match_url(
db_path: Path | None = None, db_path: Path | None = None,
) -> str | None: ) -> str | None:
"""Return a trusted scoreboard URL for an RCON window only on strong evidence.""" """Return a trusted scoreboard URL for an RCON window only on strong evidence."""
resolution = resolve_rcon_scoreboard_correlation(
server_slug=server_slug,
map_name=map_name,
started_at=started_at,
ended_at=ended_at,
duration_seconds=duration_seconds,
player_count=player_count,
peak_players=peak_players,
allied_score=allied_score,
axis_score=axis_score,
db_path=db_path,
)
match_url = resolution.get("match_url")
return str(match_url) if match_url else None
def resolve_rcon_scoreboard_correlation(
*,
server_slug: object,
map_name: object,
started_at: object,
ended_at: object,
duration_seconds: object = None,
player_count: object = None,
peak_players: object = None,
allied_score: object = None,
axis_score: object = None,
db_path: Path | None = None,
) -> dict[str, object]:
"""Return a safe candidate selection summary for one RCON match window."""
normalized_server_slug = str(server_slug or "").strip() normalized_server_slug = str(server_slug or "").strip()
normalized_map = normalize_map_name(map_name) normalized_map = normalize_map_name(map_name)
rcon_start = _parse_timestamp(started_at) rcon_start = _parse_timestamp(started_at)
rcon_end = _parse_timestamp(ended_at) rcon_end = _parse_timestamp(ended_at)
if not normalized_server_slug or not normalized_map or not rcon_start or not rcon_end: if not normalized_server_slug or not normalized_map or not rcon_start or not rcon_end:
return None return {"match_url": None, "candidate_count": 0, "reason": "invalid-rcon-window"}
if rcon_end < rcon_start: if rcon_end < rcon_start:
rcon_start, rcon_end = rcon_end, rcon_start rcon_start, rcon_end = rcon_end, rcon_start
@@ -60,15 +90,127 @@ def resolve_rcon_scoreboard_match_url(
is not None is not None
] ]
if not scored_candidates: if not scored_candidates:
return None return {
"match_url": None,
"candidate_count": len(candidates),
"reason": "no-safe-candidate",
}
scored_candidates.sort(key=lambda item: item["score"], reverse=True) scored_candidates.sort(key=lambda item: item["score"], reverse=True)
best = scored_candidates[0] best = scored_candidates[0]
if int(best["score"]) < MIN_CONFIDENCE_SCORE: if int(best["score"]) < MIN_CONFIDENCE_SCORE:
return None return {
"match_url": None,
"candidate_count": len(candidates),
"reason": "low-confidence",
}
if len(scored_candidates) > 1 and int(scored_candidates[1]["score"]) >= int(best["score"]): if len(scored_candidates) > 1 and int(scored_candidates[1]["score"]) >= int(best["score"]):
return None return {
return str(best["match_url"]) "match_url": None,
"candidate_count": len(candidates),
"reason": "ambiguous-candidate",
}
return {
"match_url": str(best["match_url"]),
"candidate_count": len(candidates),
"reason": "linked",
"selected_candidate": {
"external_match_id": best.get("external_match_id"),
"correlation_score": int(best["score"]),
},
}
def diagnose_rcon_scoreboard_correlation(
*,
server_slug: object,
map_name: object,
started_at: object,
ended_at: object,
duration_seconds: object = None,
player_count: object = None,
peak_players: object = None,
allied_score: object = None,
axis_score: object = None,
db_path: Path | None = None,
) -> dict[str, object]:
"""Describe safe candidate scoring for a single RCON correlation window."""
normalized_server_slug = str(server_slug or "").strip()
normalized_map = normalize_map_name(map_name)
rcon_start = _parse_timestamp(started_at)
rcon_end = _parse_timestamp(ended_at)
if not normalized_server_slug or not normalized_map or not rcon_start or not rcon_end:
return {
"candidate_search_window": {
"started_at": started_at,
"ended_at": ended_at,
"candidate_limit": MAX_CANDIDATES,
},
"candidate_count": 0,
"top_candidates": [],
"selected_candidate": None,
"final_reason": "invalid-rcon-window",
}
if rcon_end < rcon_start:
rcon_start, rcon_end = rcon_end, rcon_start
candidates = _list_persisted_scoreboard_candidates(
server_slug=normalized_server_slug,
db_path=db_path or get_storage_path(),
)
resolution = resolve_rcon_scoreboard_correlation(
server_slug=server_slug,
map_name=map_name,
started_at=started_at,
ended_at=ended_at,
duration_seconds=duration_seconds,
player_count=player_count,
peak_players=peak_players,
allied_score=allied_score,
axis_score=axis_score,
db_path=db_path,
)
summaries = [
_diagnostic_candidate_summary(
candidate,
server_slug=normalized_server_slug,
normalized_map=normalized_map,
rcon_start=rcon_start,
rcon_end=rcon_end,
duration_seconds=_coerce_int(duration_seconds),
player_count=_coerce_int(player_count),
peak_players=_coerce_int(peak_players),
allied_score=_coerce_int(allied_score),
axis_score=_coerce_int(axis_score),
)
for candidate in candidates
]
summaries.sort(
key=lambda item: (
-int(item["correlation_score"] or -1),
str(item.get("external_match_id") or ""),
)
)
selected_id = (
resolution.get("selected_candidate", {}).get("external_match_id")
if isinstance(resolution.get("selected_candidate"), dict)
else None
)
selected_candidate = next(
(item for item in summaries if item.get("external_match_id") == selected_id),
None,
)
return {
"candidate_search_window": {
"started_at": rcon_start.isoformat().replace("+00:00", "Z"),
"ended_at": rcon_end.isoformat().replace("+00:00", "Z"),
"candidate_limit": MAX_CANDIDATES,
},
"candidate_count": len(candidates),
"top_candidates": summaries[:5],
"selected_candidate": selected_candidate,
"final_reason": resolution["reason"],
}
def _list_persisted_scoreboard_candidates( def _list_persisted_scoreboard_candidates(
@@ -221,10 +363,61 @@ def _score_candidate(
return None return None
return { return {
"score": score, "score": score,
"external_match_id": candidate.get("external_match_id"),
"match_url": candidate["match_url"], "match_url": candidate["match_url"],
} }
def _diagnostic_candidate_summary(
candidate: dict[str, object],
*,
server_slug: str,
normalized_map: str,
rcon_start: datetime,
rcon_end: datetime,
duration_seconds: int | None,
player_count: int | None,
peak_players: int | None,
allied_score: int | None,
axis_score: int | None,
) -> dict[str, object]:
match_url = resolve_trusted_scoreboard_match_url(candidate.get("match_url"), server_slug)
safe_candidate = {**candidate, "match_url": match_url} if match_url else None
scored = (
_score_candidate(
safe_candidate,
normalized_map=normalized_map,
rcon_start=rcon_start,
rcon_end=rcon_end,
duration_seconds=duration_seconds,
player_count=player_count,
peak_players=peak_players,
allied_score=allied_score,
axis_score=axis_score,
)
if safe_candidate
else None
)
map_label = candidate.get("map_pretty_name") or candidate.get("map_name")
summary = {
"external_match_id": candidate.get("external_match_id"),
"started_at": candidate.get("started_at"),
"ended_at": candidate.get("ended_at"),
"map": map_label,
"score": {
"allied_score": _coerce_int(candidate.get("allied_score")),
"axis_score": _coerce_int(candidate.get("axis_score")),
},
"match_url": match_url,
"correlation_score": int(scored["score"]) if scored else None,
}
if not match_url:
summary["rejection_reason"] = "unsafe-url"
elif scored is None:
summary["rejection_reason"] = "map-or-window-mismatch"
return summary
def _overlap_seconds( def _overlap_seconds(
first_start: datetime, first_start: datetime,
first_end: datetime, first_end: datetime,

View File

@@ -0,0 +1,78 @@
"""Report safe scoreboard links for existing materialized RCON matches."""
from __future__ import annotations
import argparse
import json
from collections.abc import Iterable
from pathlib import Path
from .rcon_admin_log_materialization import list_materialized_rcon_matches
from .rcon_historical_read_model import build_materialized_scoreboard_correlation_input
from .rcon_scoreboard_correlation import resolve_rcon_scoreboard_correlation
DEFAULT_LIMIT = 500
def relink_materialized_matches(
*,
server_key: str | None = None,
limit: int = DEFAULT_LIMIT,
db_path: Path | None = None,
) -> dict[str, object]:
"""Scan existing matches against trusted candidates used by the detail read model."""
matches = list_materialized_rcon_matches(
target_key=server_key,
only_ended=True,
limit=limit,
db_path=db_path,
)
report: dict[str, object] = {
"matches_scanned": len(matches),
"candidates_scanned": 0,
"matches_linked": 0,
"matches_skipped_no_candidate": 0,
"matches_skipped_ambiguous": 0,
"errors": [],
}
for match in matches:
try:
resolution = resolve_rcon_scoreboard_correlation(
**build_materialized_scoreboard_correlation_input(match),
db_path=db_path,
)
except Exception as exc:
report["errors"].append(
{"match_key": match.get("match_key"), "message": str(exc)}
)
continue
report["candidates_scanned"] += int(resolution.get("candidate_count") or 0)
if resolution.get("match_url"):
report["matches_linked"] += 1
elif resolution.get("reason") == "ambiguous-candidate":
report["matches_skipped_ambiguous"] += 1
else:
report["matches_skipped_no_candidate"] += 1
return report
def main(argv: Iterable[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Resolve trusted scoreboard links for materialized RCON matches."
)
parser.add_argument("--server", dest="server_key")
parser.add_argument("--limit", type=int, default=DEFAULT_LIMIT)
parser.add_argument("--db-path", type=Path, default=None)
args = parser.parse_args(list(argv) if argv is not None else None)
report = relink_materialized_matches(
server_key=args.server_key,
limit=max(1, args.limit),
db_path=args.db_path,
)
print(json.dumps(report, ensure_ascii=False, indent=2))
return 0 if not report["errors"] else 1
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -5,11 +5,17 @@ from __future__ import annotations
import argparse import argparse
import json import json
from datetime import datetime, timezone from datetime import datetime, timezone
from collections.abc import Mapping
from typing import Iterable from typing import Iterable
from .historical_storage import initialize_historical_storage, list_historical_servers, upsert_historical_match from .historical_storage import initialize_historical_storage, list_historical_servers, upsert_historical_match
from .postgres_rcon_storage import upsert_scoreboard_candidate
from .providers.public_scoreboard_provider import PublicScoreboardHistoricalDataSource from .providers.public_scoreboard_provider import PublicScoreboardHistoricalDataSource
from .scoreboard_origins import list_trusted_public_scoreboard_origins from .scoreboard_origins import (
build_trusted_scoreboard_match_url,
get_trusted_public_scoreboard_origin,
list_trusted_public_scoreboard_origins,
)
DEFAULT_MAX_PAGES = 20 DEFAULT_MAX_PAGES = 20
DEFAULT_PAGE_SIZE = 100 DEFAULT_PAGE_SIZE = 100
@@ -34,8 +40,19 @@ def run_backfill(*, server: dict[str, object], start_at: datetime, end_at: datet
provider = PublicScoreboardHistoricalDataSource() provider = PublicScoreboardHistoricalDataSource()
server_slug = str(server["slug"]) server_slug = str(server["slug"])
base_url = str(server["scoreboard_base_url"]) base_url = str(server["scoreboard_base_url"])
counters = {"pages_processed": 0, "candidates_seen": 0, "candidates_inserted": 0, "candidates_updated": 0, "player_rows_inserted": 0, "player_rows_updated": 0} counters = {
"pages_processed": 0,
"candidates_seen": 0,
"list_candidates_inserted": 0,
"list_candidates_updated": 0,
"list_candidates_skipped": 0,
"candidates_inserted": 0,
"candidates_updated": 0,
"player_rows_inserted": 0,
"player_rows_updated": 0,
}
errors: list[dict[str, object]] = [] errors: list[dict[str, object]] = []
skipped_unsafe_urls = 0
stopped_after_window = False stopped_after_window = False
for page in range(1, max_pages + 1): for page in range(1, max_pages + 1):
try: try:
@@ -56,6 +73,27 @@ def run_backfill(*, server: dict[str, object], start_at: datetime, end_at: datet
continue continue
if ref_time and ref_time >= end_at: if ref_time and ref_time >= end_at:
continue continue
candidate = _build_list_candidate(server=server, match=match)
if candidate is None:
counters["list_candidates_skipped"] += 1
skipped_unsafe_urls += int(_list_candidate_url_is_unsafe(server=server, match=match))
else:
try:
outcome = upsert_scoreboard_candidate(
server_slug=server_slug,
candidate=candidate,
)
except Exception as exc:
counters["list_candidates_skipped"] += 1
errors.append(
{
"stage": "upsert_list_scoreboard_candidate",
"match_id": candidate["external_match_id"],
"message": str(exc),
}
)
else:
counters[f"list_candidates_{outcome}"] += 1
match_id = _stringify(match.get("id")) match_id = _stringify(match.get("id"))
if match_id: if match_id:
ids.append(match_id) ids.append(match_id)
@@ -77,7 +115,7 @@ def run_backfill(*, server: dict[str, object], start_at: datetime, end_at: datet
counters["player_rows_updated"] += _coerce_int(delta.get("player_rows_updated")) counters["player_rows_updated"] += _coerce_int(delta.get("player_rows_updated"))
if stopped_after_window: if stopped_after_window:
break break
return {"status": "ok" if not errors else "partial", "server": server_slug, "scoreboard_base_url": base_url, "requested_window": {"from": _format_timestamp(start_at), "to": _format_timestamp(end_at)}, "stopped_after_window": stopped_after_window, "skipped_unsafe_urls": 0, "errors": errors, **counters} return {"status": "ok" if not errors else "partial", "server": server_slug, "scoreboard_base_url": base_url, "requested_window": {"from": _format_timestamp(start_at), "to": _format_timestamp(end_at)}, "stopped_after_window": stopped_after_window, "skipped_unsafe_urls": skipped_unsafe_urls, "errors": errors, **counters}
def build_arg_parser() -> argparse.ArgumentParser: def build_arg_parser() -> argparse.ArgumentParser:
@@ -137,6 +175,65 @@ def _pick_match_timestamp(match: dict[str, object]) -> object:
return None return None
def _build_list_candidate(
*,
server: Mapping[str, object],
match: Mapping[str, object],
) -> dict[str, object] | None:
server_slug = _stringify(server.get("slug"))
external_match_id = _stringify(match.get("id"))
origin = get_trusted_public_scoreboard_origin(server_slug)
map_payload = match.get("map")
result_payload = match.get("result")
if (
origin is None
or not external_match_id
or not external_match_id.isdigit()
or str(server.get("scoreboard_base_url") or "").strip() != origin.base_url
or _coerce_optional_int(server.get("server_number")) != origin.server_number
or _coerce_optional_int(match.get("server_number")) != origin.server_number
or not isinstance(map_payload, Mapping)
or not isinstance(result_payload, Mapping)
):
return None
started_at = _stringify(match.get("start"))
ended_at = _stringify(match.get("end"))
match_url = build_trusted_scoreboard_match_url(
server_slug=server_slug,
external_match_id=external_match_id,
)
if not started_at or not ended_at or not match_url:
return None
return {
"external_match_id": external_match_id,
"started_at": started_at,
"ended_at": ended_at,
"map_name": _stringify(map_payload.get("id") or map_payload.get("name")),
"map_pretty_name": _stringify(map_payload.get("pretty_name")),
"allied_score": _coerce_optional_int(result_payload.get("allied")),
"axis_score": _coerce_optional_int(result_payload.get("axis")),
"player_count": _coerce_optional_int(match.get("player_count")),
"match_url": match_url,
}
def _list_candidate_url_is_unsafe(
*,
server: Mapping[str, object],
match: Mapping[str, object],
) -> bool:
external_match_id = _stringify(match.get("id"))
return bool(
external_match_id
and build_trusted_scoreboard_match_url(
server_slug=server.get("slug"),
external_match_id=external_match_id,
)
is None
)
def _stringify(value: object) -> str | None: def _stringify(value: object) -> str | None:
if value is None: if value is None:
return None return None
@@ -151,5 +248,12 @@ def _coerce_int(value: object) -> int:
return 0 return 0
def _coerce_optional_int(value: object) -> int | None:
try:
return None if value is None else int(value)
except (TypeError, ValueError):
return None
if __name__ == "__main__": if __name__ == "__main__":
raise SystemExit(main()) raise SystemExit(main())

View File

@@ -0,0 +1,83 @@
"""JSON diagnostics for missing materialized RCON scoreboard links."""
from __future__ import annotations
import argparse
import json
from collections.abc import Iterable
from pathlib import Path
from .rcon_admin_log_materialization import get_materialized_rcon_match_detail
from .rcon_historical_read_model import build_materialized_scoreboard_correlation_input
from .rcon_scoreboard_correlation import diagnose_rcon_scoreboard_correlation
def inspect_materialized_match_correlation(
*,
server_key: str,
match_key: str,
db_path: Path | None = None,
) -> dict[str, object]:
"""Return safe scoreboard correlation diagnostics for one materialized match."""
materialized = get_materialized_rcon_match_detail(
server_key=server_key,
match_key=match_key,
db_path=db_path,
)
if materialized is None:
return {
"rcon_match_key": match_key,
"server": server_key,
"candidate_count": 0,
"top_candidates": [],
"selected_candidate": None,
"final_reason": "rcon-match-not-found",
}
match = materialized["match"]
correlation_input = build_materialized_scoreboard_correlation_input(match)
correlation = diagnose_rcon_scoreboard_correlation(
**correlation_input,
db_path=db_path,
)
return {
"rcon_match_key": match.get("match_key"),
"server": match.get("external_server_id") or match.get("target_key"),
"map": match.get("map_pretty_name") or match.get("map_name"),
"started_at": match.get("started_at"),
"ended_at": match.get("ended_at"),
"closed_at": match.get("ended_at") or match.get("started_at"),
"duration_seconds": correlation_input.get("duration_seconds"),
"score": {
"allied_score": match.get("allied_score"),
"axis_score": match.get("axis_score"),
"winner": match.get("winner"),
},
**correlation,
}
def main(argv: Iterable[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Explain scoreboard candidate correlation for one RCON match."
)
parser.add_argument("--server", required=True)
parser.add_argument("--match", dest="match_key", required=True)
parser.add_argument("--db-path", type=Path, default=None)
args = parser.parse_args(list(argv) if argv is not None else None)
print(
json.dumps(
inspect_materialized_match_correlation(
server_key=args.server,
match_key=args.match_key,
db_path=args.db_path,
),
ensure_ascii=False,
indent=2,
)
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -79,3 +79,19 @@ def resolve_trusted_scoreboard_match_url(
if candidate_parts.params or candidate_parts.query or candidate_parts.fragment: if candidate_parts.params or candidate_parts.query or candidate_parts.fragment:
return None return None
return candidate return candidate
def build_trusted_scoreboard_match_url(
*,
server_slug: object,
external_match_id: object,
) -> str | None:
"""Build a trusted scoreboard match URL from one numeric public match id."""
origin = get_trusted_public_scoreboard_origin(server_slug)
match_id = str(external_match_id or "").strip()
if origin is None or not match_id.isdigit():
return None
return resolve_trusted_scoreboard_match_url(
f"{origin.base_url}/games/{match_id}",
origin.slug,
)

View File

@@ -8,7 +8,9 @@ import sqlite3
import tempfile import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from unittest.mock import patch
from app.scoreboard_candidate_backfill import run_backfill
from app.historical_storage import ( from app.historical_storage import (
get_historical_match_detail, get_historical_match_detail,
initialize_historical_storage, initialize_historical_storage,
@@ -19,9 +21,80 @@ from app.rcon_historical_storage import initialize_rcon_historical_storage
from app.rcon_historical_storage import persist_rcon_historical_sample from app.rcon_historical_storage import persist_rcon_historical_sample
from app.rcon_historical_storage import start_rcon_historical_capture_run from app.rcon_historical_storage import start_rcon_historical_capture_run
from app.rcon_historical_read_model import get_rcon_historical_match_detail from app.rcon_historical_read_model import get_rcon_historical_match_detail
from app.rcon_admin_log_materialization import materialize_rcon_admin_log
from app.rcon_admin_log_storage import persist_rcon_admin_log_entries
from app.rcon_scoreboard_relink import relink_materialized_matches
from app.scoreboard_correlation_diagnostics import inspect_materialized_match_correlation
class PersistedScoreboardMatchLinkTests(unittest.TestCase): class PersistedScoreboardMatchLinkTests(unittest.TestCase):
def test_list_backfill_persists_foy_candidate_before_detail_fetch_failure(self) -> None:
stored: dict[tuple[str, str], dict[str, object]] = {}
class FoyListProvider:
def fetch_match_page(self, *, base_url: str, page: int, limit: int) -> dict[str, object]:
return {"maps": [_foy_list_match()]} if page == 1 else {"maps": []}
def fetch_match_details(
self,
*,
base_url: str,
match_ids: list[str],
max_workers: int,
) -> list[dict[str, object]]:
raise RuntimeError("detail endpoint unavailable")
def fake_upsert(*, server_slug: str, candidate: dict[str, object]) -> str:
key = (server_slug, str(candidate["external_match_id"]))
outcome = "updated" if key in stored else "inserted"
stored[key] = dict(candidate)
return outcome
server = {
"slug": "comunidad-hispana-02",
"scoreboard_base_url": "https://scoreboard.comunidadhll.es:5443",
"server_number": 2,
}
with (
patch("app.scoreboard_candidate_backfill.initialize_historical_storage"),
patch(
"app.scoreboard_candidate_backfill.PublicScoreboardHistoricalDataSource",
return_value=FoyListProvider(),
),
patch(
"app.scoreboard_candidate_backfill.upsert_scoreboard_candidate",
side_effect=fake_upsert,
),
):
first = run_backfill(
server=server,
start_at=_backfill_timestamp("2026-05-20T00:00:00Z"),
end_at=_backfill_timestamp("2026-05-21T23:59:59Z"),
max_pages=2,
page_size=100,
detail_workers=1,
)
second = run_backfill(
server=server,
start_at=_backfill_timestamp("2026-05-20T00:00:00Z"),
end_at=_backfill_timestamp("2026-05-21T23:59:59Z"),
max_pages=2,
page_size=100,
detail_workers=1,
)
candidate = stored[("comunidad-hispana-02", "1562115")]
self.assertEqual(
candidate["match_url"],
"https://scoreboard.comunidadhll.es:5443/games/1562115",
)
self.assertEqual(first["list_candidates_inserted"], 1)
self.assertEqual(first["list_candidates_updated"], 0)
self.assertEqual(first["errors"][0]["stage"], "fetch_match_details")
self.assertEqual(second["list_candidates_inserted"], 0)
self.assertEqual(second["list_candidates_updated"], 1)
self.assertEqual(len(stored), 1)
def test_recent_and_detail_payloads_expose_safe_persisted_match_url(self) -> None: def test_recent_and_detail_payloads_expose_safe_persisted_match_url(self) -> None:
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
db_path = Path(tmpdir) / "historical.sqlite3" db_path = Path(tmpdir) / "historical.sqlite3"
@@ -201,6 +274,70 @@ class PersistedScoreboardMatchLinkTests(unittest.TestCase):
self.assertIsNone(detail["match_url"]) self.assertIsNone(detail["match_url"])
gc.collect() gc.collect()
def test_foy_relink_reports_existing_materialized_match_url(self) -> None:
with tempfile.TemporaryDirectory() as tmpdir:
db_path = Path(tmpdir) / "historical.sqlite3"
previous_storage_path = os.environ.get("HLL_BACKEND_STORAGE_PATH")
os.environ["HLL_BACKEND_STORAGE_PATH"] = str(db_path)
try:
_persist_match(
db_path,
server_slug="comunidad-hispana-02",
match_id="1562115",
map_name="Foy Warfare",
started_at="2026-05-20T20:54:11Z",
ended_at="2026-05-20T22:24:11Z",
)
persist_rcon_admin_log_entries(
target={
"target_key": "comunidad-hispana-02",
"external_server_id": "comunidad-hispana-02",
},
entries=[
{
"timestamp": "2026-05-20T20:54:11Z",
"message": "[1 min (1779310451)] MATCH START Foy Warfare",
},
{
"timestamp": "2026-05-20T22:24:11Z",
"message": "[91 min (1779315851)] MATCH ENDED `Foy Warfare` ALLIED (4 - 1) AXIS",
},
],
db_path=db_path,
)
materialize_rcon_admin_log(db_path=db_path)
report = relink_materialized_matches(
server_key="comunidad-hispana-02",
db_path=db_path,
)
detail = get_rcon_historical_match_detail(
server_key="comunidad-hispana-02",
match_id="comunidad-hispana-02:1779310451:1779315851:foywarfare",
)
diagnostics = inspect_materialized_match_correlation(
server_key="comunidad-hispana-02",
match_key="comunidad-hispana-02:1779310451:1779315851:foywarfare",
db_path=db_path,
)
finally:
if previous_storage_path is None:
os.environ.pop("HLL_BACKEND_STORAGE_PATH", None)
else:
os.environ["HLL_BACKEND_STORAGE_PATH"] = previous_storage_path
self.assertEqual(report["matches_scanned"], 1)
self.assertEqual(report["matches_linked"], 1)
self.assertGreaterEqual(report["candidates_scanned"], 1)
self.assertIsNotNone(detail)
self.assertEqual(
detail["match_url"],
"https://scoreboard.comunidadhll.es:5443/games/1562115",
)
self.assertEqual(diagnostics["final_reason"], "linked")
self.assertEqual(diagnostics["selected_candidate"]["external_match_id"], "1562115")
self.assertEqual(diagnostics["top_candidates"][0]["map"], "Foy Warfare")
gc.collect()
def _persist_match( def _persist_match(
db_path: Path, db_path: Path,
@@ -227,6 +364,23 @@ def _persist_match(
) )
def _foy_list_match() -> dict[str, object]:
return {
"id": 1562115,
"server_number": 2,
"start": "2026-05-20T20:54:11+00:00",
"end": "2026-05-20T22:24:11+00:00",
"map": {"id": "foywarfare", "pretty_name": "Foy Warfare"},
"result": {"allied": 4, "axis": 1},
}
def _backfill_timestamp(raw_value: str):
from app.scoreboard_candidate_backfill import _parse_timestamp
return _parse_timestamp(raw_value, option_name="test")
def _persist_rcon_window( def _persist_rcon_window(
db_path: Path, db_path: Path,
*, *,

View File

@@ -0,0 +1,45 @@
# Scoreboard Correlation Debugging
Use backend commands to debug a missing public scoreboard button on an RCON
historical match. Normal frontend payloads and pages should stay free of
correlation diagnostics.
## Sequence
1. Refresh trusted public scoreboard candidates for the relevant server:
```powershell
docker compose exec backend python -m app.scoreboard_candidate_backfill --server comunidad-hispana-02 --from 2026-05-20T00:00:00Z --to 2026-05-21T23:59:59Z --max-pages 5 --page-size 100
```
2. Scan existing materialized RCON matches against those candidates:
```powershell
docker compose exec backend python -m app.rcon_scoreboard_relink --server comunidad-hispana-02
```
3. Inspect one match correlation:
```powershell
docker compose exec backend python -m app.scoreboard_correlation_diagnostics --server comunidad-hispana-02 --match comunidad-hispana-02:1779310451:1779315851:foywarfare
```
4. Verify the detail endpoint used by the match page:
```powershell
Invoke-WebRequest 'http://localhost:8000/api/historical/matches/detail?server=comunidad-hispana-02&match=comunidad-hispana-02%3A1779310451%3A1779315851%3Afoywarfare' | Select-Object -ExpandProperty Content
```
## Reading Output
The diagnostic JSON includes the RCON match window, score, candidate search
window, safe top candidate summaries, the selected candidate when one is strong
enough, and `final_reason`.
- `linked` means the detail read model can expose the trusted `match_url`.
- `no-safe-candidate` means candidate persistence or map/window matching needs
inspection.
- `low-confidence` means candidates exist but evidence is insufficient.
- `ambiguous-candidate` means two candidates tie and no public URL is selected.
- `unsafe-url` in a candidate summary means the raw candidate URL is not emitted
or selected.