fix: hide pending region and improve rcon freshness

ai/tasks/done/TASK-144-home-region-and-rcon-freshness.md

@@ -0,0 +1,113 @@
+---
+id: TASK-144
+title: Home region cleanup and RCON freshness diagnostics
+status: done
+type: integration
+team: Backend Senior
+supporting_teams: [Frontend Senior]
+roadmap_item: foundation
+priority: high
+---
+
+# TASK-144 - Home region cleanup and RCON freshness diagnostics
+
+## Goal
+
+Hide placeholder region values on the home page server cards and improve RCON historical freshness diagnostics so stale recent matches are explainable from worker/runner logs.
+
+## Context
+
+The home page currently renders `Region pendiente` when a server region is missing. Recent RCON materialized matches can also appear stale without clear logs showing whether AdminLog events were seen, inserted, duplicated or materialized.
+
+## Steps
+
+1. Inspect home server-card rendering and RCON runner/worker/ingestion/materialization paths.
+2. Hide placeholder/missing region quick facts while preserving map data.
+3. Determine whether historical-runner refreshes RCON AdminLog materialization or whether rcon-historical-worker owns it.
+4. Add minimal logging/summary output for AdminLog ingestion/materialization and latest materialized match freshness.
+5. Validate frontend syntax, backend compile, pipeline scripts, Compose services, logs, manual commands and rendered pages.
+
+## Files to Read First
+
+- `ai/architecture-index.md`
+- `ai/repo-context.md`
+- `ai/orchestrator/backend-senior.md`
+- `ai/orchestrator/frontend-senior.md`
+- `frontend/assets/js/main.js`
+- `backend/app/rcon_historical_worker.py`
+- `backend/app/rcon_admin_log_ingestion.py`
+- `backend/app/rcon_admin_log_materialization.py`
+- `backend/app/historical_runner.py`
+- `docker-compose.yml`
+
+## Expected Files to Modify
+
+- `frontend/assets/js/main.js`
+- `backend/app/rcon_historical_worker.py`
+- `backend/app/rcon_admin_log_ingestion.py`
+- `backend/app/rcon_admin_log_materialization.py`
+- `ai/tasks/done/TASK-144-home-region-and-rcon-freshness.md`
+
+## Constraints
+
+- Do not expose secrets.
+- Do not reintroduce Comunidad Hispana #03.
+- Do not change recent-card visual design.
+- Do not commit runtime DB files.
+- Preserve manual RCON commands.
+
+## Validation
+
+- `python -m compileall backend/app`
+- `node --check frontend/assets/js/main.js`
+- `node --check frontend/assets/js/historico.js`
+- `node --check frontend/assets/js/historico-recent-live.js`
+- `node --check frontend/assets/js/historico-partida.js`
+- `powershell -ExecutionPolicy Bypass -File scripts/run-rcon-data-pipeline-tests.ps1`
+- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
+- `docker compose --profile advanced up -d --build backend frontend historical-runner rcon-historical-worker`
+- `docker compose --profile advanced ps`
+- `docker compose logs --tail=150 historical-runner`
+- `docker compose logs --tail=150 rcon-historical-worker`
+- `docker compose exec backend python -m app.rcon_admin_log_ingestion --minutes 360`
+- `docker compose exec backend python -m app.rcon_admin_log_materialization`
+- `Invoke-WebRequest "http://localhost:8000/health" | Select-Object -ExpandProperty Content`
+- `Invoke-WebRequest "http://localhost:8000/api/historical/recent-matches?server=all-servers&limit=20" | Select-Object -ExpandProperty Content`
+- Browser verification on `index.html?nocache=region`
+- Browser verification on `historico.html?nocache=freshness`
+
+## Outcome
+
+Implemented. Home server cards now omit the Region quick fact when the region is missing or a placeholder such as `Region pendiente`, while preserving the Mapa quick fact. RCON capture now runs AdminLog materialization after AdminLog ingestion and emits freshness diagnostics in the worker result, including event counters, materialized match counters and latest materialized/AdminLog match-end timestamps per configured server.
+
+Findings:
+
+- `historical-runner` already calls the RCON capture path when historical data source is RCON, but the capture path previously ingested AdminLog entries without materializing them into recent matches.
+- `rcon-historical-worker` owns the 10-minute capture loop and now materializes after ingestion, so recent matches can advance without requiring a manual materialization command.
+- The live diagnostics show new recent data after the previously stale `17:38` item. Recent matches now include `comunidad-hispana-01:1779299747:1779305147:carentanwarfare` closed at `2026-05-20T19:26:46.519Z` and `comunidad-hispana-02:1779296626:1779301626:carentanwarfare` closed at `2026-05-20T18:41:59.219Z`.
+
+Validation passed:
+
+- `python -m compileall backend/app`
+- `node --check frontend/assets/js/main.js`
+- `node --check frontend/assets/js/historico.js`
+- `node --check frontend/assets/js/historico-recent-live.js`
+- `node --check frontend/assets/js/historico-partida.js`
+- `powershell -ExecutionPolicy Bypass -File scripts/run-rcon-data-pipeline-tests.ps1`
+- `powershell -ExecutionPolicy Bypass -File scripts/run-integration-tests.ps1`
+- `docker compose --profile advanced up -d --build backend frontend historical-runner rcon-historical-worker`
+- `docker compose --profile advanced ps`
+- `docker compose logs --tail=150 historical-runner`
+- `docker compose logs --tail=150 rcon-historical-worker`
+- `docker compose exec backend python -m app.rcon_admin_log_ingestion --minutes 360`
+- `docker compose exec backend python -m app.rcon_admin_log_materialization`
+- `Invoke-WebRequest "http://localhost:8000/health" | Select-Object -ExpandProperty Content`
+- `Invoke-WebRequest "http://localhost:8000/api/historical/recent-matches?server=all-servers&limit=20" | Select-Object -ExpandProperty Content`
+- Browser verification on `http://localhost:8080/index.html?nocache=region`
+- Browser verification on `http://localhost:8080/historico.html?nocache=freshness`
+
+Notes:
+
+- The RCON pipeline script reports existing SQLite `ResourceWarning` messages from its test harness, but the unittest suites return `OK` and the script reports validation passed.
+- The integration script exits successfully, but the local runtime DB emits an existing `database disk image is malformed` traceback after the pass message. Runtime DB files were not modified for commit.
+- Browser plugin runtime tools were not exposed by tool discovery in this session, so rendered validation used local Chrome/Selenium fallback.

backend/app/historical_runner.py

@@ -73,7 +73,7 @@ def run_periodic_historical_refresh(
                 page_size=page_size,
                 run_number=completed_runs,
             )
-            print(json.dumps({"run": completed_runs, **payload}, indent=2))
+            print(json.dumps({"run": completed_runs, **payload}, indent=2), flush=True)
 
             if max_runs is not None and completed_runs >= max_runs:
                 break
@@ -276,6 +276,10 @@ def _rcon_capture_has_new_useful_data(rcon_capture_result: dict[str, Any]) -> bo
     totals = rcon_capture_result.get("totals")
     if isinstance(totals, dict) and int(totals.get("samples_inserted") or 0) > 0:
         return True
+    if isinstance(totals, dict) and int(totals.get("admin_log_events_inserted") or 0) > 0:
+        return True
+    if isinstance(totals, dict) and int(totals.get("materialized_matches_inserted") or 0) > 0:
+        return True
     targets = rcon_capture_result.get("targets")
     if not isinstance(targets, list):
         return False

backend/app/rcon_admin_log_materialization.py

@@ -126,6 +126,7 @@ def materialize_rcon_admin_log(*, db_path: Path | None = None) -> dict[str, obje
             except sqlite3.Error as error:
                 errors.append(str(error))
 
+    freshness = summarize_rcon_materialization_status(db_path=resolved_path)
     return {
         "matches_seen": matches_seen,
         "matches_materialized": matches_materialized,
@@ -134,6 +135,8 @@ def materialize_rcon_admin_log(*, db_path: Path | None = None) -> dict[str, obje
         "player_stats_materialized": player_stats_materialized,
         "player_stats_updated": player_stats_updated,
         "errors": errors,
+        "latest_materialized_matches": freshness["latest_materialized_matches"],
+        "latest_admin_log_match_end_events": freshness["latest_admin_log_match_end_events"],
     }
 
 
@@ -275,11 +278,56 @@ def summarize_rcon_materialization_status(*, db_path: Path | None = None) -> dic
             ORDER BY target_key ASC, event_count DESC
             """
         ).fetchall()
+        latest_matches = connection.execute(
+            """
+            SELECT
+                target_key,
+                external_server_id,
+                match_key,
+                map_pretty_name,
+                COALESCE(ended_at, started_at) AS closed_at,
+                ended_at,
+                ended_server_time,
+                source_basis,
+                updated_at
+            FROM (
+                SELECT
+                    *,
+                    ROW_NUMBER() OVER (
+                        PARTITION BY target_key
+                        ORDER BY COALESCE(ended_at, started_at) DESC,
+                                 COALESCE(ended_server_time, started_server_time) DESC,
+                                 updated_at DESC
+                    ) AS row_number
+                FROM rcon_materialized_matches
+                WHERE source_basis = ?
+            )
+            WHERE row_number = 1
+            ORDER BY target_key ASC
+            """,
+            (MATCH_RESULT_SOURCE,),
+        ).fetchall()
+        latest_match_end_events = connection.execute(
+            """
+            SELECT
+                target_key,
+                external_server_id,
+                MAX(event_timestamp) AS latest_event_timestamp,
+                MAX(server_time) AS latest_server_time,
+                COUNT(*) AS match_end_events
+            FROM rcon_admin_log_events
+            WHERE event_type = 'match_end'
+            GROUP BY target_key, external_server_id
+            ORDER BY target_key ASC
+            """
+        ).fetchall()
     return {
         "materialized_matches": int(match_count or 0),
         "matches_with_player_stats": int(stats_match_count or 0),
         "server_time_ranges": [dict(row) for row in ranges],
         "event_counts": [dict(row) for row in event_counts],
+        "latest_materialized_matches": [dict(row) for row in latest_matches],
+        "latest_admin_log_match_end_events": [dict(row) for row in latest_match_end_events],
     }
 
 

backend/app/rcon_historical_worker.py

@@ -16,6 +16,7 @@ from .config import (
     get_rcon_request_timeout_seconds,
 )
 from .rcon_admin_log_ingestion import ingest_rcon_admin_logs
+from .rcon_admin_log_materialization import materialize_rcon_admin_log
 from .rcon_client import (
     RconQueryError,
     build_rcon_target_key,
@@ -44,6 +45,8 @@ class RconHistoricalCaptureStats:
     admin_log_events_inserted: int = 0
     admin_log_duplicate_events: int = 0
     admin_log_failed_targets: int = 0
+    materialized_matches_inserted: int = 0
+    materialized_matches_updated: int = 0
 
 
 def run_rcon_historical_capture(
@@ -66,6 +69,7 @@ def run_rcon_historical_capture_unlocked(
     """Capture one prospective RCON sample assuming the shared writer lock is already held."""
     initialize_rcon_historical_storage()
     selected_targets = _select_targets(target_key)
+    selected_target_keys = {build_rcon_target_key(target) for target in selected_targets}
     admin_log_lookback_minutes = get_rcon_admin_log_lookback_minutes()
     captured_at = utc_now().isoformat().replace("+00:00", "Z")
     target_scope = target_key or "all-configured-rcon-targets"
@@ -127,6 +131,14 @@ def run_rcon_historical_capture_unlocked(
                 result=admin_log_result,
             )
 
+        materialization_result = materialize_rcon_admin_log()
+        stats.materialized_matches_inserted = int(
+            materialization_result.get("matches_materialized") or 0
+        )
+        stats.materialized_matches_updated = int(
+            materialization_result.get("matches_updated") or 0
+        )
+
         status = "success" if not errors else ("partial" if items else "failed")
         finalize_rcon_historical_capture_run(
             run_id,
@@ -158,7 +170,12 @@ def run_rcon_historical_capture_unlocked(
         "targets": items,
         "errors": errors,
         "admin_log_errors": admin_log_errors,
-        "storage_status": list_rcon_historical_target_statuses(),
+        "materialization_result": materialization_result,
+        "storage_status": [
+            status
+            for status in list_rcon_historical_target_statuses()
+            if status.get("target_key") in selected_target_keys
+        ],
         "totals": {
             "targets_seen": stats.targets_seen,
             "samples_inserted": stats.samples_inserted,
@@ -168,6 +185,8 @@ def run_rcon_historical_capture_unlocked(
             "admin_log_events_inserted": stats.admin_log_events_inserted,
             "admin_log_duplicate_events": stats.admin_log_duplicate_events,
             "admin_log_failed_targets": stats.admin_log_failed_targets,
+            "materialized_matches_inserted": stats.materialized_matches_inserted,
+            "materialized_matches_updated": stats.materialized_matches_updated,
         },
     }
 
@@ -204,7 +223,7 @@ def run_periodic_rcon_historical_capture(
                 retry_delay_seconds=retry_delay_seconds,
                 target_key=target_key,
             )
-            print(json.dumps({"run": completed_runs, **payload}, indent=2))
+            print(json.dumps({"run": completed_runs, **payload}, indent=2), flush=True)
             if max_runs is not None and completed_runs >= max_runs:
                 break
             time.sleep(interval_seconds)

frontend/assets/js/main.js

@@ -266,16 +266,19 @@ function renderServerStatsCard(server) {
     server.status === "online" ? "server-state--online" : "server-state--offline";
   const isRealSnapshot = isRealLiveSnapshot(server);
   const currentMap = server.current_map || "Sin mapa disponible";
-  const region = server.region || "Region pendiente";
+  const region = normalizeServerRegion(server.region);
   const players = Number.isFinite(server.players) ? server.players : 0;
   const maxPlayers = Number.isFinite(server.max_players) ? server.max_players : 0;
   const actionMarkup = renderServerAction(server);
   const cardVariantClass = isRealSnapshot ? "server-card--real" : "server-card--reference";
   const eyebrowLabel = isRealSnapshot ? "Servidor de comunidad" : "Referencia actual";
-  const quickFacts = renderQuickFacts([
+  const quickFactItems = [
     { label: "Mapa", value: currentMap, valueClassName: "server-card__quickfact-value--map" },
-    { label: "Region", value: region },
+  ];
-  ]);
+  if (region) {
+    quickFactItems.push({ label: "Region", value: region });
+  }
+  const quickFacts = renderQuickFacts(quickFactItems);
 
   return `
     <article class="server-card server-card--stats ${cardVariantClass}">
@@ -299,6 +302,29 @@ function renderServerSections(latestItems) {
   return latestItems.map((server) => renderServerStatsCard(server)).join("");
 }
 
+function normalizeServerRegion(value) {
+  if (typeof value !== "string") {
+    return "";
+  }
+  const trimmedValue = value.trim();
+  if (!trimmedValue) {
+    return "";
+  }
+  const normalizedValue = trimmedValue
+    .normalize("NFD")
+    .replace(/[\u0300-\u036f]/g, "")
+    .toLowerCase();
+  const placeholderValues = new Set([
+    "region pendiente",
+    "pending",
+    "unknown",
+    "desconocida",
+    "no disponible",
+    "n/a",
+  ]);
+  return placeholderValues.has(normalizedValue) ? "" : trimmedValue;
+}
+
 function renderServerAction(server) {
   const historyState = getServerHistoryState(server);
   if (!historyState.available) {