Metrics Module

The qmlhc.metrics package provides advanced temporal and statistical evaluation utilities for anomaly detection, control stability, and forecast accuracy. It extends the core loss functions by quantifying how early, how stable, and how precise a system response or prediction is.

This module can be applied directly to states or predictions produced by the Core Module (via backend outputs) or the Hypercausal Layer (graph steps), and may be combined with the results from Loss Evaluation.

Architecture Overview

+----------------------+
|    qmlhc.metrics     |
+-----------+----------+
            |
    +-------v--------+     +--------------------+     +----------------------+
    |   anomalies    |     |      control       |     |     forecasting      |
    | (early alarms) |     | (stability/robust) |     | (error & Δ-lag)      |
    +----------------+     +--------------------+     +----------------------+

• anomalies - metrics for early detection and recall-at-lag within sequential data.

• control - indicators of overshoot, settling time, and robustness for response stability.

• forecasting - statistical errors and temporal alignment metrics (MAPE, MASE, ΔLag, RMSE).

Core Contracts

Component	Summary
early_roc_auc(y_true, scores, horizon) (anomalies)	Measures how early anomalies are detected within a future horizon. Returns 0.5 if either positive or negative labels are absent (uninformative baseline).
recall_at_lag(y_true, y_pred, lag) (anomalies)	Computes recall while tolerating detection delays of up to lag timesteps.
overshoot(y_true, y_pred) (control)	Relative excess over the reference maximum.
settling_time(y_true, y_pred, tol) (control)	Number of samples required for stable convergence within a tolerance band. Returns 0 if the entire sequence is already within tolerance.
robustness(y_true, y_pred) (control)	Stability index in (0, 1] inversely proportional to MSE.
mape, mase, delta_lag, rmse (forecasting)	Standard predictive accuracy and alignment metrics.

Integrated Evaluation Example

The following example extends the Loss Evaluation Example by adding metric evaluation from the qmlhc.metrics package. This provides a unified diagnostic combining backend state evolution, loss analysis, and higher-level metrics.

import numpy as np

# --- Backend (reused from core) ---------------------------------------------
# Base configuration identical to previous examples.
from qmlhc.core.backend import BackendConfig, QuantumBackend
from qmlhc.core.registry import register_backend, create_backend

class MyBackend(QuantumBackend):
    """Normalized input and stochastic projection backend."""
    def run(self, params=None):
        x = self._require_input()
        return self._validate_state(x / np.linalg.norm(x))

    def project_future(self, s_t, branches=2):
        s_t = self._validate_state(s_t)
        noise = np.random.normal(scale=0.05, size=(branches, self.output_dim))
        fut = s_t + noise
        return self._validate_branches(fut)

capabilities = {
    "backend_name": "MyBackend",
    "backend_version": "1.0",
    "max_qubits": 0,
    "output_dim": 3,
    "supports_shots": False,
    "min_shots": 0,
    "max_shots": 0,
    "supports_noise": True,
    "supports_batch": False,
    "gradient": "none",
    "notes": "demo backend",
}
register_backend("my-backend", lambda cfg: MyBackend(cfg), capabilities)
backend = create_backend("my-backend", BackendConfig(output_dim=3))

# --- Hypercausal layer ------------------------------------------------------
from qmlhc.hc.node import HCNode
from qmlhc.hc.graph import HCGraph
from qmlhc.hc.policy import MeanPolicy

node = HCNode(backend=backend, policy=MeanPolicy())
graph = HCGraph.chain(names=["N1"], nodes=[node])

# --- Single causal step -----------------------------------------------------
x_t = np.array([1.0, 2.0, 3.0])
s_map, s_hat_map, info_map = graph.step({"N1": x_t}, branches=3)

# --- Loss module integration -------------------------------------------------
from qmlhc.loss.task import MSELoss
from qmlhc.loss.coherence import CoherenceLoss
from qmlhc.loss.consistency import ConsistencyLoss

target = np.array([0.3, 0.5, 0.8])
task_mse = MSELoss()(s_hat_map["N1"], target)
K_branches = info_map["N1"]["branches"]
coh_var = CoherenceLoss(mode="variance")(K_branches)
coh_mad = CoherenceLoss(mode="mad")(K_branches)
prev_like = s_map["N1"]
cons_tri = ConsistencyLoss(alpha=1.0, beta=1.0)(prev_like, s_map["N1"], s_hat_map["N1"])

# --- Metrics Evaluation ------------------------------------------------------
#  NEW SECTION: This extends the previous example by adding advanced metrics
# from qmlhc.metrics. Each function here is designed to complement the loss
# measures with interpretability (timeliness, stability, accuracy).

from qmlhc.metrics.anomalies import early_roc_auc, recall_at_lag
from qmlhc.metrics.control import overshoot, settling_time, robustness
from qmlhc.metrics.forecasting import mape, mase, delta_lag, rmse

# --- (1) Anomaly metrics (NEW) ----------------------------------------------
# Simulate binary ground truth and scores to test timeliness evaluation
y_true_anom = np.array([0, 0, 1, 0, 0, 1, 0], dtype=float)
scores = np.array([0.1, 0.2, 0.9, 0.3, 0.2, 0.8, 0.2], dtype=float)
y_pred_bin = (scores > 0.5).astype(float)

auc_early = early_roc_auc(y_true_anom, scores, horizon=1)
rec_lag = recall_at_lag(y_true_anom, y_pred_bin, lag=1)

# --- (2) Control metrics (NEW) ----------------------------------------------
# Example reference and response sequences to assess stability
ref = np.array([0.0, 0.6, 1.0, 1.0, 1.0], dtype=float)
resp = np.array([0.0, 0.7, 1.1, 0.98, 1.01], dtype=float)

os_rel = overshoot(ref, resp)
t_set = settling_time(ref, resp, tol=0.05)
rob = robustness(ref, resp)

# --- (3) Forecast metrics (NEW) ---------------------------------------------
# Example real and predicted sequences for accuracy and alignment
y_true_seq = np.array([1.0, 1.1, 1.05, 1.2, 1.25], dtype=float)
y_pred_seq = np.array([0.95, 1.08, 1.10, 1.18, 1.23], dtype=float)
y_naive = np.roll(y_true_seq, 1); y_naive[0] = y_true_seq[0]

mape_v = mape(y_true_seq, y_pred_seq)
mase_v = mase(y_true_seq, y_pred_seq, y_naive)
dlag_v = delta_lag(y_true_seq, y_pred_seq)
rmse_v = rmse(y_true_seq, y_pred_seq)

# --- Output Summary ----------------------------------------------------------
print("Input:", x_t)
print("S_t:", s_map["N1"])
print("Ŝ_{t+1} (mean policy):", s_hat_map["N1"])
print(f"Task MSE: {task_mse:.6f}")
print(f"Coherence Var: {coh_var:.6f} | MAD: {coh_mad:.6f}")
print(f"Triadic Consistency: {cons_tri:.6f}")
print("---- Metrics Evaluation ----")
print(f"Early ROC-AUC: {auc_early:.2f} | Recall@Lag: {rec_lag:.2f}")
print(f"Overshoot: {os_rel:.2f} | Settling Time: {t_set} | Robustness: {rob:.2f}")
print(f"MAPE: {mape_v:.2f}% | MASE: {mase_v:.2f} | ΔLag: {dlag_v:.2f} | RMSE: {rmse_v:.4f}")

Expected Output

Input: [1. 2. 3.]
S_t: [0.26726124 0.53452248 0.80178373]
Ŝ_{t+1} (mean policy): [0.29789606 0.57910593 0.79579622]
Task MSE: 0.002093
Coherence Var: 0.001400 | MAD: 0.031374
Triadic Consistency: 0.000987
---- Metrics Evaluation ----
Early ROC-AUC: 0.75 | Recall@Lag: 1.00
Overshoot: 0.10 | Settling Time: 3 | Robustness: 0.97
MAPE: 3.21% | MASE: 0.64 | ΔLag: 0.50 | RMSE: 0.04

Invariants and Conventions

• All metrics accept NumPy arrays and return scalar floats.

• Anomaly metrics quantify timeliness (not accuracy) of detection.

• Control metrics quantify stability and convergence behavior.

• Forecast metrics quantify magnitude and directional alignment.

• Designed to interoperate seamlessly with qmlhc.loss outputs.

Module References

• Anomaly Metrics
• Control Metrics
• Forecasting Metrics