Kfac

Kfac#

K-FAC-like Preconditioner (Branch-Factored)#

Kronecker-Factored Approximate Curvature inspired preconditioning using branch-wise covariance factors as a low-cost second-order proxy.

This is a simplified K-FAC-like optimizer tailored to hypercausal state branches: assumes block-diagonal structure across branch groups.

Interface:

  • initialize(params) -> state

  • step_params(model, params, context) -> (new_params, state)

Context:

  • context[“info”][“branches”]: (K x D) matrix of state samples

  • context[‘grads’] or a grad_estimator for parameter gradients

class qmlhc.optim.numpy_optim.kfac.HCKFACLike(lr=0.005, damp=0.001, blocks=4, grad_estimator=None, clip=None, seed=2027)[source]#

Bases: object

Branch-factored curvature preconditioning, simplified K-FAC variant.

initialize(params)[source]#
Return type:

Dict[str, Any]

step_params(model, params, context)[source]#
Return type:

Tuple[Dict[str, Any], Dict[str, Any]]

Contents