ScipyLeastSquares

class pmrf.optimize.ScipyLeastSquares(ftol: float | None = 1e-08, xtol: float | None = 1e-08, gtol: float | None = 1e-08, options: dict = <factory>, show_progress: bool = True, hyper_minimizer: AbstractBoundedMinimizer | None = None, max_rounds: int = 100)

Bases: AbstractLeastSquaresMinimizer

A bounded least-squares minimiser wrapping SciPy’s scipy.optimize.least_squares().

It uses the Trust Region Reflective method ('trf') and searches box space, given the box’s edges as bounds. It minimises half the squared norm of the whitened residual (ADR-0009), with its Jacobian taken by forward-mode JAX autodiff. On a long, narrow valley, such as a line model whose physical parameters are jointly near-degenerate, it converges where ScipyMinimize with L-BFGS-B can stop at a wrong point.

Given hyperparameters, such as a free noise or kernel hyperparameters of a MAP fit, it alternates. Each round is a least-squares solve with the hyperparameters held, then a hyperparameter step by hyper_minimizer on the full objective with the model parameters held. It stops when a round decreases the full objective by no more than ftol relative to it, or after max_rounds; a round that increases it is discarded. Without hyperparameters it is a single solve. metrics.rounds is the number of rounds kept, metrics.objective the final full objective, metrics.history the full objective after each round kept, and metrics.hyperparameters the final hyperparameters.

Every iterate stays strictly inside the box. A start on a closed edge is moved 1e-6 inward in box space first, as for a raw-space minimiser (ADR-0007): SciPy’s own move is so small that a fit starting with every parameter on a bound cannot leave it.

Nonfinite residuals, Jacobians or attempted coordinates raise FloatingPointError with the evaluation number before SciPy can return a failed start point. Finite nonconvergence keeps SciPy’s usual result.

run(residual_fn: Callable[[PyTree, PyTree, Any], Any], y0: PyTree, args: Any = None, bounds: tuple[PyTree, PyTree] | None = None, max_iter: int = 1024, *, hyper_fn: Callable[[PyTree, PyTree, Any], Any] | None = None, hyper0: PyTree = None, hyper_bounds: tuple[PyTree, PyTree] | None = None, **kwargs) → MinimizeResult

Execute the minimization algorithm.

Parameters:
  • residual_fn (callable) – The residual function residual_fn(y, hyper, args), returning a real 1-D vector. Half its squared norm is minimised over y. hyper is None when there are no hyperparameters.

  • y0 (PyTree) – The initial parameter guess.

  • args (Any) – Args to pass to residual_fn and hyper_fn.

  • bounds (tuple[PyTree, PyTree], optional) – The lower and upper edges of the box to search, shaped like y0. Edges may be infinite.

  • max_iter (int = 1024) – The maximum number of iterations of each solve.

  • hyper_fn (callable, optional) – The full scalar objective hyper_fn(hyper, y, args), minimised over the hyperparameters, or None. Required with hyper0; without it, only the objective reported is affected.

  • hyper0 (PyTree, optional) – The initial hyperparameters, or None if there are none.

  • hyper_bounds (tuple[PyTree, PyTree], optional) – The lower and upper edges of the hyperparameters’ box, shaped like hyper0.

  • **kwargs – Runtime arguments forward to the solver backend.

Returns:

An instance of pmrf.optimize.MinimizeResult, with the final hyperparameters in metrics.hyperparameters when any were given.

Return type:

results

ftol: float | None = 1e-08

Tolerance for termination by the change of the cost function, and of the full objective between rounds.

gtol: float | None = 1e-08

Tolerance for termination by the norm of the gradient.

hyper_minimizer: AbstractBoundedMinimizer | None = None

The minimiser of the hyperparameter step. It must honour bounds. None is ScipyMinimize, showing progress as this minimiser does.

max_rounds: int = 100

The most rounds of an alternating fit.

options: dict

Further keyword arguments to scipy.optimize.least_squares(), such as x_scale.

show_progress: bool = True

Whether to show a progress bar.

xtol: float | None = 1e-08

Tolerance for termination by the change of the independent variables.