DEQ Engine API

The DEQ engine module provides a package-native convenience interface for single-state and multi-state fixed-point systems. It is inspired by the general DEQ interface style popularized by TorchDEQ, but it uses SILVA package solvers, configuration objects, and diagnostics. The relevant entries are DEQ [4], the general engine lineage [35], and SILVA [1].

For the source-to-package derivation and scope notes, see Method Adaptation Atlas.

Equations

For a single tensor state, the engine solves

\[ z^\star=f_\theta(z^\star,x). \]

For a multi-state system,

\[ s=(z^{(1)},z^{(2)},\dots,z^{(m)}), \qquad s^\star=F_\theta(s^\star,x). \]

pack_state flattens the state tuple/list into one solver vector:

\[ v=P(s) = \operatorname{concat} \left( \operatorname{vec}z^{(1)},\dots,\operatorname{vec}z^{(m)} \right). \]

The packed transition is

\[ \tilde F(v) = P\left(F_\theta(P^{-1}(v),x)\right), \]

and the solver computes

\[ v^\star=\tilde F(v^\star), \qquad s^\star=P^{-1}(v^\star). \]

SILVAVariationalDropout reuses one dropout mask during a fixed-point solve:

\[ \tilde x = x\odot \frac{m}{1-p}, \qquad m_i\sim \operatorname{Bernoulli}(1-p). \]

The mask is reset with reset_silva_deq(model) before a new solve or training step.

Multi-State Run

import torch
from silva_networks import SILVADEQConfig, silva_deq

x = torch.randn(2, 4)
initial = (torch.zeros(2, 6), torch.zeros(2, 3))
left_input = torch.nn.Linear(4, 6)
right_link = torch.nn.Linear(6, 3)

def transition(state):
    left, right = state
    return (
        torch.tanh(left_input(x) + 0.2 * left),
        torch.tanh(right_link(left) + 0.2 * right),
    )

result = silva_deq(
    transition,
    initial,
    config=SILVADEQConfig(forward_max_iter=20, forward_tol=1e-6),
    params=(*left_input.parameters(), *right_link.parameters()),
    tensors=(x,),
    return_result=True,
)

assert result.state[0].shape == (2, 6)
assert result.state[1].shape == (2, 3)
print(result.solver_result.converged, result.solver_result.residual)

Tuple and list states are packed as one coupled vector, so their solver configuration must use anderson_batch_dims=0.

Citation Map

Object family	Cite
DEQ engine interface	SILVA package; TorchDEQ; Deep Equilibrium Models
fixed-point solvers	Anderson, Broyden, Picard, or GMRES according to the solver used
variational dropout in fixed-point solves	SILVA package and DEQ/TorchDEQ lineage when reported as a DEQ-engine practice

Public Objects

Object	Role
SILVADEQConfig	TorchDEQ-style configuration wrapper around package solver settings
SILVADEQEngine	fixed-point engine for tensor or tuple/list state
SILVADEQEngineResult	structured output with unpacked state and solver diagnostics
SILVAVariationalDropout	fixed-mask dropout module for solver calls
silva_deq_config	create SILVADEQConfig
silva_deq_engine	create SILVADEQEngine
silva_deq	solve one state or multi-state fixed point
reset_silva_deq	reset dropout masks in a module tree
pack_state	flatten tensor state structures into one solver vector
unpack_state	restore packed solver vectors into original state structures

API Docs

SILVA DEQ engine utilities for single-state and multi-state systems.

This module provides a compact, package-native counterpart to the general DEQ interface popularized by TorchDEQ. It does not vendor TorchDEQ code. The design keeps the same mathematical contract:

\[ z^\star=f_\theta(z^\star, x), \]

but accepts either one tensor state or a tuple/list of tensor states. The engine uses the package's SolverConfig and fixed_point implementations, so solver choice, damping, tolerance, and iteration budget stay consistent with SILVA layers.

References

• Silva, "SILVA Networks as Structured Implicit Layers and Vector Attractors via Dynamic Interaction Fields", 2026.
• Geng and Kolter, "TorchDEQ: A Library for Deep Equilibrium Models", GitHub repository, 2023.
• Bai, Kolter, and Koltun, "Deep Equilibrium Models", NeurIPS 2019.

SILVADEQConfig dataclass

Configuration for SILVADEQEngine.

Parameters:

Name	Type	Description	Default
forward_solver	Literal['picard', 'anderson', 'broyden']	Solver used by the forward fixed-point solve.	'anderson'
backward_mode	BackwardMode	Gradient estimator: finite unrolling, exact implicit differentiation, or phantom gradients.	'unrolled'
backward_solver	Literal['picard', 'anderson', 'broyden', 'gmres']	Linear/fixed-point method for exact implicit adjoints.	'gmres'
forward_max_iter	int	Maximum forward solver iterations.	40
backward_max_iter	int	Maximum backward linear-solver iterations used by the implicit adjoint.	40
forward_tol	float	Forward residual tolerance.	0.0001
backward_tol	float	Backward residual tolerance.	1e-06
backward_stop_mode	Literal['absolute', 'relative']	Absolute or relative backward residual criterion.	'absolute'
backward_relative_eps	float	Stabilizer used by relative backward residuals.	1e-08
alpha	float	Damping factor for the forward solve.	0.7
history	int	Anderson history size.	5
ridge	float	Anderson ridge term.	0.0001
beta	float	Anderson mixing coefficient.	1.0
eval_factor	float	Multiplier for the forward iteration budget in eval mode.	1.0
track_residuals	bool	Whether to store residuals in SolverResult.	True
reengage	bool	Whether to apply one differentiable transition after the numerical solve. This keeps gradients available when using detached acceleration history.	True
stop_mode	Literal['absolute', 'relative']	Absolute or relative forward stopping criterion.	'absolute'
relative_eps	float	Stabilizer used by relative residuals.	1e-08
anderson_batch_dims	int	Number of independent leading batch dimensions for a single tensor state. Multi-state systems are packed as one coupled vector and therefore require zero.	0
phantom_steps	int	Differentiable refinements for phantom gradients.	1
phantom_tau	float	Damping for phantom-gradient refinements.	1.0
indexing	tuple[int, ...]	One-based forward iterations retained for trajectory losses.	()
return_best	bool	Return the lowest-residual forward state.	False

Source code in src/silva_networks/deq_engine.py

@dataclass(frozen=True)
class SILVADEQConfig:
    """Configuration for `SILVADEQEngine`.

    Args:
        forward_solver: Solver used by the forward fixed-point solve.
        backward_mode: Gradient estimator: finite unrolling, exact implicit
            differentiation, or phantom gradients.
        backward_solver: Linear/fixed-point method for exact implicit adjoints.
        forward_max_iter: Maximum forward solver iterations.
        backward_max_iter: Maximum backward linear-solver iterations used by
            the implicit adjoint.
        forward_tol: Forward residual tolerance.
        backward_tol: Backward residual tolerance.
        backward_stop_mode: Absolute or relative backward residual criterion.
        backward_relative_eps: Stabilizer used by relative backward residuals.
        alpha: Damping factor for the forward solve.
        history: Anderson history size.
        ridge: Anderson ridge term.
        beta: Anderson mixing coefficient.
        eval_factor: Multiplier for the forward iteration budget in eval mode.
        track_residuals: Whether to store residuals in `SolverResult`.
        reengage: Whether to apply one differentiable transition after the
            numerical solve. This keeps gradients available when using detached
            acceleration history.
        stop_mode: Absolute or relative forward stopping criterion.
        relative_eps: Stabilizer used by relative residuals.
        anderson_batch_dims: Number of independent leading batch dimensions for
            a single tensor state. Multi-state systems are packed as one coupled
            vector and therefore require zero.
        phantom_steps: Differentiable refinements for phantom gradients.
        phantom_tau: Damping for phantom-gradient refinements.
        indexing: One-based forward iterations retained for trajectory losses.
        return_best: Return the lowest-residual forward state.
    """

    forward_solver: Literal["picard", "anderson", "broyden"] = "anderson"
    backward_mode: BackwardMode = "unrolled"
    backward_solver: Literal["picard", "anderson", "broyden", "gmres"] = "gmres"
    forward_max_iter: int = 40
    backward_max_iter: int = 40
    forward_tol: float = 1e-4
    backward_tol: float = 1e-6
    backward_stop_mode: Literal["absolute", "relative"] = "absolute"
    backward_relative_eps: float = 1e-8
    alpha: float = 0.7
    history: int = 5
    ridge: float = 1e-4
    beta: float = 1.0
    eval_factor: float = 1.0
    track_residuals: bool = True
    reengage: bool = True
    stop_mode: Literal["absolute", "relative"] = "absolute"
    relative_eps: float = 1e-8
    anderson_batch_dims: int = 0
    phantom_steps: int = 1
    phantom_tau: float = 1.0
    indexing: tuple[int, ...] = ()
    return_best: bool = False

    def __post_init__(self) -> None:
        if self.eval_factor <= 0:
            raise ValueError("eval_factor must be positive")
        self.solver_config(training=True)

    def solver_config(self, *, training: bool = True) -> SolverConfig:
        """Convert to the package's `SolverConfig`."""

        max_iter = self.forward_max_iter
        if not training:
            max_iter = max(1, round(max_iter * self.eval_factor))
        return SolverConfig(
            solver=self.forward_solver,
            max_iter=max_iter,
            tol=self.forward_tol,
            alpha=self.alpha,
            history=self.history,
            ridge=self.ridge,
            beta=self.beta,
            stop_mode=self.stop_mode,
            relative_eps=self.relative_eps,
            anderson_batch_dims=self.anderson_batch_dims,
            track_residuals=self.track_residuals,
            reengage=self.reengage,
            backward_mode=self.backward_mode,
            backward_solver=self.backward_solver,
            backward_max_iter=self.backward_max_iter,
            backward_tol=self.backward_tol,
            backward_stop_mode=self.backward_stop_mode,
            backward_relative_eps=self.backward_relative_eps,
            phantom_steps=self.phantom_steps,
            phantom_tau=self.phantom_tau,
            indexing=self.indexing,
            return_best=self.return_best,
        )

solver_config

solver_config(*, training=True)

Convert to the package's SolverConfig.

Source code in src/silva_networks/deq_engine.py

def solver_config(self, *, training: bool = True) -> SolverConfig:
    """Convert to the package's `SolverConfig`."""

    max_iter = self.forward_max_iter
    if not training:
        max_iter = max(1, round(max_iter * self.eval_factor))
    return SolverConfig(
        solver=self.forward_solver,
        max_iter=max_iter,
        tol=self.forward_tol,
        alpha=self.alpha,
        history=self.history,
        ridge=self.ridge,
        beta=self.beta,
        stop_mode=self.stop_mode,
        relative_eps=self.relative_eps,
        anderson_batch_dims=self.anderson_batch_dims,
        track_residuals=self.track_residuals,
        reengage=self.reengage,
        backward_mode=self.backward_mode,
        backward_solver=self.backward_solver,
        backward_max_iter=self.backward_max_iter,
        backward_tol=self.backward_tol,
        backward_stop_mode=self.backward_stop_mode,
        backward_relative_eps=self.backward_relative_eps,
        phantom_steps=self.phantom_steps,
        phantom_tau=self.phantom_tau,
        indexing=self.indexing,
        return_best=self.return_best,
    )

SILVADEQEngine

Bases: Module

General fixed-point engine for SILVA and DEQ-style modules.

Parameters:

Name	Type	Description	Default
config	SILVADEQConfig | SolverConfig | None	Engine configuration. A SolverConfig may be passed for a direct fixed-point configuration, or SILVADEQConfig for TorchDEQ-style naming.	None

Inputs

transition: Callable mapping a state to a state with the same structure. init_state: Tensor, tuple of tensors, or list of tensors used as the solver initialization.

Output

Equilibrium state, or SILVADEQEngineResult when return_result=True.

Source code in src/silva_networks/deq_engine.py

class SILVADEQEngine(nn.Module):
    """General fixed-point engine for SILVA and DEQ-style modules.

    Args:
        config: Engine configuration. A `SolverConfig` may be passed for a
            direct fixed-point configuration, or `SILVADEQConfig` for
            TorchDEQ-style naming.

    Inputs:
        transition: Callable mapping a state to a state with the same structure.
        init_state: Tensor, tuple of tensors, or list of tensors used as the
            solver initialization.

    Output:
        Equilibrium state, or `SILVADEQEngineResult` when `return_result=True`.
    """

    def __init__(self, config: SILVADEQConfig | SolverConfig | None = None):
        super().__init__()
        self.config = config or SILVADEQConfig()

    def solver_config(self) -> SolverConfig:
        """Return the active `SolverConfig` for the current training mode."""

        if isinstance(self.config, SolverConfig):
            return self.config
        return self.config.solver_config(training=self.training)

    def forward(
        self,
        transition: Callable[[State], State],
        init_state: State,
        *,
        params: Iterable[Tensor] | None = None,
        tensors: Iterable[Tensor] = (),
        return_result: bool = False,
    ):
        specs = _state_specs(init_state)
        is_single_tensor = torch.is_tensor(init_state)
        packed_init = init_state if is_single_tensor else pack_state(init_state)
        config = self.solver_config()
        if not is_single_tensor and config.anderson_batch_dims != 0:
            raise ValueError("anderson_batch_dims must be zero for packed multi-state solves")

        if isinstance(transition, nn.Module):
            reset_silva_deq(transition)
            tracked_params = tuple(transition.parameters()) if params is None else tuple(params)
        else:
            tracked_params = () if params is None else tuple(params)

        def packed_transition(flat_state: Tensor) -> Tensor:
            state = flat_state if is_single_tensor else unpack_state(flat_state, specs)
            next_state = transition(state)
            _validate_state_structure(next_state, specs)
            return next_state if is_single_tensor else pack_state(next_state)

        solver_result = solve_equilibrium(
            packed_transition,
            packed_init,
            config,
            params=tracked_params,
            tensors=tensors,
        )
        state = solver_result.z if is_single_tensor else unpack_state(solver_result.z, specs)
        if return_result:
            return SILVADEQEngineResult(
                state=state,
                solver_result=solver_result,
                info={
                    "num_states": len(specs),
                    "numel": int(packed_init.numel()),
                    "shapes": [tuple(spec.shape) for spec in specs],
                },
            )
        return state

solver_config

solver_config()

Return the active SolverConfig for the current training mode.

Source code in src/silva_networks/deq_engine.py

def solver_config(self) -> SolverConfig:
    """Return the active `SolverConfig` for the current training mode."""

    if isinstance(self.config, SolverConfig):
        return self.config
    return self.config.solver_config(training=self.training)

SILVADEQEngineResult dataclass

Structured output from SILVADEQEngine.

Attributes:

Name	Type	Description
state	State	Equilibrium state with the same structure as the initial state.
solver_result	SolverResult	Underlying solver output on the packed tensor state.
info	dict[str, Any]	Small metadata dictionary containing state shapes and counts.

Source code in src/silva_networks/deq_engine.py

@dataclass
class SILVADEQEngineResult:
    """Structured output from `SILVADEQEngine`.

    Attributes:
        state: Equilibrium state with the same structure as the initial state.
        solver_result: Underlying solver output on the packed tensor state.
        info: Small metadata dictionary containing state shapes and counts.
    """

    state: State
    solver_result: SolverResult
    info: dict[str, Any] = field(default_factory=dict)

SILVAVariationalDropout

Bases: Module

Variational dropout with a mask reused across solver calls.

This module follows the DEQ practice of keeping a fixed dropout mask during a fixed-point solve, avoiding a different random map at every solver step.

Parameters:

Name	Type	Description	Default
dropout	float	Probability of dropping an element.	0.5
channelwise	bool	If true for tensors with at least three dimensions, use a channelwise mask with singleton spatial dimensions.	False

Inputs

x: Tensor of any shape.

Output

Tensor with the same shape as x.

Source code in src/silva_networks/deq_engine.py

class SILVAVariationalDropout(nn.Module):
    """Variational dropout with a mask reused across solver calls.

    This module follows the DEQ practice of keeping a fixed dropout mask during
    a fixed-point solve, avoiding a different random map at every solver step.

    Args:
        dropout: Probability of dropping an element.
        channelwise: If true for tensors with at least three dimensions, use a
            channelwise mask with singleton spatial dimensions.

    Inputs:
        x: Tensor of any shape.

    Output:
        Tensor with the same shape as `x`.
    """

    def __init__(self, dropout: float = 0.5, *, channelwise: bool = False):
        super().__init__()
        if not 0.0 <= dropout < 1.0:
            raise ValueError("dropout must satisfy 0 <= dropout < 1")
        self.dropout = float(dropout)
        self.channelwise = channelwise
        self._mask: Tensor | None = None

    def reset_mask(self) -> None:
        """Clear the stored mask before a new training step or solve."""

        self._mask = None

    def forward(self, x: Tensor) -> Tensor:
        if not self.training or self.dropout == 0.0:
            return x
        mask_shape = self._mask_shape(x)
        if (
            self._mask is None
            or self._mask.shape != mask_shape
            or self._mask.device != x.device
            or self._mask.dtype != x.dtype
        ):
            keep = 1.0 - self.dropout
            self._mask = (
                torch.empty(mask_shape, device=x.device, dtype=x.dtype).bernoulli_(keep) / keep
            )
        return x * self._mask

    def _mask_shape(self, x: Tensor) -> tuple[int, ...]:
        if self.channelwise and x.dim() >= 3:
            return (x.shape[0], x.shape[1], *([1] * (x.dim() - 2)))
        return tuple(x.shape)

reset_mask

reset_mask()

Clear the stored mask before a new training step or solve.

Source code in src/silva_networks/deq_engine.py

def reset_mask(self) -> None:
    """Clear the stored mask before a new training step or solve."""

    self._mask = None

pack_state

pack_state(state)

Flatten a tensor or tensor sequence into one solver vector.

Source code in src/silva_networks/deq_engine.py

def pack_state(state: State) -> Tensor:
    """Flatten a tensor or tensor sequence into one solver vector."""

    if torch.is_tensor(state):
        return state.reshape(-1)
    tensors = _as_tensor_sequence(state)
    if not tensors:
        raise ValueError("state sequence must not be empty")
    return torch.cat([tensor.reshape(-1) for tensor in tensors], dim=0)

reset_silva_deq

reset_silva_deq(model)

Reset variational dropout masks in a module tree.

This is the package-native counterpart to resetting DEQ-specific stochastic layers before a new fixed-point solve.

Source code in src/silva_networks/deq_engine.py

def reset_silva_deq(model: nn.Module) -> None:
    """Reset variational dropout masks in a module tree.

    This is the package-native counterpart to resetting DEQ-specific stochastic
    layers before a new fixed-point solve.
    """

    for module in model.modules():
        if hasattr(module, "reset_mask") and callable(module.reset_mask):
            module.reset_mask()

silva_deq

silva_deq(transition, init_state, *, config=None, params=None, tensors=(), return_result=False)

Solve a single-state or multi-state fixed point.

Parameters:

Name	Type	Description	Default
transition	Callable[[State], State]	Callable mapping the state to the next state. The returned state must have the same tensor structure as init_state.	required
init_state	State	Tensor, tuple of tensors, or list of tensors.	required
config	SILVADEQConfig | SolverConfig | None	Engine or solver configuration.	None
params	Iterable[Tensor] | None	Trainable tensors used by a callable transition. Parameters are inferred automatically when transition is an nn.Module.	None
tensors	Iterable[Tensor]	Differentiable non-state inputs captured by the transition.	()
return_result	bool	Whether to return diagnostics.	False

Returns:

Type	Description
State | SILVADEQEngineResult	Equilibrium state, or SILVADEQEngineResult.

Source code in src/silva_networks/deq_engine.py

def silva_deq(
    transition: Callable[[State], State],
    init_state: State,
    *,
    config: SILVADEQConfig | SolverConfig | None = None,
    params: Iterable[Tensor] | None = None,
    tensors: Iterable[Tensor] = (),
    return_result: bool = False,
) -> State | SILVADEQEngineResult:
    """Solve a single-state or multi-state fixed point.

    Args:
        transition: Callable mapping the state to the next state. The returned
            state must have the same tensor structure as `init_state`.
        init_state: Tensor, tuple of tensors, or list of tensors.
        config: Engine or solver configuration.
        params: Trainable tensors used by a callable transition. Parameters are
            inferred automatically when `transition` is an `nn.Module`.
        tensors: Differentiable non-state inputs captured by the transition.
        return_result: Whether to return diagnostics.

    Returns:
        Equilibrium state, or `SILVADEQEngineResult`.
    """

    return SILVADEQEngine(config)(
        transition,
        init_state,
        params=params,
        tensors=tensors,
        return_result=return_result,
    )

silva_deq_config

silva_deq_config(**kwargs)

Create a SILVADEQConfig from keyword arguments.

Source code in src/silva_networks/deq_engine.py

def silva_deq_config(**kwargs: Any) -> SILVADEQConfig:
    """Create a `SILVADEQConfig` from keyword arguments."""

    return SILVADEQConfig(**kwargs)

silva_deq_engine

silva_deq_engine(config=None)

Create a general SILVA DEQ engine.

Source code in src/silva_networks/deq_engine.py

def silva_deq_engine(config: SILVADEQConfig | SolverConfig | None = None) -> SILVADEQEngine:
    """Create a general SILVA DEQ engine."""

    return SILVADEQEngine(config)

unpack_state

unpack_state(vector, specs)

Unpack a solver vector using _StateSpec metadata.

Source code in src/silva_networks/deq_engine.py

def unpack_state(vector: Tensor, specs: Sequence[_StateSpec]) -> State:
    """Unpack a solver vector using `_StateSpec` metadata."""

    pieces: list[Tensor] = []
    offset = 0
    for spec in specs:
        next_offset = offset + spec.numel
        pieces.append(vector[offset:next_offset].reshape(spec.shape))
        offset = next_offset
    if offset != vector.numel():
        raise ValueError("packed vector has extra entries")
    if len(pieces) == 1 and specs[0].container == "tensor":
        return pieces[0]
    if specs[0].container == "list":
        return pieces
    return tuple(pieces)

Where to Go Next

Question	Page
How does the engine connect to SILVA and optical flow?	DEQ Engine and Optical Flow
Where is a structured state executed?	DEQ Engine Bridge Example
How is the backward system solved?	Implicit Backward Guide