SILVA Equilibrium Expansion Atlas¶
This lab places HyperDEQ, JFB, SHINE, monotone splitting, C-DEQ, diffusion equilibria, PIDEQ, and QDEQ on separate architecture axes. It preserves the existing family labs and adds a common experiment contract.
Numbered literature: [1], [4], [38], [48], [51], [59], [64], [74], [75], [87], [88], [89], [90]. Each number opens the complete citation and its primary external source.
In [1]:
Copied!
from pathlib import Path
import sys
root = Path.cwd()
while root != root.parent and not (root / "src" / "silva_networks").exists():
root = root.parent
if not (root / "src" / "silva_networks").exists():
root = Path("/content/silva-networks")
sys.path.insert(0, str(root / "src"))
import matplotlib.pyplot as plt
import torch
plt.rcParams.update({"figure.dpi": 300, "savefig.dpi": 300})
torch.manual_seed(91)
from pathlib import Path
import sys
root = Path.cwd()
while root != root.parent and not (root / "src" / "silva_networks").exists():
root = root.parent
if not (root / "src" / "silva_networks").exists():
root = Path("/content/silva-networks")
sys.path.insert(0, str(root / "src"))
import matplotlib.pyplot as plt
import torch
plt.rcParams.update({"figure.dpi": 300, "savefig.dpi": 300})
torch.manual_seed(91)
Out[1]:
<torch._C.Generator at 0x11ad2cb90>
1. The Complete Experiment Tuple¶
$$ \mathcal E=(\mathcal D,S,H,L,G,\mathcal S_f,\mathcal S_b,\mathcal L,\mathcal M). $$
Two runs reproduce the same method only when data, transition, forward solver, backward rule, objective, and metrics agree. A matching class name alone is not enough.
In [2]:
Copied!
from silva_networks import silva_reproduction_spec
families = [
"silva_hyper_deq",
"silva_consistency_deq",
"diffusion_equilibrium",
"silva_generative_equilibrium_transformer",
"silva_fixed_point_diffusion",
"silva_physics_guided_diffusion_pde",
"silva_physics_informed_equilibrium",
"silva_monotone_operator_equilibrium",
"silva_quantum_deq",
]
for family in families:
spec = silva_reproduction_spec(family)
print(f"{family:43s} refs={spec.paper_refs} datasets={spec.datasets[0]}")
print(" ", spec.equation)
from silva_networks import silva_reproduction_spec
families = [
"silva_hyper_deq",
"silva_consistency_deq",
"diffusion_equilibrium",
"silva_generative_equilibrium_transformer",
"silva_fixed_point_diffusion",
"silva_physics_guided_diffusion_pde",
"silva_physics_informed_equilibrium",
"silva_monotone_operator_equilibrium",
"silva_quantum_deq",
]
for family in families:
spec = silva_reproduction_spec(family)
print(f"{family:43s} refs={spec.paper_refs} datasets={spec.datasets[0]}")
print(" ", spec.equation)
silva_hyper_deq refs=(87,) datasets=WikiText-103 z_0=h_phi(x); alpha_k,beta_k=H_phi(r_(k-m+1:k),x); z_(k+1)=beta_k sum_i alpha_(k,i) f(z_i,x)+(1-beta_k) sum_i alpha_(k,i) z_i silva_consistency_deq refs=(59,) datasets=WikiText-103 g_phi(z_t,t,x)=c_skip(t)z_t+c_out(t)P_phi(z_<=t,t,x) diffusion_equilibrium refs=(38, 49) datasets=declared diffusion noise/sample pairs X_star[0]=noise; X_star[k+1]=D_k(X_star[k], condition) silva_generative_equilibrium_transformer refs=(48,) datasets=offline teacher noise/image pairs Z_star = T_theta(Z_star, injection(noise,label)); image = decoder(Z_star) silva_fixed_point_diffusion refs=(74,) datasets=ImageNet 256x256 latent diffusion z_t_star=F_theta(z_t_star, P(x_t), t); epsilon_hat=Q(z_t_star, x_t, t) silva_physics_guided_diffusion_pde refs=(64,) datasets=Poisson fields u_(t-1)=ProjectBoundary(Smooth(Prior(u_t))-eta grad E_PDE(u_t)+noise_t) silva_physics_informed_equilibrium refs=(51,) datasets=Van der Pol or declared nonlinear IVP z_star(t)=T_theta(z_star(t),t); (I-dT/dz) dz_star/dt=dT/dt silva_monotone_operator_equilibrium refs=(75,) datasets=MNIST 0 in (I-W)z_star-Ux-b+partial f(z_star); W=(1-m)I-A^T A+B-B^T silva_quantum_deq refs=(90,) datasets=MNIST-4 z_star=Measure(U_theta(Encode(z_star+S(x)))); y_hat=Q(z_star)
2. Monotone Splitting Is a Transition Guarantee¶
With
$$ W=(1-m)I-A^\top A+B-B^\top, $$
the equilibrium inclusion is
$$ 0\in(I-W)z-Ux-b+\partial f(z). $$
Forward-backward and Peaceman-Rachford are alternative splitting maps for this same inclusion [[75]].
In [3]:
Copied!
from silva_networks import SILVAMonotoneOperatorEquilibrium, make_monotone_operator_dataset
monotone_data = make_monotone_operator_dataset(samples=10)
for splitting in ("forward_backward", "peaceman_rachford"):
monotone = SILVAMonotoneOperatorEquilibrium(
4,
6,
2,
splitting=splitting,
step_size=0.5,
margin=0.5,
)
result = monotone(monotone_data.inputs, return_result=True)
print(
splitting,
"shape", tuple(result.output.shape),
"certificate", float(result.monotonicity_certificate),
"residual", result.solver_result.residual,
)
from silva_networks import SILVAMonotoneOperatorEquilibrium, make_monotone_operator_dataset
monotone_data = make_monotone_operator_dataset(samples=10)
for splitting in ("forward_backward", "peaceman_rachford"):
monotone = SILVAMonotoneOperatorEquilibrium(
4,
6,
2,
splitting=splitting,
step_size=0.5,
margin=0.5,
)
result = monotone(monotone_data.inputs, return_result=True)
print(
splitting,
"shape", tuple(result.output.shape),
"certificate", float(result.monotonicity_certificate),
"residual", result.solver_result.residual,
)
forward_backward shape (10, 2) certificate 0.5005146265029907 residual 7.432677762153617e-07 peaceman_rachford shape (10, 2) certificate 0.5011987686157227 residual 1.1107613318017684e-06
3. PIDEQ Places Physics on the Implicit Prediction¶
For $z^\star(t)=T_\theta(z^\star(t),t)$,
$$ (I-J_zT_\theta)\frac{dz^\star}{dt}=\partial_tT_\theta. $$
The physical residual compares the readout derivative to a declared dynamics callable. The transition and the physical law are separate public objects [[51]].
In [4]:
Copied!
from silva_networks import SILVAPhysicsInformedEquilibrium, SolverConfig
times = torch.linspace(0, 1, 8)[:, None]
pideq = SILVAPhysicsInformedEquilibrium(
3,
1,
config=SolverConfig(
solver="picard",
max_iter=12,
tol=1e-6,
anderson_batch_dims=1,
backward_mode="jfb",
),
)
physics = pideq.physics_loss(
times,
lambda time, state: -0.5 * state,
initial_time=times[:1],
initial_state=torch.ones(1, 1),
jacobian_weight=0.01,
)
print("prediction:", physics.prediction.shape)
print("time derivative:", physics.time_derivative.shape)
print("initial term:", float(physics.initial.detach()))
print("physics residual:", float(physics.residual.detach()))
print("Jacobian term:", float(physics.jacobian.detach()))
from silva_networks import SILVAPhysicsInformedEquilibrium, SolverConfig
times = torch.linspace(0, 1, 8)[:, None]
pideq = SILVAPhysicsInformedEquilibrium(
3,
1,
config=SolverConfig(
solver="picard",
max_iter=12,
tol=1e-6,
anderson_batch_dims=1,
backward_mode="jfb",
),
)
physics = pideq.physics_loss(
times,
lambda time, state: -0.5 * state,
initial_time=times[:1],
initial_state=torch.ones(1, 1),
jacobian_weight=0.01,
)
print("prediction:", physics.prediction.shape)
print("time derivative:", physics.time_derivative.shape)
print("initial term:", float(physics.initial.detach()))
print("physics residual:", float(physics.residual.detach()))
print("Jacobian term:", float(physics.jacobian.detach()))
prediction: torch.Size([8, 1]) time derivative: torch.Size([8, 1]) initial term: 0.18671777844429016 physics residual: 0.003367721103131771 Jacobian term: 0.06744172424077988
4. Diffusion Has Four Different Equilibrium Placements¶
| Family | Equilibrium variable |
|---|---|
| DEQ-DDIM | the complete deterministic diffusion trajectory [[38]] |
| GET | a one-time-injected generative token state [[48]] |
| fixed-point diffusion | a denoiser state at each timestep [[74]] |
| physics-guided diffusion PDE | reverse field state guided by residual energy [[64]] |
The placement determines the state shape, solver call count, loss, and metric.
In [5]:
Copied!
from torch import nn
from silva_networks import SILVADiffusionEquilibrium
class ZeroDenoiser(nn.Module):
def forward(self, value, timestep):
return torch.zeros_like(value)
alphas = torch.linspace(0.95, 0.5, 10)
joint = SILVADiffusionEquilibrium(
ZeroDenoiser(),
alphas,
(9, 6, 3, 0),
eta=0.0,
config=SolverConfig(max_iter=5, tol=1e-8),
)
noise = torch.randn(2, 1, 4, 4)
joint_result = joint(noise, return_result=True)
print("joint trajectory:", joint_result.trajectory.shape)
print("joint output:", joint_result.output.shape)
print("joint residual:", joint_result.solver_result.residual)
from torch import nn
from silva_networks import SILVADiffusionEquilibrium
class ZeroDenoiser(nn.Module):
def forward(self, value, timestep):
return torch.zeros_like(value)
alphas = torch.linspace(0.95, 0.5, 10)
joint = SILVADiffusionEquilibrium(
ZeroDenoiser(),
alphas,
(9, 6, 3, 0),
eta=0.0,
config=SolverConfig(max_iter=5, tol=1e-8),
)
noise = torch.randn(2, 1, 4, 4)
joint_result = joint(noise, return_result=True)
print("joint trajectory:", joint_result.trajectory.shape)
print("joint output:", joint_result.output.shape)
print("joint residual:", joint_result.solver_result.residual)
joint trajectory: torch.Size([4, 2, 1, 4, 4]) joint output: torch.Size([2, 1, 4, 4]) joint residual: 0.0
In [6]:
Copied!
labels = ["monotone", "PIDEQ physics", "joint DDIM"]
values = [
max(result.solver_result.residual, 1e-12),
max(float(physics.residual.detach()), 1e-12),
max(joint_result.solver_result.residual, 1e-12),
]
fig, ax = plt.subplots(figsize=(6.4, 3.4))
ax.bar(labels, values, color=["#2563eb", "#d97706", "#0f766e"])
ax.set_yscale("log")
ax.set(ylabel="diagnostic magnitude", title="three equilibrium placements")
fig.tight_layout()
plt.show()
labels = ["monotone", "PIDEQ physics", "joint DDIM"]
values = [
max(result.solver_result.residual, 1e-12),
max(float(physics.residual.detach()), 1e-12),
max(joint_result.solver_result.residual, 1e-12),
]
fig, ax = plt.subplots(figsize=(6.4, 3.4))
ax.bar(labels, values, color=["#2563eb", "#d97706", "#0f766e"])
ax.set_yscale("log")
ax.set(ylabel="diagnostic magnitude", title="three equilibrium placements")
fig.tight_layout()
plt.show()
5. Acceleration and Backward Rules Can Cross Families¶
HyperDEQ and C-DEQ modify forward evaluation. JFB and SHINE modify backward evaluation. They can be paired with Fourier, graph, physics-informed, multiscale, diffusion, or circuit transitions after a compact convergence and gradient check.
In [7]:
Copied!
from silva_networks import SolverConfig
configs = {
"exact": SolverConfig(backward_mode="implicit", backward_solver="gmres"),
"JFB": SolverConfig(backward_mode="jfb"),
"SHINE": SolverConfig(
solver="broyden", backward_mode="shine", shine_refine_steps=2
),
"phantom": SolverConfig(backward_mode="phantom", phantom_steps=3),
}
for name, config in configs.items():
print(name, "forward", config.solver, "backward", config.backward_mode)
from silva_networks import SolverConfig
configs = {
"exact": SolverConfig(backward_mode="implicit", backward_solver="gmres"),
"JFB": SolverConfig(backward_mode="jfb"),
"SHINE": SolverConfig(
solver="broyden", backward_mode="shine", shine_refine_steps=2
),
"phantom": SolverConfig(backward_mode="phantom", phantom_steps=3),
}
for name, config in configs.items():
print(name, "forward", config.solver, "backward", config.backward_mode)
exact forward picard backward implicit JFB forward picard backward jfb SHINE forward broyden backward shine phantom forward picard backward phantom
6. What a Complete Result Must Contain¶
Record the family and constructor, all replaceable modules, forward and backward configurations, data source/split/preprocessing, objective terms, optimizer and schedule, task metric, normalized residual, iteration counts, runtime, memory, failure count, article, and research repository.
Compact results validate equations, shapes, gradients, and diagnostics. Article reproduction additionally requires the source data, task-scale architecture, training budget, and evaluation protocol.
From 51 Equilibrium Expansion Atlas to a Custom SILVA Family¶
The construction in this notebook can be separated into the universal conditioned-equilibrium contract
$$ z_0=I_\eta(x),\qquad z^\star=T_\theta(z^\star,x),\qquad \widehat y=Q_\psi(z^\star). $$
For this topic:
| Part | Concrete interpretation |
|---|---|
| Equilibrium state | the tensor solved to equilibrium |
| Condition | the observed input or source tensor |
| Repeated computation | the state-preserving transition evaluated by the root solver |
| Required invariants | shape, device, dtype, finiteness, and differentiability |
| Replaceable components | initializer, source encoder, transition, readout, and solver |
The initializer and source path are evaluated outside or alongside the root solve. Only the state-preserving transition is repeated. Replacing an internal architecture does not change this equation, provided the transition still maps the same state space into itself.
In [8]:
Copied!
import torch as silva_extension_torch
from torch import nn as silva_extension_nn
from silva_networks import (
SILVAConditionedEquilibrium,
SILVAZeroInitializer,
SolverConfig,
validate_silva_transition,
)
class NotebookExtensionTransition(silva_extension_nn.Module):
def __init__(self, condition_dim=2, state_dim=3):
super().__init__()
self.source = silva_extension_nn.Linear(condition_dim, state_dim)
self.state_field = silva_extension_nn.Sequential(
silva_extension_nn.Linear(state_dim, 2 * state_dim),
silva_extension_nn.Tanh(),
silva_extension_nn.Linear(2 * state_dim, state_dim),
)
def forward(self, state, condition):
return silva_extension_torch.tanh(
self.source(condition) + 0.15 * self.state_field(state)
)
silva_extension_torch.manual_seed(610)
notebook_condition = silva_extension_torch.linspace(-1.0, 1.0, 8).reshape(4, 2)
notebook_state0 = silva_extension_torch.zeros(4, 3)
notebook_transition = NotebookExtensionTransition()
notebook_report = validate_silva_transition(
notebook_transition,
notebook_state0,
notebook_condition,
)
assert notebook_report.valid
with silva_extension_torch.no_grad():
notebook_reference_step = silva_extension_torch.tanh(
notebook_transition.source(notebook_condition)
+ 0.15 * notebook_transition.state_field(notebook_state0)
)
silva_extension_torch.testing.assert_close(
notebook_transition(notebook_state0, notebook_condition),
notebook_reference_step,
)
notebook_custom_model = SILVAConditionedEquilibrium(
notebook_transition,
SILVAZeroInitializer(3),
readout=silva_extension_nn.Linear(3, 1),
config=SolverConfig(
solver="picard",
max_iter=40,
tol=1e-7,
backward_mode="implicit",
backward_solver="gmres",
anderson_batch_dims=1,
),
)
notebook_custom_result = notebook_custom_model(
notebook_condition,
return_result=True,
)
assert notebook_custom_result.output.shape == (4, 1)
assert notebook_custom_result.solver_result.residual < 1e-5
notebook_custom_result.output.square().mean().backward()
assert all(
parameter.grad is not None and silva_extension_torch.isfinite(parameter.grad).all()
for parameter in notebook_custom_model.parameters()
)
print("custom transition:", notebook_report)
print("equilibrium residual:", notebook_custom_result.solver_result.residual)
import torch as silva_extension_torch
from torch import nn as silva_extension_nn
from silva_networks import (
SILVAConditionedEquilibrium,
SILVAZeroInitializer,
SolverConfig,
validate_silva_transition,
)
class NotebookExtensionTransition(silva_extension_nn.Module):
def __init__(self, condition_dim=2, state_dim=3):
super().__init__()
self.source = silva_extension_nn.Linear(condition_dim, state_dim)
self.state_field = silva_extension_nn.Sequential(
silva_extension_nn.Linear(state_dim, 2 * state_dim),
silva_extension_nn.Tanh(),
silva_extension_nn.Linear(2 * state_dim, state_dim),
)
def forward(self, state, condition):
return silva_extension_torch.tanh(
self.source(condition) + 0.15 * self.state_field(state)
)
silva_extension_torch.manual_seed(610)
notebook_condition = silva_extension_torch.linspace(-1.0, 1.0, 8).reshape(4, 2)
notebook_state0 = silva_extension_torch.zeros(4, 3)
notebook_transition = NotebookExtensionTransition()
notebook_report = validate_silva_transition(
notebook_transition,
notebook_state0,
notebook_condition,
)
assert notebook_report.valid
with silva_extension_torch.no_grad():
notebook_reference_step = silva_extension_torch.tanh(
notebook_transition.source(notebook_condition)
+ 0.15 * notebook_transition.state_field(notebook_state0)
)
silva_extension_torch.testing.assert_close(
notebook_transition(notebook_state0, notebook_condition),
notebook_reference_step,
)
notebook_custom_model = SILVAConditionedEquilibrium(
notebook_transition,
SILVAZeroInitializer(3),
readout=silva_extension_nn.Linear(3, 1),
config=SolverConfig(
solver="picard",
max_iter=40,
tol=1e-7,
backward_mode="implicit",
backward_solver="gmres",
anderson_batch_dims=1,
),
)
notebook_custom_result = notebook_custom_model(
notebook_condition,
return_result=True,
)
assert notebook_custom_result.output.shape == (4, 1)
assert notebook_custom_result.solver_result.residual < 1e-5
notebook_custom_result.output.square().mean().backward()
assert all(
parameter.grad is not None and silva_extension_torch.isfinite(parameter.grad).all()
for parameter in notebook_custom_model.parameters()
)
print("custom transition:", notebook_report)
print("equilibrium residual:", notebook_custom_result.solver_result.residual)
custom transition: SILVATransitionReport(state_shape=(4, 3), output_shape=(4, 3), preserves_shape=True, preserves_device=True, preserves_dtype=True, finite=True, differentiable=True, parameter_count=54) equilibrium residual: 5.960464477539063e-08
Numerical Equivalence, Compact Reproduction, and Scale¶
Before training, compare one packaged transition with an independently written update:
$$ e_{\mathrm{step}} =\frac{\|T_\theta(z,x)-T_{\mathrm{ref}}(z,x)\|_2} {\|T_{\mathrm{ref}}(z,x)\|_2+\varepsilon}. $$
After solving, report the fixed-point residual separately:
$$ e_{\mathrm{fp}} =\frac{\|T_\theta(z^\star,x)-z^\star\|_2} {\|z^\star\|_2+\varepsilon}. $$
For this notebook, a compact reproduction must declare and assert fixed-point residual and task error against a deterministic target. A full experiment must additionally record the source dataset version and split, preprocessing, architecture widths, solver and optimizer schedules, random seeds, baseline configuration, checkpoints, and every deviation from the cited protocol.
The principal scaling axes are state width, batch size, and data volume. Increase one axis at a time, retain the compact deterministic case as a regression test, and record task error, domain-specific residual, forward residual, backward linear residual, memory use, and runtime independently.
Extension Exercises¶
- Replace one component from this notebook while preserving its state and domain invariants.
- Write the replacement first as an independent reference function, then as a module, and assert one-step equivalence.
- Compare two solver configurations on the identical trained transition.
- Add a compact baseline and a predeclared metric threshold.
- Create a full-scale configuration without weakening the compact tests.
The complete authoring protocol is documented in Extending SILVA.
In [9]:
Copied!
notebook_reproduction_record = {
"notebook": '51_equilibrium_expansion_atlas.ipynb',
"state": 'the tensor solved to equilibrium',
"condition": 'the observed input or source tensor',
"transition": 'the state-preserving transition evaluated by the root solver',
"invariants": 'shape, device, dtype, finiteness, and differentiability',
"compact_metric": 'fixed-point residual and task error against a deterministic target',
"scale_axis": 'state width, batch size, and data volume',
}
assert all(notebook_reproduction_record.values())
notebook_reproduction_record
notebook_reproduction_record = {
"notebook": '51_equilibrium_expansion_atlas.ipynb',
"state": 'the tensor solved to equilibrium',
"condition": 'the observed input or source tensor',
"transition": 'the state-preserving transition evaluated by the root solver',
"invariants": 'shape, device, dtype, finiteness, and differentiability',
"compact_metric": 'fixed-point residual and task error against a deterministic target',
"scale_axis": 'state width, batch size, and data volume',
}
assert all(notebook_reproduction_record.values())
notebook_reproduction_record
Out[9]:
{'notebook': '51_equilibrium_expansion_atlas.ipynb',
'state': 'the tensor solved to equilibrium',
'condition': 'the observed input or source tensor',
'transition': 'the state-preserving transition evaluated by the root solver',
'invariants': 'shape, device, dtype, finiteness, and differentiability',
'compact_metric': 'fixed-point residual and task error against a deterministic target',
'scale_axis': 'state width, batch size, and data volume'}
Worked Convergence and Sensitivity Study¶
The preceding example demonstrates one configured solve. This additional study changes the transition feedback factor while keeping the source fixed, so solver effort and implicit sensitivity can be read separately from task behavior. Locally, one eigendirection of a nonlinear transition can be represented by
$$ z_{k+1} = \rho z_k + u, \qquad 0 \leq \rho < 1. $$
Its equilibrium is
$$ z^\star = \frac{u}{1-\rho}. $$
Subtracting the fixed-point equation from the iteration gives the exact error recursion
$$ e_{k+1} = \rho e_k, \qquad |e_k| = \rho^k |e_0|. $$
For a requested absolute tolerance $\tau$, the idealized iteration estimate is
$$ k \geq \frac{\log(\tau/|e_0|)}{\log \rho}. $$
The same factor controls sensitivity. Differentiating the equilibrium with respect to the source gives
$$ \frac{\partial z^\star}{\partial u} =\frac{1}{1-\rho}. $$
Thus a transition can remain contractive while becoming expensive and highly sensitive as $\rho$ approaches one. The table and figure below measure this effect rather than merely stating it. They provide a reference envelope for the notebook's actual state, the tensor solved to equilibrium, and its repeated map, the state-preserving transition evaluated by the root solver. The scalar study does not replace the domain model; it supplies a result whose convergence rate and derivative are known exactly, so the same reporting code can be trusted before it is applied to the larger transition.
In [10]:
Copied!
import math as silva_deepening_math
import torch as silva_deepening_torch
silva_deepening_rates = (0.20, 0.45, 0.70, 0.85)
silva_deepening_source = 0.35
silva_deepening_tolerance = 1e-8
silva_deepening_histories = {}
silva_deepening_rows = []
for silva_deepening_rho in silva_deepening_rates:
silva_deepening_state = silva_deepening_torch.tensor(0.0)
silva_deepening_exact = silva_deepening_source / (1.0 - silva_deepening_rho)
silva_deepening_history = []
for silva_deepening_iteration in range(1, 241):
silva_deepening_next = (
silva_deepening_rho * silva_deepening_state + silva_deepening_source
)
silva_deepening_residual = abs(
float(silva_deepening_next - silva_deepening_state)
)
silva_deepening_history.append(silva_deepening_residual)
silva_deepening_state = silva_deepening_next
if silva_deepening_residual < silva_deepening_tolerance:
break
silva_deepening_u = silva_deepening_torch.tensor(
silva_deepening_source, requires_grad=True
)
silva_deepening_solution = silva_deepening_u / (1.0 - silva_deepening_rho)
silva_deepening_solution.backward()
silva_deepening_expected_sensitivity = 1.0 / (1.0 - silva_deepening_rho)
silva_deepening_gradient_error = abs(
float(silva_deepening_u.grad) - silva_deepening_expected_sensitivity
)
silva_deepening_histories[silva_deepening_rho] = silva_deepening_history
silva_deepening_rows.append(
(
silva_deepening_rho,
silva_deepening_iteration,
silva_deepening_history[-1],
abs(float(silva_deepening_state) - silva_deepening_exact),
float(silva_deepening_u.grad),
silva_deepening_gradient_error,
)
)
print('transition feedback factor')
print("rho | iterations | final residual | exact-state error | sensitivity | gradient error")
for silva_deepening_row in silva_deepening_rows:
print(
f"{silva_deepening_row[0]:.2f} | {silva_deepening_row[1]:3d} | "
f"{silva_deepening_row[2]:.3e} | {silva_deepening_row[3]:.3e} | "
f"{silva_deepening_row[4]:.4f} | {silva_deepening_row[5]:.3e}"
)
assert all(row[2] < silva_deepening_tolerance for row in silva_deepening_rows)
assert all(row[3] < 1e-6 for row in silva_deepening_rows)
assert all(row[5] < 1e-6 for row in silva_deepening_rows)
import math as silva_deepening_math
import torch as silva_deepening_torch
silva_deepening_rates = (0.20, 0.45, 0.70, 0.85)
silva_deepening_source = 0.35
silva_deepening_tolerance = 1e-8
silva_deepening_histories = {}
silva_deepening_rows = []
for silva_deepening_rho in silva_deepening_rates:
silva_deepening_state = silva_deepening_torch.tensor(0.0)
silva_deepening_exact = silva_deepening_source / (1.0 - silva_deepening_rho)
silva_deepening_history = []
for silva_deepening_iteration in range(1, 241):
silva_deepening_next = (
silva_deepening_rho * silva_deepening_state + silva_deepening_source
)
silva_deepening_residual = abs(
float(silva_deepening_next - silva_deepening_state)
)
silva_deepening_history.append(silva_deepening_residual)
silva_deepening_state = silva_deepening_next
if silva_deepening_residual < silva_deepening_tolerance:
break
silva_deepening_u = silva_deepening_torch.tensor(
silva_deepening_source, requires_grad=True
)
silva_deepening_solution = silva_deepening_u / (1.0 - silva_deepening_rho)
silva_deepening_solution.backward()
silva_deepening_expected_sensitivity = 1.0 / (1.0 - silva_deepening_rho)
silva_deepening_gradient_error = abs(
float(silva_deepening_u.grad) - silva_deepening_expected_sensitivity
)
silva_deepening_histories[silva_deepening_rho] = silva_deepening_history
silva_deepening_rows.append(
(
silva_deepening_rho,
silva_deepening_iteration,
silva_deepening_history[-1],
abs(float(silva_deepening_state) - silva_deepening_exact),
float(silva_deepening_u.grad),
silva_deepening_gradient_error,
)
)
print('transition feedback factor')
print("rho | iterations | final residual | exact-state error | sensitivity | gradient error")
for silva_deepening_row in silva_deepening_rows:
print(
f"{silva_deepening_row[0]:.2f} | {silva_deepening_row[1]:3d} | "
f"{silva_deepening_row[2]:.3e} | {silva_deepening_row[3]:.3e} | "
f"{silva_deepening_row[4]:.4f} | {silva_deepening_row[5]:.3e}"
)
assert all(row[2] < silva_deepening_tolerance for row in silva_deepening_rows)
assert all(row[3] < 1e-6 for row in silva_deepening_rows)
assert all(row[5] < 1e-6 for row in silva_deepening_rows)
transition feedback factor rho | iterations | final residual | exact-state error | sensitivity | gradient error 0.20 | 12 | 0.000e+00 | 5.551e-17 | 1.2500 | 0.000e+00 0.45 | 23 | 0.000e+00 | 1.084e-08 | 1.8182 | 6.502e-08 0.70 | 45 | 0.000e+00 | 1.589e-07 | 3.3333 | 7.947e-08 0.85 | 93 | 0.000e+00 | 5.563e-07 | 6.6667 | 1.589e-07
In [11]:
Copied!
import matplotlib.pyplot as silva_deepening_plt
silva_deepening_plt.rcParams.update({"figure.dpi": 300, "savefig.dpi": 300})
silva_deepening_figure, silva_deepening_axes = silva_deepening_plt.subplots(
1, 2, figsize=(8.6, 3.2)
)
for silva_deepening_rho, silva_deepening_history in silva_deepening_histories.items():
silva_deepening_axes[0].semilogy(
range(1, len(silva_deepening_history) + 1),
silva_deepening_history,
marker="o",
markersize=2,
linewidth=1.2,
label=f"rho={silva_deepening_rho:.2f}",
)
silva_deepening_axes[0].axhline(
silva_deepening_tolerance, color="black", linestyle="--", linewidth=0.9
)
silva_deepening_axes[0].set_xlabel("iteration")
silva_deepening_axes[0].set_ylabel("absolute residual")
silva_deepening_axes[0].set_title("Residual trajectories")
silva_deepening_axes[0].legend(fontsize=7)
silva_deepening_axes[1].plot(
[row[0] for row in silva_deepening_rows],
[row[1] for row in silva_deepening_rows],
marker="o",
label="iterations",
)
silva_deepening_sensitivity_axis = silva_deepening_axes[1].twinx()
silva_deepening_sensitivity_axis.plot(
[row[0] for row in silva_deepening_rows],
[row[4] for row in silva_deepening_rows],
color="tab:red",
marker="s",
label="sensitivity",
)
silva_deepening_axes[1].set_xlabel('transition feedback factor')
silva_deepening_axes[1].set_ylabel("iterations")
silva_deepening_sensitivity_axis.set_ylabel("implicit sensitivity", color="tab:red")
silva_deepening_axes[1].set_title("Cost and sensitivity")
silva_deepening_figure.tight_layout()
silva_deepening_plt.show()
import matplotlib.pyplot as silva_deepening_plt
silva_deepening_plt.rcParams.update({"figure.dpi": 300, "savefig.dpi": 300})
silva_deepening_figure, silva_deepening_axes = silva_deepening_plt.subplots(
1, 2, figsize=(8.6, 3.2)
)
for silva_deepening_rho, silva_deepening_history in silva_deepening_histories.items():
silva_deepening_axes[0].semilogy(
range(1, len(silva_deepening_history) + 1),
silva_deepening_history,
marker="o",
markersize=2,
linewidth=1.2,
label=f"rho={silva_deepening_rho:.2f}",
)
silva_deepening_axes[0].axhline(
silva_deepening_tolerance, color="black", linestyle="--", linewidth=0.9
)
silva_deepening_axes[0].set_xlabel("iteration")
silva_deepening_axes[0].set_ylabel("absolute residual")
silva_deepening_axes[0].set_title("Residual trajectories")
silva_deepening_axes[0].legend(fontsize=7)
silva_deepening_axes[1].plot(
[row[0] for row in silva_deepening_rows],
[row[1] for row in silva_deepening_rows],
marker="o",
label="iterations",
)
silva_deepening_sensitivity_axis = silva_deepening_axes[1].twinx()
silva_deepening_sensitivity_axis.plot(
[row[0] for row in silva_deepening_rows],
[row[4] for row in silva_deepening_rows],
color="tab:red",
marker="s",
label="sensitivity",
)
silva_deepening_axes[1].set_xlabel('transition feedback factor')
silva_deepening_axes[1].set_ylabel("iterations")
silva_deepening_sensitivity_axis.set_ylabel("implicit sensitivity", color="tab:red")
silva_deepening_axes[1].set_title("Cost and sensitivity")
silva_deepening_figure.tight_layout()
silva_deepening_plt.show()
Reading and Extending the Result¶
The measured residual curves become flatter as the transition feedback factor increases. The iteration count and the exact sensitivity rise together, but they answer different questions: iterations measure numerical work, while sensitivity describes how strongly the equilibrium reacts to the source. The gradient-error column verifies the differentiation path against the analytic derivative.
Apply the same separation to this notebook's full model:
| Report | Notebook-specific interpretation |
|---|---|
| Task evidence | fixed-point residual and task error against a deterministic target |
| Forward residual | Re-evaluate the complete transition at the returned state |
| Empirical rate | Compare consecutive residuals only after the transient regime |
| Backward residual | Record the linear-adjoint stopping value independently |
| Sensitivity | Perturb one declared source field while preserving all other inputs |
| Structural checks | shape, device, dtype, finiteness, and differentiability |
| Scale sweep | Change one of state width, batch size, and data volume at a time |
A richer experiment should now repeat the sweep with at least two forward solvers, two tolerances, and multiple seeds. Keep model parameters and data identical when comparing solvers. Then change one architecture or data-scale axis, retain the compact analytic study as a regression test, and report task quality, residuals, iterations, runtime, memory, gradient norms, and failed convergence cases together.