Configuration

Every reservoir weight is drawn from a configurable distribution, then (for the recurrent matrix) sparsified and rescaled to the target spectral radius. Defaults follow the paper.

Weight distributions

Parameter	Controls	Options	Default
input_init	input weights \(\mathbf{W}_{xu}\)	uniform, gaussian, bernoulli, laplace	uniform
reservoir_init	reservoir weights \(\mathbf{W}_{xx}\)	same	uniform
bias_init	neuron bias	same	uniform

• uniform: \(\mathcal{U}[-s, s]\) (the paper’s choice; \(s\) is input_scaling for the input)

• gaussian: \(\mathcal{N}(0, s^2)\)

• bernoulli: bimodal \(\{-s, +s\}\)

• laplace: \(\mathrm{Laplace}(0, s)\)

For the reservoir the absolute scale is irrelevant, since it is set afterwards by the spectral-radius rescaling, so only the shape of the distribution matters there.

Reservoir knobs

Parameter	Meaning	Range	Default
spectral_radius	\(\mathbf{W}_{xx}\) rescaled to this \(\rho\)	below 1	0.9
sparsity	fraction of \(\mathbf{W}_{xx}\) set to zero (\(\varphi\))	\([0, 1)\)	0.9
bias_scaling	scale of the bias (0 = none)	\(\ge 0\)	0.0
noise	std \(\sigma\) of state noise \(\eta(t)\sim\mathcal{N}(0,\sigma^2 I)\)	\(\ge 0\)	0.0

Activation functions

The reservoir nonlinearity is chosen with activation=. Seventeen options are built in, each implemented in the Cython kernel with a matching NumPy fallback:

Name	Definition
tanh (default)	\(\tanh(x)\)
sigmoid	\(1/(1+e^{-x})\)
relu	\(\max(0, x)\)
leaky_relu	\(x\) if \(x>0\) else \(0.01x\)
elu	\(x\) if \(x>0\) else \(e^{x}-1\)
selu	scaled ELU
softplus	\(\log(1+e^{x})\)
swish / silu	\(x\,\sigma(x)\)
gelu	Gaussian error linear unit
hard_tanh	\(\mathrm{clip}(x, -1, 1)\)
softsign	\(x/(1+\lvert x\rvert)\)
mish	\(x\tanh(\mathrm{softplus}(x))\)
hard_sigmoid	\(\mathrm{clip}(0.2x+0.5, 0, 1)\)
relu6	\(\mathrm{clip}(x, 0, 6)\)
tanhshrink	\(x-\tanh(x)\)
sin / sine	\(\sin(x)\)
identity / linear	\(x\)

In the two-reservoir models the activation can differ per reservoir, for example activation=("tanh", "sin").

Broadcasting rules

Per-reservoir hyperparameters accept either a single value (applied to every reservoir) or a list matching the reservoir count. Otherwise a ValueError is raised.

• MultiESN: a scalar broadcasts to all n_reservoirs; a list must have length n_reservoirs.

• DoubleReservoirESN / fESN / wESN: a scalar broadcasts to both; a (vanilla, memory) pair sets them individually.

• random_state: a single int derives distinct per-reservoir seeds (reproducible but not identical); a list sets the seeds explicitly and must match the reservoir count; None disables reproducibility.

from memory_esn import fESN

fESN(
    spectral_radius=(0.95, 0.85),   # vanilla / memory
    sparsity=0.9,                   # both
    noise=(0.0, 0.02),              # only the memory reservoir
    activation=("tanh", "sin"),     # per-reservoir nonlinearity
    reservoir_init="uniform",
    random_state=0,
)