Long-Memory Echo State Networks#

RESERVOIR COMPUTING · LONG-RANGE DEPENDENCE

A reservoir-computing library that augments the classic Echo State Network with a dedicated memory reservoir for long-range dependence. Two mechanisms, a fractional-difference filter (fESN) and a wavelet-smoothed one (wESN), capture slowly decaying correlations while keeping all internal weights fixed and training only a linear ridge readout. BLAS-backed Cython kernels make it fast, and NumPy fallbacks keep it portable.

pip install memory-esn

Meet the two models#

fESN · Fractional ESN

f(t) = ((1-B)ᵈ − 1) u(t)

A memory reservoir driven by a fractional-difference filter. Its weights fade polynomially, carrying information from the distant past that a plain reservoir forgets.

Models

wESN · Wavelet ESN

f(t) = ((1-B)ᵈ − 1) MODWT(u(t))

Same idea, but a shift-invariant wavelet smooth cleans the signal first. It differences the trend instead of the noise, so it forecasts what actually persists.

Models

🌊 Long memory

The memory reservoir is driven by ((1-B)^d - 1) u(t), giving polynomially decaying weights over the past, in contrast to the reservoir’s exponential forgetting.

⚡ Fast kernels

Reservoir updates, fractional differencing and MODWT run as Cython extensions (dgemv, nogil), with NumPy fallbacks verified equal to the bit.

🧩 Clean hierarchy

BaseESN, MultiESN, DoubleReservoirESN, fESN / wESN. Each is a genuine specialization of the one above.

Performance: ours vs. others#

The reservoir update is where a forecaster spends most of its time. memory-esn picks its storage adaptively, dense BLAS for dense reservoirs and a fused sparse kernel for large sparse ones, so it is faster than a widely used reservoir-computing baseline across the board, at identical accuracy.

≈ 2×

faster state generation on dense reservoirs

up to 2.2×

faster on large sparse reservoirs

0

accuracy lost, states match to 2e-16

State generation, ours vs. others (higher is better)#
reservoir	ours	others	speedup
dense, 300 units	83 ms	166 ms	2.0×
sparse 90%, 2000 units	1100 ms	1316 ms	1.2×
sparse 98%, 2000 units	238 ms	428 ms	1.8×

Full methodology and numbers are in the repository’s benchmarks/ results.

Defaults follow the paper

Out of the box: uniform weight initialization, reservoirs sparsified to 90 % zeros and rescaled to a spectral radius below 1, and a pure-past fractional memory filter. Optional isotropic state noise η(t) ~ N(0, σ²I) via noise=σ.

Quickstart#

import numpy as np
from memory_esn import fESN

# univariate series u(t); forecast H steps ahead
u = np.cumsum(np.random.randn(600)).reshape(-1, 1) * 0.1
Y = np.column_stack([np.roll(u[:, 0], -h) for h in (1, 2, 3)])

fesn = fESN(n_reservoir=(200, 150), d=0.4, K=80, random_state=0)
fesn.fit(u, Y, washout=100)
preds = fesn.predict(u, continuation=True)   # (T, H): one column per horizon

Seventeen reservoir activations are available (tanh, relu, gelu, mish, sin, and more), selected with activation=. See Quickstart for one example per class and Configuration for the full option reference.

Authors#

memory-esn is developed by:

Rahul Goswami | Shinjini Paul | Palash Ghosh | Tanujit Chakraborty

If you use memory-esn in your research, please cite it (see CITATION.cff in the repository). Released under the MIT License.

Acknowledgements#

We sincerely thank Ms. Donia Besher and Mr. Rajdeep Pathak of Sorbonne University for their constructive suggestions, which greatly improved the presentation of this work.