Metaplasticity

AdditiveNorm(; τ, operator = +)

Additive synaptic normalization, to be passed to SynapseNormalization (or MetaPlasticity). Every τ, a common offset is added to the incoming weights of each postsynaptic neuron so that their sum returns to its value at construction time (see SynapseNormalization).

Fields

• τ::Float32: interval between two normalizations (ms). Required, no default.
• operator::Function = +: kept for compatibility, not used (the update is selected by the type).

MultiplicativeNorm(; τ, operator = *)

Multiplicative synaptic normalization, to be passed to SynapseNormalization (or MetaPlasticity). Every τ, the incoming weights of each postsynaptic neuron are scaled by a common factor so that their sum returns to its value at construction time (see SynapseNormalization).

Fields

• τ::Float32: interval between two normalizations (ms). Required, no default.
• operator::Function = *: kept for compatibility, not used (the update is selected by the type).

Example

using SpikingNeuralNetworks
SNN.@load_units
norm = SNN.MultiplicativeNorm(τ = 20ms)

NormParam <: MetaPlasticityParameter

Abstract type of the normalization parameters: MultiplicativeNorm, AdditiveNorm (used by SynapseNormalization) and AggregateScalingParameter (used by AggregateScaling).

SynapseNormalization{VFT, VIT, VST} <: AbstractNormalization

Metaplasticity object that keeps the total excitatory input weight of each postsynaptic neuron close to its initial value. It acts on one or more sparse synapses (synapses) that share the same postsynaptic population, and is added to the model as a connection (it transmits nothing: its forward! is a no-op).

Update

At construction $W^0_i = \sum_{s \in \text{in}(i)} W_s$ (sum over all synapses of all synapses onto neuron $i$). Under train! only, when the step counter is a multiple of round(Int, τ / dt), with $W^1_i$ the current sum:

• MultiplicativeNorm: $\mu_i = W^0_i / W^1_i$ and $W_s \leftarrow W_s\,\mu_i$, which restores $\sum_s W_s = W^0_i$ exactly.
• AdditiveNorm: $\mu_i = (W^0_i - W^1_i)/n_i$ ($n_i$ inputs of neuron $i$) and $W_s \leftarrow W_s + \mu_i$, which also restores $\sum_s W_s = W^0_i$ exactly.

Neurons without inputs ($n_i = 0$), and for MultiplicativeNorm neurons with $W^1_i = 0$, are left unchanged. The period is max(1, round(Int, τ / dt)) steps.

The normalization runs at its position in the connection list, after the forward! and plasticity! of the connections listed before it.

Fields

• param::NormParam: MultiplicativeNorm or AdditiveNorm.
• synapses::Vector{<:AbstractSparseSynapse}: normalized connections.
• W0::Vector{Float32}: initial summed input weight per postsynaptic neuron.
• W1::Vector{Float32}: summed input weight at the last normalization.
• μ::Vector{Float32}: last normalization factor/offset per neuron.
• t::Vector{Int32} = [0, 1]: unused.
• id, name = "SynapseNormalization", targets (:post id and :synapses ids), records.

References

Reference not given in the code.

SynapseNormalization(synapses; param::NormParam, kwargs...)

Build a SynapseNormalization acting on the vector synapses (all AbstractSparseSynapse with the same postsynaptic population; asserted). Computes W0 from the current weights. kwargs... are forwarded to the struct (e.g. name).

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 100)
EE = SNN.SpikingSynapse(E, E, :ge; conn = (p = 0.2, μ = 2.0), LTPParam = SNN.STDPGerstner())
norm = SNN.SynapseNormalization([EE]; param = SNN.MultiplicativeNorm(τ = 20ms))
model = SNN.compose(; E, EE, norm)
SNN.train!(model; duration = 100ms)

MetaPlasticity(param::NormParam, synapses; kwargs...)

Same as SynapseNormalization(synapses; param, kwargs...).

plasticity!(c::SynapseNormalization, param::NormParam, dt::AbstractFloat, T::Time)

Apply the normalization (see SynapseNormalization) when get_step(T) is a multiple of max(1, round(Int, param.τ / dt)); otherwise do nothing. Called only by train!.

plasticity!(c::SynapseNormalization, param::NormParam)

Apply one normalization step immediately: recompute W1, the factors/offsets μ, and update the weights of all c.synapses.

AggregateScaling{VFT, VST} <: AbstractNormalization

Homeostatic aggregate scaling of the excitatory input of each postsynaptic neuron: a rate estimate $y_i$ drives a target total weight $W^T_i$, and the incoming weights are periodically rescaled so that their sum follows $W^T_i$.

Equations

\[\tau_a \frac{dy_i}{dt} = -y_i + \sum_f \delta(t - t_i^f), \qquad \tau_e \frac{dW^T_i}{dt} = \left(1 - \frac{W^T_i}{W_{max}}\right)\left(1 - \frac{y_i}{Y_i}\right)\]

$y_i$ is an estimate of the firing rate of neuron $i$ (1/ms; a spike increases it by $1/\tau_a$), compared with the target rate $Y_i$. $W^T_i$ starts from the summed input weight at construction.

Update

• forward! (every step of sim! and train!): forward Euler step of $y$.
• plasticity! (every step of train! only): forward Euler step of $W^T$; every max(1, round(Int, τ / dt)) steps, with $W^t_i$ the current summed input weight,

\[\mu_i = \frac{\max(W^T_i - n_i W_{min}, 0)}{W^t_i}, \qquad W_s \leftarrow W_s\,\mu_i + W_{min}\]

for every synapse $s$ onto neuron $i$ ($n_i$ incoming synapses), so that the summed weight equals $W^T_i$ (neurons with no input weight are skipped).

Fields

• param::AggregateScalingParameter; synapses: the scaled sparse synapses (same postsynaptic population).
• Wt::Vector{Float32}: summed input weight at the last rescaling.
• WT::Vector{Float32}: target summed input weight.
• y::Vector{Float32}: rate estimate (1/ms); μ::Vector{Float32}: last scaling factors.
• fire: reference to the postsynaptic fire vector.
• N::Int32: number of postsynaptic neurons.
• id, targets, records.

References

Reference not given in the code.

AggregateScaling(N, synapses; param::AggregateScalingParameter, kwargs...)

Build an AggregateScaling for the vector synapses (sparse synapses with the same postsynaptic population, asserted). N is the number of postsynaptic neurons, either an Int or any object with a field N (e.g. the postsynaptic population). WT is initialised with the current summed input weight of each neuron.

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 100)
EE = SNN.SpikingSynapse(E, E, :ge; conn = (p = 0.2, μ = 2.0))
AS = SNN.AggregateScaling(E, [EE]; param = SNN.AggregateScalingParameter(E.N, 5Hz))
model = SNN.compose(; E, EE, AS)
SNN.train!(model; duration = 100ms)

AggregateScalingParameter(; τ = 10ms, τa, τe, Y, Wmin = 0.5pF, Wmax = 250pF)
AggregateScalingParameter(N, rate = 10Hz; τ = 10ms, τa = 100ms, τe = 100ms, Wmin = 0.5pF)

Parameters of AggregateScaling (homeostatic scaling of the summed input weight towards a target firing rate).

Fields

• τ::Float32 = 10ms: interval between two rescalings of the weights (ms).
• τa::Float32: time constant of the rate estimate y (ms); positional default 100ms.
• τe::Float32: time constant of the target total weight WT (ms); positional default 100ms.
• Y::Vector{Float32}: target firing rate of each postsynaptic neuron, in library units (1/ms; write it with the Hz unit, e.g. 5Hz). The positional constructor fills it with rate (default 10Hz).
• Wmin::Float32 = 0.5pF: offset added to every weight after rescaling. The pF unit in the code is a numerical factor (1); the weight unit is that of the synapse target.
• Wmax::Float32 = 250pF: soft upper bound of WT.

The positional form builds the vector Y for N neurons and does not take Wmax. (Up to SNNModels 1.8.4 its Wmin default was 0.05, against 0.5pF in the keyword form.)

forward!(c::AggregateScaling, param::AggregateScalingParameter, dt::AbstractFloat, T::Time)

Update the rate estimate y (see AggregateScaling). Runs at every step of sim! and train!.

plasticity!(c::AggregateScaling, param::AggregateScalingParameter, dt::AbstractFloat, T::Time)

Update the target summed weight WT and, when get_step(T) is a multiple of max(1, round(Int, param.τ / dt)), rescale the incoming weights (see AggregateScaling). Called only by train!.

plasticity!(c::AggregateScaling, param::AggregateScalingParameter)

Rescale the incoming weights immediately: W ← W μ + Wmin with μ = max(WT - n Wmin, 0) / Wt (n incoming synapses), so that the summed weight becomes WT (or n Wmin if WT < n Wmin). Neurons whose summed input weight Wt is 0 are left unchanged.

ActivityDependentTurnover(; rate = -1, τ = 1 / rate, fraction = 0.1f0,
                          τpre = 250ms, τpost = 250ms, μ = 3.0f0)

Parameters of activity-dependent structural turnover: every τ, the fraction of synapses with the lowest pre/post co-activity are moved to new postsynaptic targets.

Fields

• rate = -1: turnover rate (1/ms); only used for the default τ. Pass rate or τ.
• τ = 1 / rate: interval between turnover events (ms).
• fraction::Float32 = 0.1: quantile of co-activity below which synapses are rewired.
• τpre::Float32 = 250ms, τpost::Float32 = 250ms: time constants of the pre- and postsynaptic activity traces (ms).
• μ::Float32 = 3.0: mean of the new weights, $\mathcal{N}(\mu, \sqrt{\mu})$ truncated to positive values.

The element type FT is inferred from the arguments (no default), so give rate/τ as Float32 values for a Float32 struct.

RandomTurnover(; rate = -1.0f0, τ = 1 / rate, threshold = 0.1f0, μ = 3.0f0)

Parameters of random structural turnover: every τ, each synapse is moved with probability threshold to a new, uniformly chosen postsynaptic target.

Fields

• rate::Float32 = -1: turnover rate (1/ms); only used to compute the default τ.
• τ::Float32 = 1 / rate: interval between turnover events (ms).
• threshold::Float32 = 0.1: probability that a synapse is rewired at a turnover event.
• μ::Float32 = 3.0: mean of the new weights, drawn from $\mathcal{N}(\mu, \sqrt{\mu})$ truncated to positive values.

Up to SNNModels 1.8.4 there was no plasticity!(c::Turnover, ::RandomTurnover, dt, T) method (train! raised a MethodError) and threshold was unused.

Turnover{VFT, MFT, ST} <: AbstractMetaPlasticity

Structural turnover acting on one SpikingSynapse (synapse). Built with MetaPlasticity(param::TurnoverParam, synapse) and added to the model as a connection; it transmits nothing (forward! is a no-op), its plasticity! runs only under train!.

Update (ActivityDependentTurnover)

At every train! step the presynaptic and postsynaptic traces are updated as

\[a^{pre}_j \leftarrow a^{pre}_j + \frac{-a^{pre}_j\,dt + \delta_j}{\tau_{pre}}, \qquad a^{post}_i \leftarrow a^{post}_i + \frac{-a^{post}_i\,dt + \delta_i}{\tau_{post}}\]

($\delta = 1$ for a spike in this step). Every round(Int, τ / dt) steps:

1. $p_s = a^{pre}_{j(s)}\, a^{post}_{i(s)}$ for every synapse $s$;
2. p_rewire = the fraction-quantile of $p$;
3. synaptic_turnover! moves every synapse with $p_s \le$ p_rewire to a new postsynaptic neuron, chosen without replacement among the neurons not yet targeted by the same presynaptic neuron (uniform weights p[post, pre] = 1), and draws its weight from $\mathcal{N}(\mu, \sqrt{\mu})$; the sparse structure is then rebuilt.

Fields

• param::TurnoverParam; synapse: the rewired connection.
• pre, post::Vector{Float32}: activity traces.
• p::Matrix{Float32}: N_post x N_pre sampling weights of new targets (all ones).
• p_rewire::Vector{Float32}: current rewiring threshold (1 element).
• p_values::Vector{Float32}: co-activity of every synapse.
• id, name = "Turnover", targets, records.

References

Reference not given in the code.

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 100)
EE = SNN.SpikingSynapse(E, E, :ge; conn = (p = 0.2, μ = 2.0))
TO = SNN.MetaPlasticity(SNN.ActivityDependentTurnover(τ = 50.0f0ms), EE)
model = SNN.compose(; E, EE, TO)
SNN.train!(model; duration = 100ms)

TurnoverParam <: MetaPlasticityParameter

Abstract type of the structural-turnover rules used by Turnover: RandomTurnover and ActivityDependentTurnover.

MetaPlasticity(param::TurnoverParam, synapse; kwargs...)

Build a Turnover acting on synapse.

plasticity!(c::Turnover, param::ActivityDependentTurnover, dt::AbstractFloat, T::Time)

Update the activity traces and, every max(1, round(Int, τ / dt)) steps, rewire the least co-active synapses (see Turnover). Called only by train!.

plasticity!(c::Turnover, param::RandomTurnover, dt::AbstractFloat, T::Time)

Every max(1, round(Int, τ / dt)) steps, rewire each synapse with probability param.threshold (see RandomTurnover). Called only by train!.

plasticity!(c::Turnover, param::TurnoverParam)

Run synaptic_turnover! on c.synapse with the current c.p_rewire[1], c.p_values and param.μ.

synaptic_turnover!(C; p_rewire = 0.05, p_new = (post, pre) -> 1.0, μ = 3.0, p_values = nothing)

Rewire, in place, the synapses of a sparse connection C (e.g. SpikingSynapse).

Keyword arguments

• p_rewire = 0.05: threshold; synapse $s$ is rewired if p_values[s] <= p_rewire.
• p_values = nothing: one value per synapse (CSC order of C.W); if nothing, drawn uniformly in [0, 1], so that a fraction p_rewire of synapses is rewired on average.
• p_new = (post, pre) -> 1.0: function giving the sampling weight of each candidate new postsynaptic neuron (default: uniform). (Up to SNNModels 1.8.4 the default was the one-argument x -> rand(), which raised a MethodError.)
• μ = 3.0: new weights are drawn from $\mathcal{N}(\mu, \sqrt{\mu})$ truncated to positive values (up to 1.8.4 negative weights could be drawn).

Procedure

1. For every presynaptic neuron, the candidate targets are the postsynaptic neurons it does not contact yet, weighted by p_new(post, pre).
2. The synapses to rewire are selected with p_values[s] <= p_rewire.
3. For each presynaptic neuron, as many new targets as rewired synapses are sampled without replacement (if there are fewer free targets than selected synapses, only as many synapses as free targets are rewired; up to 1.8.4 this case raised an error); the postsynaptic index C.I[s] and the weight C.W[s] are replaced, and the short-term efficacy ρ[s] of a rewired synapse is reset to 1.
4. update_sparse_matrix!(C) rebuilds colptr, rowptr, J and index and reorders the per-synapse arrays (ρ, delays) consistently.