Connections

AbstractConnectionParameter <: AbstractParameter

Supertype of the param field of every connection (AbstractConnection). The concrete parameter type selects, by dispatch, the forward!, update_traces! and plasticity! methods that the simulation loop calls for the connection.

AbstractConnectivity

Abstract placeholder type for connectivity specifications. No concrete subtype is defined in SNNModels 1.8.4; connectivity is passed as conn::Union{NamedTuple,AbstractMatrix} (see sparse_matrix).

AbstractMetaPlasticity <: AbstractConnection

Connections that do not transmit activity but act on the weights of other connections (SynapseNormalization, AggregateScaling, Turnover). They are added to the model like any connection, and their plasticity! runs only under train!.

AbstractNormalization <: AbstractMetaPlasticity

Metaplasticity objects that rescale the total input weight of each postsynaptic neuron (SynapseNormalization, AggregateScaling).

AbstractSparseSynapse <: AbstractConnection

Connections that store their weights in the double sparse layout produced by dsparse (fields rowptr, colptr, I, J, index, W). Required by the connectivity helpers (matrix, presynaptic, postsynaptic, ...) and by the metaplasticity objects.

AbstractSpikingSynapse <: AbstractSparseSynapse

Sparse connections driven by presynaptic spikes (SpikingSynapse).

AbstractSpikingSynapseParameter <: AbstractConnectionParameter

Parameter types of SpikingSynapse: SpikingSynapseParameter (no delays) and the internal SpikingSynapseDelayParameter (per-synapse delays and delivery queues).

MetaPlasticityParameter <: AbstractConnectionParameter

Supertype of the parameters of metaplasticity objects: NormParam (synaptic normalization, aggregate scaling) and TurnoverParam (structural turnover).

PlasticityParameter

Supertype of plasticity rules: long-term rules (LTPParameter, passed to SpikingSynapse with the LTPParam keyword) and short-term rules (STPParameter, keyword STPParam). Plasticity is applied only by train!, never by sim!.

PlasticityVariables

Supertype of the per-connection state of a plasticity rule (traces, last spike times, STP variables). Created by plasticityvariables(rule, Npre, Npost); every concrete subtype has an active flag vector that switches the rule on or off.

convert_precision(x, ::Type{T})

Copy of the parameter object x (plasticity rule, metaplasticity or stimulus parameter, ...) with its floating-point fields and type parameters converted to T; x itself if nothing changes. Other fields are shared, not copied. Used by the connection and stimulus constructors. A Float32 rule converted to Float64 keeps the Float32 values: build it with FT = Float64 (e.g. STDPGerstner(FT = Float64, A_pre = 1e-4)) for exact Float64 values.

forward!(c::AbstractConnection, param::AbstractConnectionParameter, dt::AbstractFloat, T::Time)
forward!(c, param)

Propagate the activity of the presynaptic population of connection c to its postsynaptic target for one time step.

sim! and train! call the four-argument form once per step for every connection, after all stimuli and populations have been updated. The generic four-argument method defined here forwards to the two-argument form forward!(c, param), which connection types that do not need the time implement. SpikingSynapse implements the four-argument form directly.

What is added to the target depends on the connection type:

• SpikingSynapse: for every presynaptic neuron $j$ that fired in this step, $g_i \leftarrow g_i + W_{ij}\,\rho_{ij}$ for all its targets $i$ ($g$ is the postsynaptic variable selected by sym, $\rho$ the short-term efficacy). With delays, the increment is queued and added once the delay has elapsed.
• RateSynapse: $g_i \leftarrow g_i + \sum_j W_{ij} r_j$.
• FLSynapse, PINningSynapse: $g = W r$ (overwrite), plus the FORCE feedback term.
• metaplasticity objects (SynapseNormalization, Turnover): no-op, except AggregateScaling, which updates its activity trace and target in forward!.

The parameter type of a connection must be a subtype of AbstractConnectionParameter for the generic four-argument method to apply.

prepare_spikes!(c::AbstractConnection)
release_spikes!(c::AbstractConnection)

Called by train! and sim! around the update of every connection: between the two calls the connection's spike index lists are reused by all its consumers. No-op for connections without them.

presynaptic_spikes(c)
postsynaptic_spikes(c)

Ascending Int32 indices of the presynaptic (c.fireJ) and postsynaptic (c.fireI) neurons that fired in this step. SpikingSynapse serves them from its SpikeIndex lists; the generic methods (synapse types without the lists) scan the firing vector and allocate.

EmptySynapse(; id = randstring(12), param = EmptyParam(), targets = Dict(), records = Dict())

Connection that does nothing: its forward!, update_traces! and plasticity! are no-ops. It is the default connection list of sim! and train! (C = [EmptySynapse()]) when a network has no connections, so both sim!([pop]; duration) and train!([pop]; duration) work.

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 10)
SNN.sim!([E], [SNN.EmptySynapse()]; duration = 10ms)

SpikingSynapse(pre, post, sym, comp = nothing; conn, delay_dist = nothing, dt = 0.125f0,
               LTPParam = NoLTP(), STPParam = NoSTP(), name = "SpikingSynapse")

Sparse synapse that propagates the spikes of pre to the target variable sym (and compartment comp) of post.

Transmission

At every step (forward!, called by both sim! and train!), for every presynaptic neuron $j$ with pre.fire[j] == true and every stored synapse $(i, j)$:

\[g_i \leftarrow g_i + W_{ij}\,\rho_{ij}\]

where $g$ is the postsynaptic variable resolved by synaptic_target(targets, post, sym, comp) (for generalized IF models the generic mapping of get_synapse_symbol sends :ge/:he to the receptor array :glu and :gi/:hi to :gaba; other symbols are used as given), $W_{ij}$ is the weight and $\rho_{ij}$ the short-term efficacy (1 without STP). The units of $W$ are the units of the target variable (e.g. nS for conductances, pA for current synapses); $W$ is added once per spike, the postsynaptic model then integrates the target.

With delay_dist, every synapse has a fixed delay $d_{ij}$ (ms), rounded up to whole steps; the increment computed at emission is added to $g_i$ at the arrival step (see SpikingSynapseDelayParameter).

Arguments

• pre, post: populations (AbstractPopulation).
• sym::Symbol: target conductance/current of post (e.g. :ge, :gi, :he, :hi, :g, or a receptor name); comp: compartment for multicompartment models (e.g. :d1, :s).
• conn: either a NamedTuple of sparse_matrix options (p or ρ, μ, σ, dist, rule, γ, kmin) or an explicit Npost x Npre matrix (dense or sparse, any element type).
• delay_dist: optional Distribution; one delay (ms) per synapse is drawn from it (stored once if all drawn delays are equal, e.g. Dirac(d)).
• dt: unused (kept for backward compatibility).
• LTPParam: long-term plasticity rule (STDPGerstner, STDPTriplet, STDPWeightDependent, iSTDPRate, vSTDPParameter, ...). Applied only when the network is run with train!; sim! propagates spikes but never updates the weights.
• STPParam: short-term plasticity rule acting on the efficacy ρ (e.g. MarkramSTPParameter), also applied only by train!.
• name: name used in print_model and records.

LTPParam and STPParam are keyword-argument (and field) names, not types: the abstract rule types are LTPParameter and STPParameter.

Fields

• rowptr, colptr, I, J, index, W: double sparse storage as returned by dsparse (CSC column pointers colptr, postsynaptic index I and presynaptic index J of every synapse, row pointers rowptr of the transposed matrix and the map index from row-major to CSC position); W::Vector{FT} holds the weights in CSC order. J and index are built only when something reads them (a rule that needs_transpose, presynaptic, presynaptic_idxs, SynapseNormalization, AggregateScaling); otherwise empty. Call SNNModels.ensure_transpose!(c) before reading them in user code.
• ρ::Vector{FT}: short-term efficacy per synapse, stored only with a short-term rule; otherwise empty (ρ = 1). Read it with SNNModels.efficacy(c); call SNNModels.materialize_efficacy!(c) before writing to it.
• fireI, fireJ: references to post.fire and pre.fire.
• g, v_post: references to the target variable and to the membrane potential of the target compartment.
• param: SpikingSynapseParameter or SpikingSynapseDelayParameter.
• LTPParam, STPParam, LTPVars, STPVars: plasticity rules and their state.
• targets::Dict: ids of pre/post, target symbol, connection type.
• records::Dict: recorded variables.
• pre_spikes, post_spikes: SpikeIndex lists of the neurons of pre/post that fired in the current step, shared by forward! and the event-driven rules (the population's own list when it keeps one).
• chunkptr::ChunkPtr: cached sub-row pointers of the threaded kernels.

Threading

With neurons_per_thread > 0 (see set_neurons_per_thread!) and more than one thread, forward! and the event-driven STDP, iSTDP and STP updates are partitioned by postsynaptic range, with the chunks of the postsynaptic population. Each target receives its additions in serial order, so results are bit-identical for any number of threads. Synapses with fewer than SNNModels.SYN_MIN_OPS[] operations per step run serially; vSTDP and iSTDPPotential are always serial.

Notes

• Precision: the synapse has the floating-point type FT of the target population (float_type(post)). W, ρ, delays and the plasticity rules and state are converted to it (convert_precision; a Float32 rule keeps Float32 values). A target population whose target variable and membrane potential differ in precision is an ArgumentError.
• Weights of the event-driven vSTDPParameter are updated lazily: read them with plasticity_value(syn, :W).
• If pre == post, autapses are removed structurally (no stored zero-weight self synapses that plasticity could grow). With rule = :Fixed this lowers the in-degree by one for the neurons that had drawn themselves.

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 400); I = SNN.IF(N = 100)
EI = SNN.SpikingSynapse(E, I, :ge; conn = (p = 0.2, μ = 3.0))
IE = SNN.SpikingSynapse(I, E, :gi; conn = (p = 0.2, μ = 5.0), LTPParam = SNN.iSTDPRate(r = 5Hz))
EE = SNN.SpikingSynapse(E, E, :ge; conn = (p = 0.1, μ = 2.0), delay_dist = SNN.SNNModels.Uniform(1ms, 3ms))
model = SNN.compose(; E, I, EI, IE, EE)
SNN.sim!(model; duration = 100ms)

SpikingSynapseDelayParameter(; delaytime)

Parameter of a SpikingSynapse with per-synapse transmission delays (not exported; built by the SpikingSynapse constructor when delay_dist is given). Each delay is rounded up to whole steps, ceil(delay / dt), at the first forward! and whenever dt changes; the increments $W_{ij}\,\rho_{ij}$ (evaluated at emission) wait in a ring buffer indexed by arrival step. A delay common to all synapses without LTP and STP delays the spike lists instead (delay_kernels.jl).

Fields

• delaytime::Vector{FT}: delay of each synapse (ms) in the CSC order of W, or one value common to all synapses.
• delaysteps::Vector{Int32}: the same in steps of the dt of the last forward!.
• buffer::Matrix{FT}: Npost x (max(delaysteps) + 1) ring of pending increments, indexed from pos.
• dt::Vector{FT}: the step delaysteps was computed for (NaN before the first forward!).
• pos::Vector{Int}: ring slot (0-based) of the current step.
• spike_ring::Vector{Vector{Int32}}: spike lists of the last d + 1 steps (spike-list path only).
• drain::Vector{Int}: steps left to deliver the buffer after a switch to the spike-list path.

SpikingSynapseParameter()

Parameter of a SpikingSynapse without transmission delays (no fields). Selects the forward! method that adds $W_{ij}\,\rho_{ij}$ to the target in the same step in which the presynaptic neuron fires.

efficacy(c::SpikingSynapse)
materialize_efficacy!(c::SpikingSynapse)

c.ρ is stored only with a short-term rule; otherwise it is empty and ρ = 1. efficacy(c) returns c.ρ when stored, else a new vector of ones (writes to it have no effect). materialize_efficacy!(c) stores ones if needed and returns c.ρ, whose values forward! then uses.

forward!(c::SpikingSynapse, param::SpikingSynapseDelayParameter, dt, T)

Delayed transmission: add each increment $W_{ij}\rho_{ij}$ to the ring buffer of target $i$ at its arrival step, then add the arrivals of the current step to c.g (see SpikingSynapseDelayParameter).

forward!(c::SpikingSynapse, param::SpikingSynapseParameter, dt, T)

For each presynaptic neuron $j$ that fired, add $W_{ij}\rho_{ij}$ to c.g[i] for all its targets $i$ (no delay).

has_transpose(c::SpikingSynapse)
ensure_transpose!(c::SpikingSynapse)

Whether the transposed index arrays c.J, c.index are present; build them in place if not. Not thread-safe: call outside the threaded kernels.

update_plasticity!(c::SpikingSynapse; LTP = nothing, STP = nothing)

Replace the long-term (LTP) and/or short-term (STP) plasticity rule of c by a rule of the same type (e.g. other parameters) and re-create the corresponding state with plasticityvariables. Arguments left to nothing are not changed. A rule of another type is an ArgumentError: the rule types are type parameters of SpikingSynapse, use with_plasticity instead.

with_plasticity(c::SpikingSynapse; LTP = nothing, STP = nothing)
without_plasticity(c::SpikingSynapse)

New SpikingSynapse with the connectivity, weights and delays of c and the long-term rule LTP and/or the short-term rule STP, converted to the precision of c, with fresh state. A rule left to nothing is kept with its state; NoLTP() / NoSTP() switch that part off; without_plasticity(c) switches both off. This is how a rule of another type is installed (update_plasticity! and change_plasticity! accept only the current type); the caller replaces c by the result in its model. c is not modified.

Switching on builds the arrays the new rules need (J/index for a rule that needs_transpose, ρ for a short-term rule), reusing those of c when present. Switching off first writes back deferred weight updates (sync_weights!); the result has the type, memory and per-step work of a synapse built without plasticity. With STP = NoSTP() the efficacy is 1 again (to freeze ρ instead, keep the rule and call set_STP!(c, false)).

What is shared

W, the connectivity, g, v_post, fireI, fireJ, param (delays and delay queue), records, id, name, the spike lists, the chunk cache and any kept rule with its state are the same objects in c and in the result: use only one of the two in a model, or deepcopy(c) first. A SynapseNormalization or AggregateScaling built on c keeps using c.

Records

:W, :ρ and the population records continue across the switch. A rule variable continues if the new rule has it (e.g. :tpre from STDPGerstner to STDPWeightDependent); otherwise its record stops (for c too, records is shared) and stays readable. A stopped record is not reopened by a later switch.

Replacing the synapse in a model

model = compose(; pop = model.pop, syn = merge(model.syn, (; EE = EE2)), stim = model.stim, time = model.time), or assign into a Vector{AbstractConnection}.

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 100); E.I .= 300.0
EE = SNN.SpikingSynapse(E, E, :ge; conn = (p = 0.2, μ = 0.5))
EE = SNN.with_plasticity(EE; LTP = SNN.STDPGerstner())
model = SNN.compose(; E, EE)
SNN.train!(; model, duration = 100ms)
EE = SNN.without_plasticity(model.syn.EE)                  # W frozen
model = SNN.compose(; pop = model.pop, syn = (; EE), time = model.time)
SNN.sim!(; model, duration = 100ms)

without_plasticity(c::SpikingSynapse)

with_plasticity(c; LTP = NoLTP(), STP = NoSTP()): a synapse with the weights of c frozen and no rule, with the type, memory and per-step work of a synapse built without plasticity. See with_plasticity.

connect!(c, j, i, μ = randn(Float32))

Set the weight of the synapse from presynaptic neuron j to postsynaptic neuron i of the sparse connection c to μ, creating the synapse if it does not exist, and rebuild the sparse storage with update_sparse_matrix!(c, W).

The per-synapse arrays are kept consistent (see update_sparse_matrix!): existing synapses keep their short-term efficacy ρ and delay, a new synapse gets ρ = 1 and the mean delay. (Up to SNNModels 1.8.4 ρ and the delays kept their old length, so forward! read out of bounds after a synapse was created.)

dsparse(A::SparseMatrixCSC) -> (rowptr, colptr, I, J, index, V)

Double sparse representation of the N_post x N_pre matrix A, used by all sparse connections:

• colptr, I (row indices = postsynaptic neurons), V (values): the CSC arrays of A, ordered by presynaptic neuron;
• J: the column (presynaptic neuron) of every stored entry;
• rowptr: column pointers of sparse(A'), i.e. row pointers of A;
• index: map from row-major position to CSC position, so that V[index[rowptr[i]:(rowptr[i+1]-1)]] are the entries of row i.

The returned arrays alias the internal arrays of A (colptr, I, V).

ensure_transpose!(c::AbstractConnection)
has_transpose(c::AbstractConnection)

Build the transposed index arrays c.J, c.index if missing (only a SpikingSynapse can lack them, see its docstring) and return c; has_transpose tells whether they are present. Call it before reading them in user code.

has_plasticity(synapse::AbstractConnection)

Whether the connection has an active plasticity rule (see set_plasticity!).

indices(c::AbstractConnection, js::AbstractVector, is::AbstractVector)

CSC positions (indices into c.W) of all existing synapses from presynaptic neurons in js to postsynaptic neurons in is.

matrix(c::AbstractConnection)
matrix(c::AbstractConnection, sym::Symbol)

Return the weights of the sparse connection c as a SparseMatrixCSC of size N_post x N_pre (rows: postsynaptic, columns: presynaptic). With sym, the matrix holds the per-synapse field sym of c instead of W (e.g. :ρ).

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 50); I = SNN.IF(N = 20)
EI = SNN.SpikingSynapse(E, I, :ge; conn = (p = 0.2, μ = 2.0))
W = SNN.matrix(EI)          # 20 x 50
size(W)

matrix(c::AbstractConnection, W, time::Number)
matrix(c::AbstractConnection, W, time::AbstractVector)

Sparse N_post x N_pre matrix of the interpolated record W (callable as W(synapses, time)) at time; for a vector of times, a 3-dimensional array. Used by matrix_record.

matrix_record(c::AbstractConnection, sym::Symbol, time::Number)
matrix_record(c::AbstractConnection, sym::Symbol, time::AbstractVector)

Rebuild the N_post x N_pre sparse matrix of the recorded per-synapse variable sym (e.g. :W, recorded with monitor!(c, [:W])) at time time (ms), interpolating the record (record(c, sym, range = true)). For a vector of times, the matrices are concatenated along the third dimension. time must lie within the recorded range (asserted). The sparsity pattern is the current one of c.

postsynaptic(c::AbstractConnection)
postsynaptic(c::AbstractConnection, j::Int)
postsynaptic(c::AbstractConnection, js::AbstractVector)

Postsynaptic neuron indices of the connection c: for every presynaptic neuron (first form), for neuron j, or for each neuron in js.

postsynaptic_idxs(c::AbstractConnection, j::Int)

Range of CSC positions of the synapses of presynaptic neuron j: c.colptr[j]:(c.colptr[j+1]-1). These index c.W, c.I and the other per-synapse arrays directly.

presynaptic(c::AbstractConnection)
presynaptic(c::AbstractConnection, i::Int)
presynaptic(c::AbstractConnection, is::AbstractVector)

Presynaptic neuron indices of the connection c: for every postsynaptic neuron (first form, a vector of vectors), for neuron i (second form) or for each neuron in is (third form).

Example

using SpikingNeuralNetworks
SNN.@load_units
E = SNN.IF(N = 50); I = SNN.IF(N = 20)
EI = SNN.SpikingSynapse(E, I, :ge; conn = (p = 0.2, μ = 2.0))
SNN.SNNModels.presynaptic(EI, 1)     # inputs of I neuron 1
SNN.SNNModels.postsynaptic(EI, 1)    # targets of E neuron 1

presynaptic_idxs(c::AbstractConnection, i::Int)

Range of positions, in the row-major (transposed) ordering, of the synapses onto postsynaptic neuron i: c.rowptr[i]:(c.rowptr[i+1]-1). These are not indices into c.W: the corresponding synapses are c.index[presynaptic_idxs(c, i)] (CSC positions), e.g. c.W[c.index[presynaptic_idxs(c, i)]] are the input weights of i.

remove_autapses!(w::SparseMatrixCSC)

Structurally remove the diagonal (self-connections) of a square connectivity matrix, without densifying it.

set_plasticity!(synapse::AbstractConnection, bool::Bool)
has_plasticity(synapse::AbstractConnection)

Switch / read the plasticity of a connection. For a sparse synapse with plasticity variables (SpikingSynapse) set_plasticity! sets the active flag of both LTPVars and STPVars (as set_LTP! and set_STP!), and has_plasticity returns whether a long-term or short-term rule is present and active. For other connections the active field of param is used if it exists; otherwise set_plasticity! is a no-op and has_plasticity returns false. (Up to SNNModels 1.8.4 both raised a FieldError for SpikingSynapse.)

sparse_matrix(Npre, Npost; p | ρ, μ = 1, σ = 0, dist = :Normal, rule = :Fixed, γ, kmin)

Random connectivity matrix of size Npost x Npre (rows: postsynaptic, columns: presynaptic) as a SparseMatrixCSC{FT,Int} (keyword FT = Float32), built directly in CSC form in O(nnz + Npre + Npost) time and memory.

Exactly one of p and ρ (connection probability / density, in [0, 1]) must be given. Unknown keywords are ignored with a warning; w is not used (pass a matrix as conn instead).

Connectivity rules (rule)

• :Fixed / :FixedIn (default): every postsynaptic neuron receives exactly K = Npre - round(Int, (1 - ρ) * Npre) inputs, sampled without replacement.
• :FixedOut: every presynaptic neuron projects to exactly K = Npost - round(Int, (1 - ρ) * Npost) targets, sampled without replacement.
• :Bernoulli: every pair is connected independently with probability ρ. Sampled by geometric skipping over the column-major index space (O(nnz) random draws).
• :PowerLaw: the out-degree of each presynaptic neuron is min(round(Int, rand(Pareto(γ, kmin))), Npost - 1); targets sampled without replacement.

Weights

One draw per connection from dist(|μ|, σ), where dist is a two-parameter Distributions type or its Symbol (default :Normal; for :LogNormal, μ and σ are the parameters of the logarithm), with Float32 parameters. Draws <= 0 are not synapses and are removed, so with σ > 0 the realised degrees can be lower than K; a negative μ flips the sign of all weights afterwards (with a warning). Weights are in the units of the target variable (e.g. nS).

Autapses are not removed here; SpikingSynapse removes them when pre == post.

Example

using SpikingNeuralNetworks
SNN.@load_units
w = SNN.SNNModels.sparse_matrix(100, 40; p = 0.1, μ = 2.0, σ = 0.5, rule = :Bernoulli)
size(w)   # (40, 100): Npost x Npre

The dense generator of SNNModels before 1.8.2 (another random stream, same statistics) is available as SNNModels.sparse_matrix_dense_legacy.

sparse_matrix(Npre, Npost, conn::NamedTuple, FT = Float32)
sparse_matrix(Npre, Npost, conn::AbstractMatrix, FT = Float32)

Connectivity of a connection constructor: a NamedTuple is splatted into the keyword form of sparse_matrix; a matrix (dense or sparse, any element type, size Npost x Npre, asserted) is converted to SparseMatrixCSC{FT,Int} as is (stored zeros of a sparse input are kept, explicit zeros of a dense input are dropped).

update_sparse_matrix!(c::AbstractConnection, W::SparseMatrixCSC)
update_sparse_matrix!(c::AbstractConnection)

Rebuild the double sparse storage of c (rowptr, colptr, I, J, index, W). The first form replaces it with the matrix W (same size, asserted; the number of synapses may change). The second form rebuilds it from the current c.I, presynaptic indices, c.W (used after synaptic_turnover! has changed postsynaptic indices), keeping the size of the matrix (length(c.rowptr) - 1 postsynaptic, length(c.colptr) - 1 presynaptic neurons).

The per-synapse arrays are resized and reordered with the synapses: the short-term efficacy ρ and, for delayed synapses, the delays param.delaytime. In the first form a synapse (i, j) that already existed keeps its values and a new synapse gets ρ = 1 and the mean delay; in the second form every stored synapse keeps its values (its CSC position may change). The plasticity variables are per neuron and are not affected. For a SpikingSynapse, an efficacy ρ that is not stored stays not stored, J/index are rebuilt only if they were present or a rule needs them, and equal delays are stored once.

update_weights!(c::AbstractConnection, j, i, w)
update_weights!(c::AbstractConnection, js::Vector, is::Vector, w::Real)

Set to w the weight of the existing synapse from presynaptic j to postsynaptic i (first form), or of every existing synapse from any j in js to any i in is (second form). Synapses that do not exist are not created (see connect!).

RateSynapse(pre, post; μ = 0.0, p = 0.0, kwargs...)

Sparse connection between rate populations (pre and post need a rate field r; the target is the field g of post, e.g. Rate).

The weight matrix is $W = \frac{\mu}{\sqrt{p N_{pre}}} X$ with $X$ a sparse N_post x N_pre matrix with density p and standard normal non-zeros (sprandn).

Transmission (forward!, every step of sim! and train!)

\[g_i \leftarrow g_i + \sum_j W_{ij}\, r_j\]

g is not reset by the synapse; the postsynaptic Rate/WilsonCowan resets it after using it, so g is the input of the current step.

Plasticity (plasticity!, only under train!)

For every presynaptic neuron $j$, with learning rate $\eta$ = param.lr:

\[\Delta_j = \eta \left(r_j - \sum_i r_i W_{ij}\right), \qquad W_{ij} \leftarrow W_{ij} + r_i\, \Delta_j\]

($r_i$ postsynaptic, $r_j$ presynaptic rate). Reference not given in the code (the docstring previously linked a Brian2 tutorial on synapses, which does not describe this rule).

Keyword arguments

• μ = 0.0: weight scale; p: connection density, must be in (0, 1] (an ArgumentError is raised otherwise; up to SNNModels 1.8.4 the default p = 0 produced a dense matrix of NaN).

RateSynapse stores only colptr, I, W (no rowptr, J, index), so the connectivity helpers that need them (matrix, presynaptic, ...) do not apply.

• kwargs...: forwarded to the struct (e.g. param = RateSynapseParameter(lr = 1e-4), name).

Example

using SpikingNeuralNetworks
SNN.@load_units
R = SNN.Rate(N = 100)
RR = SNN.RateSynapse(R, R; μ = 1.0, p = 0.2)
SNN.sim!([R], [RR]; duration = 10ms)

RateSynapseParameter(; lr = 1e-3)

Parameter of RateSynapse (and of the non-exported SpikeRateSynapse).

Fields

• lr::Float32 = 1e-3: learning rate of the weight update applied by plasticity! (dimensionless; the update is applied once per train! step, independent of dt).

forward!(c::RateSynapse, param::RateSynapseParameter)

Add $\sum_j W_{ij} r_j$ to the postsynaptic input c.g[i].

plasticity!(c::RateSynapse, param::RateSynapseParameter, dt, T)

Weight update of RateSynapse (see its docstring); called only by train!.

FLSynapse(pre, post; μ = 1.5, p = 0.0, α = 1, kwargs...)

Dense recurrent connection between rate populations trained with FORCE learning (recursive least squares on a linear readout fed back into the network).

Model

State: recurrent weights $W$ (N_post x N_pre), readout weights $w$, feedback weights $u$, running inverse correlation matrix $P$, readout $z$ and target $f$ (scalar field c.f, to be set by the user at every step).

forward!(c, param):

\[z = w^\top r^{post}, \qquad q = P\, r^{pre}, \qquad g = W\, r^{pre} + z\, u\]

(g is overwritten). plasticity!(c, param, dt, T):

\[C = \frac{1}{1 + q^\top r^{post}}, \qquad w \leftarrow w + C\,(f - z)\, q, \qquad P \leftarrow P - C\, q\, q^\top\]

Initialisation

• $W_{ij} = \mu\, \xi_{ij} / \sqrt{N_{pre}}$, $\xi_{ij} \sim \mathcal{N}(0, 1)$ (dense).
• $w_i \sim \mathcal{U}(-1, 1) / \sqrt{N_{post}}$, $u_i \sim \mathcal{U}(-1, 1)$.
• $P = \alpha\, \mathbb{1}$ (N_post x N_post), $z \sim 0.5\,\mathcal{N}(0, 1)$, $f = 0$.

Keyword arguments

• μ = 1.5: gain of the recurrent weights; p: unused (the matrix is dense); α = 1: initial value of the diagonal of $P$; kwargs...: forwarded to the struct.

Notes

• pre and post must have a rate field r, and post a target :g (e.g. Rate). $P$ is N_post x N_post and multiplies $r^{pre}$, so the connection only works for pre.N == post.N (recurrent use).
• sim! applies forward! every step; train! also applies the RLS plasticity!. Set c.f before each step (e.g. with a stimulus or a loop of one-step train! calls). Up to SNNModels 1.8.4 FLSynapseParameter was not a subtype of AbstractConnectionParameter and sim!/train! raised a MethodError.

References

Sussillo D, Abbott LF (2009). Generating coherent patterns of activity from chaotic neural networks. Neuron 63:544-557 (linked in the code).

Example

using SpikingNeuralNetworks
SNN.@load_units
R = SNN.Rate(N = 100)
F = SNN.FLSynapse(R, R; μ = 1.5, α = 1)
SNN.SNNModels.forward!(F, F.param)                 # one transmission step
F.f = 1.0f0                                        # target of the readout
SNN.SNNModels.plasticity!(F, F.param, 0.125f0, SNN.SNNModels.Time())

FLSynapseParameter()

Parameter of FLSynapse (no fields), a subtype of AbstractConnectionParameter: sim! calls forward!(c, param) through the generic four-argument method, train! also calls plasticity!.

forward!(c::FLSynapse, param::FLSynapseParameter)

Compute the readout c.z, the vector c.q = P r_pre and overwrite the target with g = W r_pre + z u.

plasticity!(c::FLSynapse, param::FLSynapseParameter, dt, T)

Recursive-least-squares update of the readout w and of P (see FLSynapse).

FLSparseSynapse(pre, post; μ = 1.5, p = 0.0, α = 1, kwargs...)

Sparse variant of FLSynapse (FORCE learning), not exported. Recurrent weights $W = \mu X / \sqrt{p N_{pre}}$ with $X$ = sprandn(N_post, N_pre, p); $P$ is stored only on the sparsity pattern of $W$ and initialised to $\alpha$ on the diagonal entries.

Update: as FLSynapse, with $q$ and $P$ restricted to the stored synapses. p must be in (0, 1] (an ArgumentError is raised otherwise).

References

Sussillo D, Abbott LF (2009). Neuron 63:544-557 (linked in the code).

FLSparseSynapseParameter()

Parameter of the non-exported FLSparseSynapse (no fields; a subtype of AbstractConnectionParameter).

PINningSynapse(pre, post; μ = 1.5, p = 0.0, α = 1, kwargs...)

Dense recurrent connection between rate populations trained with partial in-network training (PINning): the recurrent weights themselves are adjusted by recursive least squares so that the input of every postsynaptic unit follows its target $f_i$.

Model

forward!(c, param):

\[q = P\, r^{pre}, \qquad g = W\, r^{pre}\]

(g is overwritten). plasticity!(c, param, dt, T):

\[C = \frac{1}{1 + q^\top r^{post}}, \qquad W \leftarrow W + C\,(f - g)\, q^\top, \qquad P \leftarrow P - C\, q\, q^\top\]

c.f (vector of length N_post, initialised to zeros) holds the targets and is set by the user.

Initialisation

$W_{ij} = \mu\, \xi_{ij}/\sqrt{N_{pre}}$, $\xi_{ij}\sim\mathcal{N}(0,1)$; $P = \alpha\,\mathbb{1}$ (N_post x N_post); q = f = 0.

Keyword arguments

• μ = 1.5: gain; p: unused (dense); α = 1: initial diagonal of $P$; kwargs...: forwarded to the struct.

Notes

• pre and post must have a rate field r and post a target :g (e.g. Rate); only pre.N == post.N works.
• sim! applies forward!; train! also applies the RLS plasticity!. Up to SNNModels 1.8.4 the parameter type had no supertype and sim!/train! raised a MethodError.

References

Rajan K, Harvey CD, Tank DW (2016). Recurrent network models of sequence generation and memory. Neuron 90:128-142 (PubMed 26971945, linked in the code).

Example

using SpikingNeuralNetworks
SNN.@load_units
R = SNN.Rate(N = 100)
P = SNN.PINningSynapse(R, R; μ = 1.5)
SNN.SNNModels.forward!(P, P.param)
P.f .= 0.1f0
SNN.SNNModels.plasticity!(P, P.param, 0.125f0, SNN.SNNModels.Time())

PINningSynapseParameter()

Parameter of PINningSynapse (no fields), a subtype of AbstractConnectionParameter (so sim! and train! dispatch on it).

forward!(c::PINningSynapse, param::PINningSynapseParameter)

Compute c.q = P r_pre and overwrite the target with g = W r_pre.

plasticity!(c::PINningSynapse, param::PINningSynapseParameter, dt, T)

Recursive-least-squares update of W and P (see PINningSynapse).

PINningSparseSynapse(pre, post; μ = 1.5, p = 0.0, α = 1, kwargs...)

Sparse variant of PINningSynapse (not exported). $W = \mu X/\sqrt{p N_{pre}}$ with $X$ = sprandn(N_post, N_pre, p); $P$ lives on the sparsity pattern of $W$, initialised to $\alpha$ on stored diagonal entries.

forward!(c, param) resets g and q and accumulates, over stored synapses, $g_i = \sum_j W_{ij} r_j$ and $q_i = \sum_j P_{ij} r_j$. plasticity!(c, param, dt, T) with $C = 1/(1 + q^\top r^{post})$ updates every stored synapse: $P_{ij} \leftarrow P_{ij} - C q_i q_j$ and $W_{ij} \leftarrow W_{ij} + C (f_i - g_i) q_j$.

p must be in (0, 1] (an ArgumentError is raised otherwise). Up to SNNModels 1.8.4 the parameter type had no supertype, so sim!/train! did not dispatch on it.

References

Rajan K, Harvey CD, Tank DW (2016). Neuron 90:128-142 (PubMed 26971945, linked in the code).

PINningSparseSynapseParameter()

Parameter of the non-exported PINningSparseSynapse (no fields; a subtype of AbstractConnectionParameter).

SpikeRateSynapse(pre, post; μ = 0.0, p = 0.0, kwargs...)

Sparse connection from a spiking population to a rate population (not exported). $W = \mu X/\sqrt{p N_{pre}}$ with $X$ = sprandn(N_post, N_pre, p); p must be in (0, 1]. The target is post.g (see synaptic_target of Rate).

forward!(c, param, dt, T): for every presynaptic neuron $j$ that fired in this step, $g_i \leftarrow g_i + W_{ij}/dt$, i.e. each spike is a delta input that makes the state $x_i$ of the Rate unit jump by $W_{ij}$ (independent of dt; Rate resets g after each step).

The connection has no learning rule: plasticity! is a no-op (the RateSynapse rule needs a presynaptic rate, which a spiking population does not have).

forward!(c::SpikeRateSynapse, param::RateSynapseParameter, dt::Float32, T::Time)

For each presynaptic spike add $W_{ij}/dt$ to c.g[i] (a delta input of integral $W_{ij}$).