Specksim is a high performance Spiking-Convolutional Neural Network simulator which is written in C++ and bound to our backend library
samna. It simulates the SNN completely event-based and is a good representation of how our
DYNAP-CNN architecture hardware processes events. It is designed to help users who do not have access to our chips to test their networks to see how they would perform if they were deployed on the chip. Furthermore, the network architectures that normally could not have been deployed to our hardware due to memory constrains can be tested in
specksim. Specksim package is completely bound to Python, which means that any and all of its features can be used easily from Python.
Specksim is implemented partly in
sinabs-dynapcnn and partly in
samna. If this library is installed via
pip both of these libraries will be present. Therefore, no additional packages are necessary.
specksim simulator only supports sequential models. Typically the container module we use is
torch.nn.Sequential, however nested modules are also supported.
Each weight layer,
torch.nn.Linear should be followed by a spiking layer
The output layer of the network has to be a
This simulation supports two weight layer, namely
torch.nn.Linear. Any other parameter layer will be discarded. Linear layer will be converted into a Conv2d layer keeping all its features.
not supported. We currently do not have a way to simulate biases as they behave on our hardware.
The simulation supports two pooling layers, namely
sinabs.layers.SumPool2d. Any other pooling layer will be discarded. AvgPool2d layer will be converted to a SumPool2d layer, and the weights of the following parameter layer will be scaled down based on the kernel size of the pooling layer. Padding is not supported. Stride is considered to be equal to the kernel size for simulating the on-chip behaviour.
sinabs.layers.IAFSqueeze layers are supported.
LIF neurons are not supported.
Specksim expects events in the format of
np.record arrays. The record array has to have the following 4 keys.
y are the coordinates,
t stands for timestamps and
p stands for polarity (or in the context of non DVS input channels). The four keys are converted in
np.uint32 format. Output format which can be seen in the next cell can be used for the input format as well.
event_format = np.dtype([("x", np.uint32), ("y", np.uint32), ("t", np.uint32), ("p", np.uint32)])
import torch.nn as nn from sinabs.from_torch import from_model from sinabs.backend.dynapcnn.specksim import from_sequential
Define an artificial neural network#
ann = nn.Sequential( nn.Conv2d(1, 20, 5, 1, bias=False), nn.ReLU(), nn.AvgPool2d(2, 2), nn.Conv2d(20, 32, 5, 1, bias=False), nn.ReLU(), nn.AvgPool2d(2, 2), nn.Conv2d(32, 128, 3, 1, bias=False), nn.ReLU(), nn.AvgPool2d(2, 2), nn.Flatten(), nn.Linear(128, 500, bias=False), nn.ReLU(), nn.Linear(500, 10, bias=False), )
You can then load weights.
Convert to SNN using sinabs#
snn = from_model(ann, add_spiking_output=True, batch_size=1).spiking_model
Now that a
sinabs spiking neural network is defined, we can convert this model to a
SpecksimNetwork using the method
Convert sequential SNNs to SpecksimNetwork#
input_shape = (1, 28, 28) # MNIST shape specksim_network_dynapcnn = from_sequential(snn, input_shape=input_shape)
Please note that the input shape of the network has to be passed explicitly.
Convert from sequential DynapcnnNetwork to SpecksimNetwork#
In order to do a more realistic simulation, the sequential SNN network can be quantized by converting it into a
DynapcnnNetwork object, then the quantized
DynapcnnNetwork can be converted into the
input_shape = (1, 28, 28) # MNIST shape dynapcnn_network = DynapcnnNetwork(snn=snn, input_shape=input_shape, dvs_input=False, discretize=True) # the dynapcnn_network weights are quantized as we passed discretize=True specksim_network_dynapcnn = from_sequential(dynapcnn_network, input_shape=input_shape)
Send events to the simulated SNN#
Now that the network is created we can send events and see if we receive any.
x = 10 # in pixels y = 10 # in pixels t = 100_000 # typically in microseconds p = 0 input_event = np.array([x,y,t,p], dtype=specksim_network.output_dtype) output_event = specksim_network(input_event) print(output_event)
Specksim is mainly designed for benchmarking network performances in an event-driven way. In benchmarking we typically reset the spiking layer states. This can be achived by the following.
Drawbacks and possible questions#
This simulator is inference only and it does not support training.
Biases for weight layers, namely
torch.nn.Linear are not supported. The biases should be set to
False explicitly in the network before conversion.
Breadth-first vs Depth-first#
DYNAP-CNN architecture is completely asynchronous. This means that for a sequential model, each event comes after the event that created it from the previous layer. That means the hardware processes the events in a
depth-first manner. This simulation however processes events layer-by-layer. If there are multiple events received by a layer. That layer first finishes processing these events and then adds them to a queue, so the next layer can start their processing. This is done to make the simulation more efficient and faster. Furthermore, for models that were trained in a rate-based manner, the change in the processing scheme does not create too big of a difference.
TLDR; The chip processes events in
depth-first manner, whereas due to implementation efficiency
specksim processes events in
Our hardware with the
DYNAP-CNN architecture are completely asynchronous. Therefore, event timestamps do not play any role in calculation in the chip itself. The output events however do have timestamps, which are handled by the development boards. The timestamps depend on the load on the cores and therefore in a simulation cannot be replicated with a lot of success. Therefore, in this simulation for output events we assign the timestamp of the input event that led to its creation.
It is not possible to reliably run spiking neural network in real-time using
specksim. Although event-based processing is implemented completely in C++ for any complex architecture with a lot of parameters and fan-out there will be certain delays.
Try it yourself#
An example of running a converted
SNN trained on
specksim can be found under