ctn-waterloo/nengo_spinnaker — pull request #78 by mossblaser (2014-06-20 05:41:43+00:00)
Title: Basic Filter Routing Lookup-table Optimisation

Work done in collaboration with @neworderofjamie.

This branch adds basic optimisation for the lookup table used to direct arriving packets to the appropriate filter banks. This optimisation step is both limited in scope and also relies on knowledge of the format of routing keys used which is not available at the point of optimisation. Despite these limitations it is important for networks with large ensembles (see motivation below).

We have both attempted to justify to ourselves that the optimiser is correct and have tested the new code with another non-trivial model (Chris' letter-drawing model) where things appeared to continue to work. I've also tested it on some of the example scripts which also continue to work.

# Motivation

These changes are the result of a six hour bug tracking mission by myself and @neworderofjamie for which three hours can be squarely blamed on ST's flood-fill region specification format which is as epicly badass as it is undocumented.

This optimisation is required to allow Lisanne Huurdeman to run her grid cell model on SpiNNaker. This network essentially consists of a single recurrently-connected 47-dimensional ensemble with an enormous number of neurons which must be split across a number of cores. In a "small" case which fails the ensemble has 6,400 neurons. This is split across roughly 8 chips using approximately all 16 cores on each.

Without optimisation, this resulted in tables many tens of kilobytes in size which exhaust the memory of a core causing it to crash in turn causing loading to fail as it checks for chips being successfully loaded. This causes PACMAN to crash in a place which reveals only the flood-fill region it last loaded in the debug messages. Much reverse engineering later and we found the cores which had been loaded and which crashed and diagnosed the memory exhaustion.

After our simple optimisation pass in the best case only one entry is required per chip which connects to the ensemble. Tests on Lisanne's network will take place tomorrow.

A simplified example of this problem (requires 48-node board) is given below:

import numpy as np
import nengo

class NullSolver(nengo.decoders.Solver):
"""A decoder solver which simply does nothing.
"""

def __init__(self):
self.weights = False

def __call__(self, A, Y, rng=None):
N = A.shape[1]
D = Y.shape[1]
return np.zeros((N,D)), {}

model = nengo.Network(label="Large ensemble bug test")

with model:
null_solver = NullSolver()
# Problem known to kick-in somewhere around 6000 neurons
a = nengo.Ensemble(12000, 47, label="a")
nengo.Connection(a, a, solver=null_solver)

import nengo_spinnaker
sim = nengo_spinnaker.Simulator(model)

# All that matters is that it starts
sim.run(1)