How Neuralink Analyzes Your Brain
By: Frederik Bussler
Tags: Machine Learning, Artificial Intelligence, Elon Musk, Neuralink, AI

Neuralink version 0.9 is an electronic implant made up of tiny wires (silicon carbide microwires) that enter your brain, each containing many electrodes, for a total of 1,024 channels. Prior to Neuralink, the industry benchmark for an implantable brain-machine interface was the Utah array (a silicon microelectrode array), containing just 100 channels.

The Utah array looks like a set of large boxes on your head, invasively implanted by a medical expert, while the Neuralink is invisible and auto-installed by a robot in under an hour.

Richard A. Normann’s US Patent #5,215,088. Three-dimensional electrode device. Public domain.

Writing

Each of the Neuralink’s channels has simultaneous read/write capability, potentially reaching a thousand to ten thousand neurons per electrode, such that millions of neurons could be written to. Future versions of Neuralink are expected to be orders of magnitude better. Let’s imagine that Neuralink v4 has 100,000 channels — it could then theoretically write to up to a billion neurons.

Reading

Neurons communicate by propagating “action potentials,” also known as “neural spikes.” The 1,024 microelectrodes in Neuralink v0.9 can measure these neural spikes. The Neuralink’s custom ASIC then does spike sorting to separate the activity of neurons, including spike detection and assigning spikes to specific neurons. Presumably, they use AI to track channels with neural data (and discard noise channels) and then automatically find characteristic shapes.

Exploring Current Models

Since we can’t look into how Neuralink’s AI models work under the hood, let’s see how public neural AI models, like SpikeDeeptector, actually function. At a high level, three steps were used to create SpikeDeeptector:

Collect and label spike data.

Train a DNN to detect and categorize channels (as neural or noise).

Do inference (run the model online on patients with microelectrode implants).

SpikeDeeptector used data from tetraplegic patients implanted with the aforementioned Utah array and patients implanted with depth electrodes in preparation for epileptic surgery. The patients with microwires had 8–16 electrodes, while those with dual Utah arrays had 196 electrodes (two 10x10 grids with the four corner electrodes left unconnected in one array). The Utah array electrodes were just about a millimeter long, with 400 µm spacing between electrodes. For the tetraplegic patients, data was collected during two to four in-person study sessions every week, over two years.

Analog electrical activity collected by the electrodes was amplified and digitized at a sampling rate of 30 KHz, and spike events were extracted with a threshold procedure. Detected waveforms consisted of 48 samples with a time duration of 1.6ms, containing 15 samples before and 32 samples after the threshold.

SpikeDeeptector casts spike classification as a supervised learning problem, such that ground-truth labels are needed to train the model. In one session, a channel records hundreds or even thousands of raw waveforms. With 136 recording sessions and 8 sessions, there were 31.21 million raw waveforms corresponding both to neural events and external artifacts.

Data was labeled in a semi-automated way, like so:

PCA was applied to all detected waveforms in a given channel.

The first two principal components were visualized on a subset of waveforms, capturing the direction of the highest variability.

A Gaussian Mixture Model was deployed on data in the PCA space to assign them to clusters.

The number of clusters were manually defined. In cases with visually unsatisfactory clustering, waveforms were manually labeled.

At this stage, only batch classification was done, such that if all concatenated waveforms were labeled as non-neural activities, then that batch was classified as an artifact, while if there was at least one neural spike label, then that batch was classified as a neural event.

With 31.21 million labeled waveforms and a batch of size 20, there were 1.56 million labeled feature vectors. The data was then split into train-test groups, and two model variations were built: A Convolutional Neural Network (CNN) and an FNN (Fully-connected Neural Network).

Non-novel architectural details, regularization, optimization, and hyperparameter tuning were employed with both variations.

Interestingly, both variations were significantly more accurate on data collected from patients with microwires (as opposed to those with microarrays), even though the microwire patients had an around an order of magnitude fewer channels. The top-performing accuracy was 98.9%, achieved with the CNN on microwire patients.

DetectCS

Another library is DetectCS, short for detecting complex spikes. This is a more niche library, as a “complex spike” refers to a large, fast electrical event, followed by a slow tail of smaller amplitude spikes.

This is a particularly interesting library, as it proposes a novel approach to complex spike detection. Complex spikes are so rare that, prior to this approach, accurate detection required human visual inspection.

The library enables easy analysis of electrophysiological signals. For instance, below we see a raw broad-band signal of a cerebellar Purkinje cell sampled at 25 KHz.

Looking at the “high-passed” electrophysiological signal reveals spikes. A “high-pass filter” is applied to filter out slow frequencies, typically under 0.1 Hz, and thus reduce noise.

Below, we can see an example of a detected complex spike, characteristic of the aforementioned large, fast spike followed by a slow tail.

As with SpikeDeeptector, a Convolutional Neural Network is used, while the Alpha Omega Engineering Multi Spike Detector (MSD) was used for real-time sorting of simple spikes in the data collection process for DetectCS.

Wave Clus

Another option for spike detection and sorting is wave clus, a Matlab algorithm. To give an example of its usage, I’ll follow this tutorial, using a dataset of continuous human single-neuron recordings.

The data I downloaded is a .ncs file, which is just the format used by the recording system manufacturer. Following Neuralynx’ data formatting instructions, we can import the data like so:

header_size = 16 * 1024 # Header has 16 kilobytes length

fid = open('CSC4.Ncs', 'rb')

fid.seek(header_size) # Skip header by shifting position by header size

data_format = np.dtype([('TimeStamp', np.uint64),

('ChannelNumber', np.uint32),

('SampleFreq', np.uint32),

('NumValidSamples', np.uint32),

('Samples', np.int16, 512)])

raw = np.fromfile(fid, dtype=data_format)

fid.close() # Close file

sf = raw['SampleFreq'][0] # Get sampling frequency

data = raw['Samples'].ravel() # Create data vector

As we can see, there’s a SampleFreq (datapoints recorded per second) and NumValidSamples , which means we can plot the signal over time with a time vector. First, let’s create the time vector.

dur_sec = data.shape[0]/sf

time = np.linspace(0, dur_sec,data.shape[0])

Here’s our data so far:

data.shape[0] is 61,000,192 rows of data

is 61,000,192 rows of data sf is 32,051 datapoints per second (Hertz)

is 32,051 datapoints per second (Hertz) dur_sec (the length of the sample) is therefore 1,903 seconds

(the length of the sample) is therefore time is our numpy array for plotting

Let’s plot just the first second using the time vector.

fig, ax = plt.subplots(figsize=(15, 5))

ax.plot(time[0:sf], data[0:sf])

ax.set_title('32 KHz Broadband Sampling'.format(sf), fontsize=23)

ax.set_xlim(0, time[sf])

ax.set_xlabel('Time in seconds')

ax.set_ylabel('Amplitude in microvolts')

plt.show()

There’s a clear cyclical pattern, so at first glance we may be able to deduce something interesting. However, if we count the number of spikes, we’ll find that there are 60 spikes in one second — 60 Hz — which is equal to the US power supply frequency. What we’re seeing is simply electrical noise.

While fascinating, we want to find electrical signals — action potentials just 1 to 2 milliseconds long — not noise. To do so, we’ll filter the raw broadband signal from 500 to 9,000 Hz (excluding the 60 Hz noise).

from scipy.signal import butter, lfilter

def filter_data(data, low=500, high=9000, order=2):

nyq = sf/2 # Determine Nyquist frequency

low = low/nyq # Setting bands

high = high/nyq

b, a = butter(order, [low, high], btype='band') # Calculate coefficients

filtered_data = lfilter(b, a, data) # Filter signal

return filtered_data

Then, we’ll call our function to create our new data.

data = filter_data(data)

Finally, we can run the same graphing code as before.

There’s no longer any 60 Hz oscillation, and just around half a second in there’s a clear spike. Since we’ve done it over 1 second, we can do it for the whole 1,903 second dataset, by simply scaling our process. We’ll find datapoints above a certain threshold, define a window on these events to highlight them, and align them at their peak altitude.

First, we define the get_spikes function.

Now, we can plot the waveforms.

This rejects frequencies below a low threshold, such as standard electrical noise, and frequencies above a high threshold, such as larger electrical artifacts like a lightswitch being turned. It shows 100 random spikes extracted from the signal.

It looks like there may be two types of waveforms — one group with a sharp higher amplitude peak, slightly earlier on, and the other with a broader, lower amplitude peak. These were likely generated by more than one neuron, so we could also group the waveforms into different clusters.

Here’s an example of three clustered waveforms from the GitHub for the full tutorial, using k-means clustering applied on the principle components of data.

Summary

We’ve explored a few methods of spike analysis, which is an important field of study in neuroscience. Consider the complexity of Neuralink’s spike analysis.

A year ago, the Neuralink recorded 20,000 samples per second, with 10bit resolution, equal to 200,000 bits per second (200 Mbps / 200 kilobits) per chip. At 1,024 channels, that’s around 204,800 Mbps, compressed 200x by on-chip algorithms, down to, coincidentally, 1,024 Mbps, or 1 Gbps. With a Bluetooth low-energy radio, that’s way too much data to send off the chip, so spike detection and sorting is done on-chip as well, such that much smaller info is sent off-chip.

We’re only scratching the surface, and Neuralink’s evolution will surely continue on a path of breakthroughs.

Gain Access to Expert View — Subscribe to DDI Intel