Does the recording's duration affect the quality of spike sorting?
Does the recording's duration affect the quality of spike sorting?¶
This notebook investigates if and how the duration of the recording affects spike sorting.
Obviously, each sorter engine needs a minimum number of events to detect a "cluster", and therefore a unit. If a neuron doesn't fire enough during a recording it won't be detected. The number of event per units depends on the recording duration and the each individual firing rates.
In order to test this phenomenon, we use the same dataset (with the same neurons and firing rates), but we vary the duration of the recording.
The simulated recording is generated with MEArec using a Neuronexus-32 probe. This specific dataset seems relatively easy to sort. The "SYNTH_MEAREC_NEURONEXUS" dataset in SpikeForest (which uses the same probe), in fact, shows quite good results for all sorters. The original duration is 600s (10 min).
Here we have generated a new but similar recording with a duration of 1800s. Then we have shortened it to 60s, 300s, 600s and 1800s (original). The recording can be downloaded from Zenodo: https://doi.org/10.5281/zenodo.4058272
The dataset name is: recordings_10cells_Neuronexus-32_1800.0_10.0uV_2020-02-28.h5. It contains 10 neurons recorded on a Neuronexus-32 probe. The duration is 1800s and the noise level is 10uV.
Let's see if spike sorters are robust to fewer events and if are able to deal with long durations or they end up finding too many events.
Author: Samuel Garcia, CRNL, Lyon
Requirements¶
For this need you will need the following Python packages:
- numpy
- pandas
- matplotlib
- seaborn
- spikeinterface
To run the MATLAB-based sorters, you would also need a MATLAB license. For other sorters, please refer to the documentation on how to install sorters.
Installation and imports¶
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from pathlib import Path
import spikeinterface as si
import spikeinterface.extractors as se
import spikeinterface.sorters as ss
import spikeinterface.widgets as sw
from spikeinterface.comparison import GroundTruthStudy
# clone and install MATLAB sorters
# kilosort2
!git clone https://github.com/MouseLand/Kilosort2.git
kilosort2_path = './Kilosort2'
ss.Kilosort2Sorter.set_kilosort2_path(kilosort2_path)
# kilosort
!git clone https://github.com/cortex-lab/KiloSort.git
kilosort_path = './KiloSort'
ss.KilosortSorter.set_kilosort_path(kilosort_path)
# ironclust
!git clone https://github.com/flatironinstitute/ironclust.git
ironclust_path = './ironclust'
ss.IronclustSorter.set_ironclust_path(ironclust_path)
%matplotlib inline
# some matplotlib hack to prettify figure
SMALL_SIZE = 12
MEDIUM_SIZE = 14
BIGGER_SIZE = 16
plt.rc('font', size=SMALL_SIZE) # controls default text sizes
plt.rc('axes', titlesize=SMALL_SIZE) # fontsize of the axes title
plt.rc('axes', labelsize=MEDIUM_SIZE) # fontsize of the x and y labels
plt.rc('xtick', labelsize=SMALL_SIZE) # fontsize of the tick labels
plt.rc('ytick', labelsize=SMALL_SIZE) # fontsize of the tick labels
plt.rc('legend', fontsize=SMALL_SIZE) # legend fontsize
plt.rc('figure', figsize=(10.0, 8.0)) # figsize
plt.rc('figure', titlesize=BIGGER_SIZE) # fontsize of the figure title
def clear_axes(ax):
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
Check spikeinterface version and sorter version¶
si.print_spikeinterface_version()
ss.print_sorter_versions()
Setup global path¶
# Change this path to point to where you downloaded the dataset
p = Path('/home/samuel/Documents/DataSpikeSorting/mearec/')
study_folder = p / 'study_mearec_neuronexus_several_durations/'
Setup ground truth study¶
mearec_filename = p / 'recordings_10cells_Neuronexus-32_1800.0_10.0uV_2020-02-28.h5'
rec = se.MEArecRecordingExtractor(mearec_filename, locs_2d=True)
gt_sorting = se.MEArecSortingExtractor(mearec_filename)
fs = rec.get_sampling_frequency()
gt_dict = {}
durations = [60, 300, 600, 1800]
for duration in durations:
sub_rec = se.SubRecordingExtractor(rec, start_frame=0, end_frame=int(duration*fs))
sub_sorting = se.SubSortingExtractor(gt_sorting, start_frame=0, end_frame=int(duration*fs))
gt_dict[f'rec{duration}'] = (sub_rec, sub_sorting)
study = GroundTruthStudy.create(study_folder, gt_dict)
Run all sorters¶
sorter_list = ['herdingspikes', 'ironclust', 'kilosort2', 'kilosort',
'mountainsort4', 'spykingcircus', 'tridesclous']
study = GroundTruthStudy(study_folder)
sorter_params = {}
study.run_sorters(sorter_list, sorter_params=sorter_params, mode='keep', verbose=True)
Get signal to noise ratio for all units¶
Units are the same in each recording so the snr is the same lets take from the longest one
study = GroundTruthStudy(study_folder)
snr = study.get_units_snr(rec_name='rec1800')
snr
fig, ax = plt.subplots()
ax.hist(snr['snr'].values, bins=10)
ax.set_xlabel('GT units SNR')
Run comparison with ground truth and retreive result tables¶
# this copy sorting is necessary to copy results from sorter
# into a centralize folder with all results
study.copy_sortings()
# this run all comparison sto GT
study.run_comparisons(exhaustive_gt=True, match_score=0.1, overmerged_score=0.2)
# this retrieve results
comparisons = study.comparisons
dataframes = study.aggregate_dataframes()
Run times¶
run_times = dataframes['run_times']
run_times
# insert durations
run_times['duration'] = run_times['rec_name'].apply(lambda s: float(s.replace('rec', '')))
g = sns.catplot(data=run_times, x='duration', y='run_time', hue='sorter_name', kind='bar')
Accuracy vs duration¶
perf = dataframes['perf_by_units']
perf
# insert durations
perf['duration'] = perf['rec_name'].apply(lambda s: float(s.replace('rec', '')))
g = sns.catplot(data=perf, x='duration', y='accuracy', hue='sorter_name', kind='bar')
Count good, bad, false positive units vs duration¶
count_units = dataframes['count_units']
count_units
# insert durations
count_units['duration'] = count_units['rec_name'].apply(lambda s: float(s.replace('rec', '')))
num_well_detected vs duration¶
the more the better
g = sns.catplot(data=count_units, x='duration', y='num_well_detected', hue='sorter_name', kind='bar')
num_false_positive vs duration¶
the less the better
g = sns.catplot(data=count_units, x='duration', y='num_false_positive', hue='sorter_name', kind='bar')
# same as previous but with other limits
g = sns.catplot(data=count_units, x='duration', y='num_false_positive', hue='sorter_name', kind='bar')
g.fig.axes[0].set_ylim(0, 10)
num_redundant vs duration¶
the less the better
g = sns.catplot(data=count_units, x='duration', y='num_redundant', hue='sorter_name', kind='bar')
Conlusion¶
For this simple simulated dataset we have observed that:
-
Focusing on the average accuracy, all sorters have similar performance for long or short recordings. The only exception is Kilosort: it has a clear drop in performence for the shortest duration (60s).
-
Very surprinsingly, some sorters (e.g. tridesclous, ironclust) have better performence at 60s than 300s. This could be specific to this dataset and have to be instigate more.
-
Looking at the number of "num_false_positive" and "num_well_detected" the situation is the following:
- kilosort is not affected by the duration
- herdingspikes (one of the most affected): the longer the duration, the more "num_false_positive"
- ironclust seems to have a slight increase in "num_false_positive" for longer duration
- kilosort2 has random oscillations of "num_false_positive" across durations
- tridesclous has a few more "num_false_positive" for long durations
- moutainsort is heavily affected by the duration
- spykingcircus is affected by long durations as more "num_false_positive" units are found