Skill: lhs183019/spikecv-helper (version 1.0.2)

--- name: spikecv-helper description: Help AI Agents answer questions and execute tasks for SpikeCV, an ultra-high-speed spike camera vision framework. Use when the user asks about spike cameras, SpikeCV repository, spike dataset download, running vision task like tracking, reconstruction through spike vision, etc. --- # SpikeCV Agent Skill *last updated: 2026-04-28* This document is designed to equip an AI Agent (or developer) with the necessary prior knowledge and instructions to answer questions and execute tasks within the SpikeCV repository. Any command execution or conceptual question should be informed by the context in this document and the attached references. > **Before reading**: If you're unfamiliar with spike cameras or SpikeCV's architecture, first read `references/About_SpikeCV.md`. --- ## 🧭 Quick Lookup Use this table to jump to the right section. If it's not here, the answer isn't in this skill. | You want... | Go to... | |---|---| | What is SpikeCV? | `references/About_SpikeCV.md` | | Install SpikeCV | [§ Installation](#-installation) | | SpikeCV API docs | [spikecv.readthedocs.io](https://spikecv.readthedocs.io) | | SpikeCV Project directory structure | [§ Project Layout](#-project-layout) | | Run `spikecv` CLI (download, track, reconst) | [§ CLI Reference](#-cli-reference) | | Full publication list (up-to-date) | [spikecv.github.io/publications.html](https://spikecv.github.io/publications.html) | | Algorithm and it's theory | search for `docs/source/核心操作.rst` 🇨🇳 or source code under `SpikeCV/spkProc/` 🇬🇧 `SpikeCV/examples` 🇬🇧, you might get it wrong, so notify user to look up the details in publication for exact theory | | SpikeCV event, competition, Team contact, contribution, hardware | `references/About_SpikeCV.md` | | Dataset descriptions | `references/About_SpikeCV.md` | | Troubleshooting | [§ Gotchas & Common Errors](#-gotchas--common-errors) | **Not in this skill?** → Defer to the user or suggest SpikeCV [website](https://spikecv.github.io) and SpikeCV team contact ([REDACTED_SECRET]). --- ## 📦 Installation **Prerequisites**: - Python ≥ 3.10 - Python virtual environment (**Conda** recommended): see [install_miniconda.md](references/install_miniconda.md) for a step-by-step setup guide from scratch. ```bash git clone https://github.com/Zyj061/SpikeCV cd SpikeCV pip install .[cli] ``` Optional extras: ```bash pip install .[tracking] # for SNNTracker/SpikeSORT ``` **Verify**: ```bash spikecv --help ``` **Quick health check** (downloads ~40 MB and runs TFSTP reconstruction): ```bash spikecv data download --dataset recVidarReal2019 --agent-used cd datasets/ spikecv proc reconst --agent-used ``` --- ## 🛠 CLI Reference SpikeCV provides the `spikecv` command for wrapper interactions. After installation, you can use it to download datasets and run processing algorithms. All commands accept `--agent-used` to return structured JSON output. **Always include this flag.** > **Note**: Feel free to explore other CLI options and parameters as needed. The following examples only showcase partial subcommands and parameters, the CLI may support additional features or algorithms. Always refer to `--help` for the most up-to-date command options and usage instructions. ### `spikecv data download` Download a dataset from the OpenI platform: ```bash spikecv data download --dataset <name> --local-dir <path> --agent-used ``` | Dataset | For Task | |---|---| | `recVidarReal2019` | Reconstruction | | `motVidarReal2020` | Tracking | **⚠️ Path gotcha**: Data lands in `<local-dir>/<dataset>/`. Default local-dir is `datasets/`, so your dat files end up at `datasets/recVidarReal2019/classA/car-100kmh.dat`, NOT `recVidarReal2019/classA/car-100kmh.dat`. ### `spikecv proc reconst` Reconstruct visible images from a raw `.dat` spike stream. Default algorithm: **TFSTP**. ```bash # From the datasets/ directory: cd datasets/ spikecv proc reconst --dat-file-path recVidarReal2019/classA/car-100kmh.dat --agent-used # Or with an absolute/explicit path: spikecv proc reconst --dat-file-path /path/to/recVidarReal2019/classA/car-100kmh.dat \ --yaml-file-path /path/to/recVidarReal2019/config.yaml \ --agent-used ``` **Key parameters**: - `--dat-file-path`: Path to `.dat` spike stream file - `--yaml-file-path`: Path to `config.yaml` for the dataset (default: `recVidarReal2019/config.yaml`) - `--begin-idx`: Starting frame index (default: 500) - `--block-len`: How many spike frames to process (default: 1500) - `--stp-d`, `--stp-F`, `--stp-f`: STP model parameters **⚠️ Path gotcha**: The default `yaml-file-path` is `recVidarReal2019/config.yaml` (relative to cwd). If you download the dataset using the CLI, it lives under `datasets/recVidarReal2019/`. You must either `cd datasets/` first or pass explicit paths. ### `spikecv proc track` Run multi-object tracking on spike data. Default algorithm: **SNNTracker**. ```bash spikecv proc track --scene-idx 0 --metrics --agent-used ``` **Key parameters**: - `--scene-idx`: Scene index 0–6 - `--metrics`: (Optional) Calculate MOTA/IDF1 metrics (requires ground truth labels) **⚠️ Dataset**: Tracking uses `motVidarReal2020`, not `recVidarReal2019`. --- ## 📁 Project Layout ``` SpikeCV/ ├── SpikeCV/ │ ├── cli/ # CLI command definitions (typer) │ ├── device/ # Hardware driver for spike cameras │ ├── examples/ # Standalone test scripts (**NOT for agent use**, use CLI instead) │ ├── metrics/ # Quantitative evaluation (PSNR, SSIM, MOTA, IDF1, AEPE) │ ├── spkData/ # Data loaders + config.yaml parsing │ │ └── load_dat.py # ParaDict generation, SpikeStream class │ ├── spkProc/ # ⭐ Core algorithms │ │ ├── filters/ # STP filter (background removal) │ │ ├── reconstruction/ # TFI, TFP, TFSTP, SSML │ │ ├── tracking/ # SNNTracker, SpikeSORT │ │ ├── detection/ # STDP, Motion-based detection │ │ ├── recognition/ # RPSNet, SVM, VGG │ │ ├── depth_estimation/ # SpikeT (Transformer) │ │ ├── optical_flow/ # SCFlow │ │ └── augment/ # Data augmentation │ ├── utils/ # Path helpers │ └── visualization/ # Video generation from spikes ├── docs/ │ ├── spike_algo.md # deprecated │ ├── data_processing.md # deprecated │ ├── tools.md # deprecated │ ├── examples.md # deprecated │ └── source/ # 🇨🇳 Sphinx sources (readthedocs) ├── Publications.md # 📄 Publication list (may be stale — use website instead) ├── README.md / README_en.md └── CONTRIBUTING.md / CONTRIBUTING_en.md ``` ### 🌐 Documentation by Language | File(s) | Language | Best for | |---|---|---| | `docs/source/*.rst` | 🇨🇳 Chinese | API, little Algorithm theory, code examples | | `SpikeCV/spkProc/**/*.py` (source code) | 🇬🇧 English | Algorithm math + implementation details | | `README_en.md`, `CONTRIBUTING_en.md` | 🇬🇧 English | Setup, contribution workflow | | `Publications.md` | 🇬🇧 English | Publication list (stale — see website) | --- ## 🚨 Rules (SHOULD NOT DO) These are not suggestions — following them prevents errors and hallucinations: 1. **Use CLI, not example scripts** — If a task has a `spikecv` CLI command, use it. Do NOT run `SpikeCV/examples/test_snntracker.py` or similar standalone scripts directly. 2. **Always include `--agent-used`** — Otherwise the CLI prints log text, not structured JSON. 3. **Do not modify SpikeCV code** — Suggest the user contact the team instead ([REDACTED_SECRET]). 4. **Do not hallucinate external resources** — For contacts, purchases, or contributions, only use info from `references/About_SpikeCV.md`. 5. **For publication/paper queries → always use the website** — `Publications.md` in the repo may be outdated. Point to [spikecv.github.io/publications.html](https://spikecv.github.io/publications.html) as the canonical source. --- ## 🐛 Gotchas & Common Errors | Symptom | Cause | Fix | |---|---|---| | `"Data path '...config.yaml' does not exist"` | `--yaml-file-path` is relative but you're not in the right directory | `cd datasets/` first, or pass full path | | `spikecv proc track` fails | Wrong dataset — using `recVidarReal2019` instead of `motVidarReal2020` | Download correct dataset first | | `--scene-idx` out of range | `motVidarReal2020` only has scenes 0–6 | Use a value in [0, 6] | | `--scene-idx` in range but still broken | some of the scene data might not have uploaded by the developer, check carefully if the corresponding scene appear in the downloaded dataset | don't use the scene that's not downloaded | | `spikecv` not found | CLI not installed or not in PATH | Run `pip install .[cli]` from SpikeCV root | | `--agent-used` not working (no JSON output) | Using an older SpikeCV version | Update to latest, or check `spikecv --help` for flag name |

path: references/About_SpikeCV.md
content: # More About SpikeCV

*last updated: 2026-04-22*

**SpikeCV** is an open-source project for `spike camera`, mainly including spiking datasets, algorithms library and hardware interface.

---

## Introduction

### spike camera

Mimicking the sampling mechanism of the fovea of primate retina, a retina-inspired camera, named **spike camera**, outputs asynchronous binary spike streams by continuously capturing photons. The photon accumulation reaches a preset threshold, a spike stream is generated. Generally, 1 means a spike is fired, and 0 means no spike. The difference from the traditional camera imaging. The traditional digital camera records scenes with a constant shutter speed (e.g., 33 ms), which loses much visual information and suffers motion blur. The spike camera can record the external information with a sampling rate of 40 Khz, and retains high-speed spatio-temporal information for both moving and static objects, which can be applied to tasks as acquisition, detection, tracking and recognition of ultra-high-speed scenes.

### Architecture of SpikeCV

SpikeCV mainly consists of three parts：

1. **Datasets**：
   - `Raw Spikes`: Off-line files of spike streams that generated by spike cameras or simulators.
   - `HFR Videos`: High-Frame-Rate  (HFR) Videos at the same sampling frequency as the spike camera,  which can be used to test the ultra-high-speed scene reconstruction algorithm based on the spike camera
   - `Optical Flow`: Ground truth optical flow information of the simulated scenes
   - `Stereo Depth`: Depth information of scenes captured by binocular and monocular spike cameras
   - `Object Labels`: Class labels of objects in the scene
   - `Bounding Boxes`: Candidate boxes represent the location and size of moving/foreground objects, which can be used for tasks such as detection and tracking
2. **Algorithm Library**：
   - `SpkData`: Mainly used to load data generated by the spike camera, Ground Truth (GT) information corresponding to spike stream, and simulate spikes generation
   - `SpkProc`: Models and algorithms for spike cameras, including *Filters*, *Reconstruction algorithm*, *Optical Flow (OF) Estimation*, *Depth Estimation*, *Object Detection*, *Multi-target Tracking* and *Recognition* and other modules
   - **Tool**：
     - `Visualization` —— Visualization of spike flow or results of various tasks
     - `Metrics` —— Quantitative Evaluation Metrics for the algorithms
     - `Utils` —— Auxiliary development tools, such as data path processing, etc.
3. **Device Interface**: Providing spike camera driver and supporting data acquisition, real-time data processing interface. Supported for the currently released spike cameras, or use offline files for simulated camera data debugging.

> **Further Reading:** For a comprehensive overview, see the SpikeCV team's publication: [*SpikeCV: open a continuous computer vision era*](https://link.springer.com/article/10.1007/s11432-023-4565-6).


## Resources

* **Official webpage** of SpikeCV: [spikecv](https://spikecv.github.io/index.html) . You would need it if:
   + You want to know more or get in touch with the SpikeCV team
   + You wanna check out the spikecv-related events or competitions hold by SpikeCV team
   + You wanna learn more about SpikeCV and spike camera, including a further understanding of the project architecture, research topics, and their publications

* **Joining and Contributing** SpikeCV community: you should check out the contributing workflow on the [webpage](https://spikecv.github.io/join.html) 

* **Publications**: If you want further links to **publications and researches** conducted by SpikeCV team, check out [this](https://spikecv.github.io/publications.html). **This official publication list is a comprehensive and up-to-date source**; any other lists found elsewhere may be incomplete or outdated. You would need this when confronting some difficult question about certain algorithm that required in-depth reading of the Paper (you can further user request to the publication listed out in this webpage).

* **Code Repository**: You can check out the `spikecv` python library from [**github repo**](https://github.com/Zyj061/SpikeCV), where integrate many classical implementation of Algorithm and provided Device interface and Data processing Modules that help you read spike data and perform visual task on spike data.

* **Documentation**: To learn more about the SpikeCV library, please refer to the **documentation** at [readthedocs](https://spikecv.readthedocs.io) . You would need it if:
+ You want to know the API of a certain function/class/method provided in `spikecv` library, which can be downloaded through GitHub
+ You want to take a look at some code example of the `spikecv` library 

* **Datasets**: You might want to check out **spike datasets** provided by the team, which can be download directly from [OpenI](https://openi.pcl.ac.cn/Cordium/SpikeCV/datasets) . You can also take a look at the datasets they provided for the competitions WUJI_INF (which hold in IJCAI2025) at [OpenI](https://openi.pcl.ac.cn/IJCAl2025)

* **Hardware Acquisition**: If you wonder "where to buy a spike camera?", you should check out [spikesee](https://spikesee.com/), a company producing high quality **spike camera**.


## Citing SpikeCV

If you find SpikeCV useful, please cite as following:

```bib
@misc{SpikeCV,
    title = {SpikeCV},
    author = {Zheng, Yajing and Zhang, Jiyuan and Zhao Rui and Ding, Jianhao and Li, Jianing and Chen, Shiyan and Yu, Zhaofei, Huang, Tiejun and other contributors},
    year = {2022},
    howpublished = {\url{https://git.openi.org.cn/Cordium/SpikeCV.git}},
}
```

If you have any questions, please contact the team through the following email address:

* [REDACTED_SECRET]

path: references/install_miniconda.md
content: # How to Install Conda (through Miniconda)

## Linux / macOS (bash)

```bash
# Detect architecture and download appropriate Miniconda installer
ARCH=$(uname -m)
OS=$(uname -s)

if [ "$OS" = "Linux" ]; then
    if [ "$ARCH" = "x86_64" ]; then
        MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh"
    elif [ "$ARCH" = "aarch64" ]; then
        MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh"
    else
        echo "Unsupported Linux architecture: $ARCH"
        exit 1
    fi
elif [ "$OS" = "Darwin" ]; then
    if [ "$ARCH" = "x86_64" ]; then
        MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh"
    elif [ "$ARCH" = "arm64" ]; then
        MINICONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh"
    else
        echo "Unsupported macOS architecture: $ARCH"
        exit 1
    fi
else
    echo "Unsupported OS: $OS"
    exit 1
fi

# Download installer (prefer wget, fallback to curl)
if command -v wget > /dev/null; then
    wget "$MINICONDA_URL" -O ~/miniconda.sh
elif command -v curl > /dev/null; then
    curl -L "$MINICONDA_URL" -o ~/miniconda.sh
else
    echo "Neither wget nor curl found. Please install one."
    exit 1
fi

# Silent install to $HOME/miniconda
bash ~/miniconda.sh -b -p $HOME/miniconda

# Initialize conda (adds conda to PATH and enables `conda activate`)
# NOTE: YOU MUST RESTART YOUR TERMINAL after this step
~/miniconda/bin/conda init

# Alternative to restarting: Reload shell configuration
if [ -f ~/.bashrc ]; then source ~/.bashrc; fi
if [ -f ~/.zshrc ]; then source ~/.zshrc; fi

# Verify installation
conda --version
```

## Windows (PowerShell, run as normal user)

```powershell
# Download Miniconda installer
$url = "https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe"
$installer = "$env:TEMP\miniconda.exe"
Invoke-WebRequest -Uri $url -OutFile $installer

# Silent install for current user (no admin required)
# Note: /D defines the custom installation path and MUST be the last argument
Start-Process -Wait -FilePath $installer -ArgumentList "/InstallationType=JustMe /RegisterPython=0 /S /D=$env:USERPROFILE\miniconda3"

# Remove installer
Remove-Item $installer

# Initialize conda (adds to PATH and enables conda in PowerShell)
& "$env:USERPROFILE\miniconda3\Scripts\conda.exe" init powershell

# RESTART POWERSHELL to apply changes, then verify:
# conda --version
```

## Post-installation: Create an environment

```bash
# Create a new environment with a specific Python version
conda create -n myenv python=3.10 -y

# IMPORTANT: You may need to run `conda activate myenv` 
# If `conda activate` fails, use `source activate myenv` or restart your shell.
conda activate myenv

# Verify Python version
python --version
```

## Notes

- The `-b` flag in the bash installer runs silently (batch mode), accepting the license.
- `-p $HOME/miniconda` sets the installation path.
- `conda init` modifies your shell configuration file (`.bashrc`, `.zshrc`, or PowerShell profile). You **must** restart your terminal or manually source the file for changes to take effect.
- To update conda itself after installation: `conda update -n base conda -y`

## Troubleshooting

- **Conda command not found:** Ensure you restarted your terminal. If it still fails, check if the installation path was correctly added to your `~/.bashrc` or your system PATH.
- **Permission Denied:** Ensure you have write permissions to `$HOME`. No `sudo` is required for this installation method.
- **SSL/Network Errors:** If you are in a restricted network, consider using a mirror or checking proxy settings.


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)