Package: facemap
FacemapFacemap is a framework for predicting neural activity from mouse orofacial movements. It includes a pose estimation model for tracking distinct keypoints on the mouse face, a neural network model for predicting neural activity using the pose estimates, and also can be used compute the singular value decomposition (SVD) of behavioral videos.Please find the detailed documentation atfacemap.readthedocs.io.To learn about Facemap, read thepaperor check out the tweetthread. For support, please open anissue.For latest released version (from PyPI) including svd processing only, runpip install facemapfor headless version orpip install facemap[gui]for using GUI. Note:pip install facemapnot yet available for latest tracker and neural model, instead install withpip install git+https://github.com/mouseland/facemap.gitCITATIONIf you use Facemap, please cite the Facemappaper:Syeda, A., Zhong, L., Tung, R., Long, W., Pachitariu, M.*, & Stringer, C.* (2022). Facemap: a framework for modeling neural activity based on orofacial tracking.bioRxiv.
[bibtex]If you use the SVD computation or pupil tracking components, please also cite our previouspaper:Stringer, C.*, Pachitariu, M.*, Steinmetz, N., Reddy, C. B., Carandini, M., & Harris, K. D. (2019). Spontaneous behaviors drive multidimensional, brainwide activity.Science, 364(6437), eaav7893.
[bibtex]The MATLAB version of the GUI is no longer supported (see olddocumentation).LogoLogo was designed by Atika Syeda andTzuhsuan Ma.InstallationIf you have an olderfacemapenvironment you can remove it withconda env remove -n facemapbefore creating a new one.If you are using a GPU, make sure its drivers and the cuda libraries are correctly installed.Install anAnacondadistribution of Python. Note you might need to use an anaconda prompt if you did not add anaconda to the path.Open an anaconda prompt / command prompt which hascondaforpython 3in the pathCreate a new environment withconda create --name facemap python=3.8. We recommend python 3.8, but python 3.9 and 3.10 will likely work as well.To activate this new environment, runconda activate facemapTo install the minimal version of facemap, runpython -m pip install facemap.To install facemap and the GUI, runpython -m pip install facemap[gui]. If you're on a zsh server, you may need to use ' ' around the facemap[gui] call: `python -m pip install 'facemap[gui]'.To upgrade facemap (packagehere), run the following in the environment:python-mpipinstallfacemap--upgradeNote you will always have to runconda activate facemapbefore you run facemap. If you want to run jupyter notebooks in this environment, then alsopip install notebookandpython -m pip install matplotlib.You can also try to install facemap and the GUI dependencies from your base environment using the commandpython-mpipinstallfacemap[gui]If you haveissueswith installation, see thedocsfor more details. You can also use the facemap environment file included in the repository and create a facemap environment withconda env create -f environment.ymlwhich may solve certain dependency issues.If these suggestions fail, open an issue.GPU version (CUDA) on Windows or LinuxIf you plan on running many images, you may want to install a GPU version oftorch(if it isn't already installed).Before installing the GPU version, remove the CPU version:pip uninstall torchFollow the instructionshereto determine what version to install. The Anaconda install is strongly recommended, and then choose the CUDA version that is supported by your GPU (newer GPUs may need newer CUDA versions > 10.2). For instance this command will install the 11.3 version on Linux and Windows (note thetorchvisionandtorchaudiocommands are removed because facemap doesn't require them):conda install pytorch==1.12.1 cudatoolkit=11.3 -c pytorchand this will install the 11.7 toolkitconda install pytorch pytorch-cuda=11.7 -c pytorchSupported videosFacemap supports grayscale and RGB movies. The software can process multi-camera videos for pose tracking and SVD analysis. Please seeexample moviesfor testing the GUI. Movie file extensions supported include:'.mj2','.mp4','.mkv','.avi','.mpeg','.mpg','.asf'For more details, please refer to thedata acquisition page.SupportFor any issues or questions about Facemap, pleaseopen an issue. Please find solutions to some common issues below:Download of pretrained modelsThe models will be downloaded automatically from our website when you first run Facemap for processing keypoints. If download of pretrained models fails, please try the following:to resolve certificate error try:pip install –upgrade certifi, ordownload the pretrained model files:model stateandmodel parametersand place them in themodelssubfolder of the hiddenfacemapfolder located in home directory. Path to the hidden folder is:C:\Users\your_username\.facemap\modelson Windows and/home/your_username/.facemap/modelson Linux and Mac.Running FacemapTo get started, run the following command in terminal to open the GUI:python -m facemapClick "File" and load a single video file ("Load video"), or click "Load multiple videos" to choose a folder from which you can select movies to run. The video(s) will pop up in the left side of the GUI. You can zoom in and out with the mouse wheel, and you can drag by holding down the mouse. Double-click to return to the original, full view.Next you can extract information from the videos like track keypoints, compute movie SVDs, track pupil size etc. Also you can load in neural activity and predict it from these extracted features.I. Pose trackingFacemap provides a trained network for tracking distinct keypoints on the mouse face from different camera views (some examples shown below). Check thekeypointsbox then clickprocess. Next a bounding box will appear -- focus this on the face as shown below. Then the processed keypoints*.h5file will be saved in the output folder along with the corresponding metadata file*.pkl.Keypoints will be predicted in the selected bounding box region so please ensure the bounding box focuses on the face. See example frameshere.For more details on using the tracker, please refer to theGUI Instructions. Check out thenotebookfor processing keypoints in colab.📢 User contributions 📹 📷Facemap aims to provide a simple and easy-to-use tool for tracking mouse orofacial movements. The tracker's performance for new datasets could be further improved by expand our training set. You can contribute to the model by sharing videos/frames on the following email address(es):asyeda1[at]jh.eduorstringerc[at]janelia.hhmi.org.II. ROI and SVD processingFacemap allows pupil tracking, blink tracking and running estimation, see more detailshere. Also, Facemap can compute the singular value decomposition (SVD) of ROIs on single and multi-camera videos. SVD analysis can be performed across static frames called movie SVD (movSVD) to extract the spatial components or over the difference between consecutive frames called motion SVD (motSVD) to extract the temporal components of the video. The first 500 principal components from SVD analysis are saved as output along with other variables.You can draw ROIs to compute the motion/movie SVD within the ROI, and/or compute the full video SVD by checkingmultivideo. Then checkmotSVDand/ormovSVDand clickprocess. The processed SVD*_proc.npy(and optionally*_proc.mat) file will be saved in the output folder selected.For more details seeSVD python tutorialorSVD MATLAB tutorial.(videowith old install instructions)III. Neural activity predictionFacemap includes a deep neural network encoding model for predicting neural activity or principal components of neural activity from mouse orofacial pose estimates extracted using the tracker or SVDs.The encoding model used for prediction is described as follows:Please see neural activity predictiontutorialfor more details.