Package: torch-hrp
torch-hrpHashed Random Projection layer for PyTorch.UsageHashed Random Projections (HRP), binary representations, encoding/decoding for storage(notebook)Generate a HRP layer with a new hyperplaneThe random projection or hyperplane is randomly initialized.
The initial state of the PRNG (random_state) is required (Default: 42) to ensure reproducibility.importtorch_hrpasthrpimporttorchBATCH_SIZE=32NUM_FEATURES=64OUTPUT_SIZE=1024# demo inputsinputs=torch.randn(size=(BATCH_SIZE,NUM_FEATURES))# instantiate layerlayer=thrp.HashedRandomProjection(output_size=OUTPUT_SIZE,input_size=NUM_FEATURES,random_state=42# Default: 42)# run itoutputs=layer(inputs)assertoutputs.shape==(BATCH_SIZE,OUTPUT_SIZE)Instiantiate HRP layer with given hyperplaneimporttorch_hrpasthrpimporttorchBATCH_SIZE=32NUM_FEATURES=64OUTPUT_SIZE=1024# demo inputsinputs=torch.randn(size=(BATCH_SIZE,NUM_FEATURES))# use an existing hyperplanemyhyperplane=torch.randn(size=(NUM_FEATURES,OUTPUT_SIZE))# init layerlayer=thrp.HashedRandomProjection(hyperplane=myhyperplane)# run itoutputs=layer(inputs)Multiprocessing on GPU-ServerTheHashedRandomProjectionlayer has methods for multiprocessing of large numbers of examples for inference purposes (i.e. millions). These methods were adopted from theSentenceTransformer code.
With the following script can be used to figure out how many example fit into the the RAM (e.g. 20 Mio.),
and how big the chunk of example for each process can be to fit into the GPU memory (e.g. 4.5 Mio.)importtorchimporttorch_hrpasthrpmodel_hrp=thrp.HashedRandomProjection(output_size=1024,input_size=768,# the output dimension of the upstream embedding/transformer modelrandom_state=42)# Requirements: 2x GPUs w 80 Gb; approx 200 Gb RAMif__name__=='__main__':# multiprocessing spawning requires mainx=torch.rand(int(20e6),768)# 20 Mio examplespool=model_hrp.start_pool()hashed=model_hrp.infer(x,pool,chunk_size=int(45e5))# chunks of 4.5 Mio examplesmodel_hrp.stop_pool(pool)torch.cuda.empty_cache()seedemo/multiprocessing-on-gpu-server.pyAppendixInstallationThetorch-hrpgit repois available asPyPi packagepipinstalltorch-hrp
pipinstallgit+ssh://git@github.com/ulf1/torch-hrp.gitInstall a virtual environment (CPU)python3-mvenv.venvsource.venv/bin/activate
pipinstall--upgradepip
pipinstall-rrequirements.txt--no-cache-dir
pipinstall-rrequirements-dev.txt--no-cache-dir
pipinstall-rrequirements-demo.txt--no-cache-dir(If your git repo is stored in a folder with whitespaces, then don't use the subfolder.venv. Use an absolute path without whitespaces.)Install with conda (GPU)condainstall-ypip
condacreate-y--namegpu-venv-torch-hrp-devpython=3.9pip
condaactivategpu-venv-torch-hrp-dev

condainstall-ycudatoolkit=11.3.1cudnn=8.3.2-cconda-forgeexportLD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
pipinstalltorch==1.12.1+cu113torchvisiontorchaudio-fhttps://download.pytorch.org/whl/torch_stable.html# install other packagespipinstall-e.# pip install -r requirements.txt --no-cache-dirpipinstall-rrequirements-dev.txt--no-cache-dir
pipinstall-rrequirements-demo.txt--no-cache-dirPython commandsJupyter for the examples:jupyter labCheck syntax:flake8 --ignore=F401 --exclude=$(grep -v '^#' .gitignore | xargs | sed -e 's/ /,/g')Run Unit Tests:PYTHONPATH=. pytestPublish# pandoc README.md --from markdown --to rst -s -o README.rstpythonsetup.pysdisttwineupload-rpypidist/*Clean upfind.-typef-name"*.pyc"|xargsrm
find.-typed-name"__pycache__"|xargsrm-r
rm-r.pytest_cache
rm-r.venvSupportPleaseopen an issuefor support.ContributingPlease contribute usingGithub Flow. Create a branch, add commits, andopen a pull request.AcknowledgementsThe "Evidence" project was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) -433249742(GU 798/27-1; GE 1119/11-1).Maintenancetill 31.Aug.2023 (v0.1.1) the code repository was maintained within the DFG project433249742since 01.Sep.2023 (v0.2.0) the code repository is maintained by@ulf1.CitationPlease cite the arXiv Preprint when using this software for any purpose.@misc{hamster2023rediscovering,
      title={Rediscovering Hashed Random Projections for Efficient Quantization of Contextualized Sentence Embeddings}, 
      author={Ulf A. Hamster and Ji-Ung Lee and Alexander Geyken and Iryna Gurevych},
      year={2023},
      eprint={2304.02481},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}