Saravanan666/pypi-org huggingface dataset:
Package: torchnmf
Non-negative Matrix Fatorization in PyTorchPyTorch is not only a good deep learning framework, but also a fast tool when it comes to matrix operations and convolutions on large data.
A great example isPyTorchWavelets.In this package I implement NMF, PLCA and their deconvolutional variations in PyTorch based ontorch.nn.Module,
so the models can be moved freely among CPU/GPU devices and utilize parallel computation of cuda.
We also utilize the computational graph fromtorch.autogradto derive updated coefficients so the amount of codes is reduced and easy to maintain.ModulesNMFBasic NMF and NMFD module minimizing beta-divergence using multiplicative update rules.The interface is similar tosklearn.decomposition.NMFwith some extra options.NMF: Original NMF algorithm.NMFD: 1-D deconvolutional NMF algorithm.NMF2D: 2-D deconvolutional NMF algorithm.NMF3D: 3-D deconvolutional NMF algorithm.PLCABasic PLCA and SIPLCA module using EM algorithm to minimize
KL-divergence between the target distribution and the estimated
distribution.PLCA: Original PLCA (Probabilistic Latent Component Analysis)
algorithm.SIPLCA: Shift-Invariant PLCA algorithm (similar to NMFD).SIPLCA2: 2-D deconvolutional SIPLCA algorithm.SIPLCA3: 3-D deconvolutional SIPLCA algorithm.UsageHere is a short example of decompose a spectrogram using deconvolutional NMF:importtorchimportlibrosafromtorchnmf.nmfimportNMFDfromtorchnmf.metricsimportkl_divy,sr=librosa.load(librosa.util.example_audio_file())y=torch.from_numpy(y)windowsize=2048S=torch.stft(y,windowsize,window=torch.hann_window(windowsize),return_complex=True).abs().cuda()S=S.unsqueeze(0)R=8# number of componentsT=400# size of convolution windownet=NMFD(S.shape,rank=R,T=T).cuda()# run extremely fast on gpunet.fit(S)# fit to target matrix SV=net()print(kl_div(V,S))# KL divergence to SA more detailed version can be foundhere.
See ourdocumentationto find out more usage of this package.Compare to sklearnThe barchart shows the time cost per iteration with different
beta-divergence. It shows that pytorch-based NMF has a much more constant process time across
different beta values, which can take advantage when beta is not 0, 1, or 2.
This is because our implementation use the same computational graph regardless which beta-divergence are we minimizing.
It runs even faster when computation is done on GPU. The test is conducted on a
Acer E5 laptop with i5-7200U CPU and GTX 950M GPU.Installationpip install torchnmfRequirementsPyTorchtqdmTipsIf you notice significant slow down when operating on CPU, please flush denormal numbers bytorch.set_flush_denormal(True).TODOSupport sparse matrix target (only onNMFmodule).Regularization.NNDSVD initialization.2/3-D deconvolutional module.PLCA.Documentation.ipynb examples.Refactor PLCA module.
Package: torchode
A Parallel ODE Solver for PyTorchtorchode is a suite of single-step ODE solvers such asdopri5ortsit5that are
compatible with PyTorch's JIT compiler and parallelized across a batch. JIT compilation
often gives a performance boost, especially for code with many small operations such as an
ODE solver, while batch-parallelization means that the solver can take a step of0.1for
one sample and0.33for another, depending on each sample's difficulty. This can avoid
performance traps for models of varying stiffness and ensures that the model's predictions
are independent from the compisition of the batch. See thepaperfor details.DocumentationIf you get stuck at some point, you think the library should have an example onxor you
want to suggest some other type of improvement, please open anissue on
github.InstallationYou can get the latest released version from PyPI withpipinstalltorchodeTo install a development version, clone the repository and install in editable mode:gitclonehttps://github.com/martenlienen/torchodecdtorchode
pipinstall-e.Usageimportmatplotlib.pyplotasppimporttorchimporttorchodeastodeff(t,y):return-0.5*yy0=torch.tensor([[1.2],[5.0]])n_steps=10t_eval=torch.stack((torch.linspace(0,5,n_steps),torch.linspace(3,4,n_steps)))term=to.ODETerm(f)step_method=to.Dopri5(term=term)step_size_controller=to.IntegralController(atol=1e-6,rtol=1e-3,term=term)solver=to.AutoDiffAdjoint(step_method,step_size_controller)jit_solver=torch.compile(solver)sol=jit_solver.solve(to.InitialValueProblem(y0=y0,t_eval=t_eval))print(sol.stats)# => {'n_f_evals': tensor([26, 26]), 'n_steps': tensor([4, 2]),# =>  'n_accepted': tensor([4, 2]), 'n_initialized': tensor([10, 10])}pp.plot(sol.ts[0],sol.ys[0])pp.plot(sol.ts[1],sol.ys[1])CitationIf you build upon this work, please cite the following paper.@inproceedings{lienen2022torchode,
  title = {torchode: A Parallel {ODE} Solver for PyTorch},
  author = {Marten Lienen and Stephan G{\"u}nnemann},
  booktitle = {The Symbiosis of Deep Learning and Differential Equations II, NeurIPS},
  year = {2022},
  url = {https://openreview.net/forum?id=uiKVKTiUYB0}
}
Package: torchopenl3
Torchopenl3Please cite the following TorchOpenL3 in your work:[1]Gyanendra Das, Humair Raj Khan, Joseph Turian (2021). torchopenl3 (version 1.0.1). DOI 10.5281/zenodo.5168808,https://github.com/torchopenl3/torchopenl3.TorchopenL3 is a pytorch port of theOpenL3Python library for computing deep audio embeddings.ContributorsPlease refer to theOpenl3 Libraryfor keras version.ComparisonWe run torchopenl3 over 100 audio files and compare with openl3 embeddings. Below is the MAE (Mean Absolute Error) TableContent_typeInput_reprEmd_sizeMAEEnvLinear5121.1522600237867664e-06EnvLinear61441.027089645617707e-06EnvMel1285121.2094695046016568e-06EnvMel12861441.0968088741947213e-06EnvMel2565121.1641358707947802e-06EnvMel25661441.0069775197507625e-06MusicLinear5121.173499645119591e-06MusicLinear61441.048712784381678e-06MusicMel1285121.1837427564387327e-06MusicMel12861441.0497348176841115e-06MusicMel2565121.1619711483490392e-06MusicMel25661449.881532906774738e-07InstallationInstall via pippip install torchopenl3Install the package with all dev libraries (i.e. tensorflow openl3)git clone https://github.com/turian/torchopenl3.git
pip3 install -e ".[dev]"Install Docker and work within the Docker environment.
Unfortunately this Docker image is quite big (about 4 GB) becausedocker pull turian/torchopenl3
# Or, build the docker yourself
#docker build -t turian/torchopenl3 .Using TorchpenL3To help you get started with TorchopenL3 please go through the colab file.AcknowledgeSpecial Thank you toJoseph Turianfor his help[1]Look, Listen and Learn More: Design Choices for Deep Audio EmbeddingsJason Cramer, Ho-Hsiang Wu, Justin Salamon, and Juan Pablo Bello.IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pages 3852–3856, Brighton, UK, May 2019.[2]Look, Listen and LearnRelja Arandjelović and Andrew ZissermanIEEE International Conference on Computer Vision (ICCV), Venice, Italy, Oct. 2017.Model Weights LicenseThe model weights are made available under
Package: torch-optimi
optimīFast, Modern, and Low Precision PyTorch Optimizersoptimi enables accurate low precision training via Kahan summation, supports fully decoupled weight decay, and features fast implementations of modern optimizers.Low Precision Training with Kahan Summationoptimi optimizers can match the performance of mixed precision whentraining in BFloat16 by using Kahan summation.Training in BFloat16 with Kahan summation can reduce non-activation training memory usage by37.5 to 45.5 percentwhen using an Adam optimizer. BFloat16 training increases single GPUtraining speed by ~10 percentat the same batch size.Fully Decoupled Weight DecayIn addition to supporting PyTorch-style decoupled weight decay, optimi optimizers also supportfully decoupled weight decay.Fully decoupled weight decay decouples weight decay from the learning rate, more accurately followingDecoupled Weight Decay Regularization. This can help simplify hyperparameter tuning as the optimal weight decay is no longer tied to the learning rate.Foreach ImplementationsAll optimi optimizers have fastforeach implementations, which can significantly outperform the for-loop versions. optimi reuses the gradient buffer for temporary variables to reduce foreach memory usage.Documentationhttps://optimi.benjaminwarner.devInstalloptimi is available to install from pypi.pipinstalltorch-optimiUsageTo use an optimi optimizer with Kahan summation and fully decoupled weight decay:importtorchfromtorchimportnnfromoptimiimportAdamW# create or cast model in low precision (bfloat16)model=nn.Linear(20,1,dtype=torch.bfloat16)# instantiate AdamW with parameters and fully decoupled weight decay# Kahan summation is automatically enabled since model & inputs are bfloat16opt=AdamW(model.parameters(),lr=1e-3,weight_decay=1e-5,decouple_lr=True)# forward and backward, casting input to bfloat16 if neededloss=model(torch.randn(20,dtype=torch.bfloat16))loss.backward()# optimizer stepopt.step()opt.zero_grad()To use with PyTorch-style weight decay with float32 or mixed precision:# create modelmodel=nn.Linear(20,1)# instantiate AdamW with parametersopt=AdamW(model.parameters(),lr=1e-3,weight_decay=1e-2)Differences from PyTorchoptimi optimizers do not support compilation, differentiation, complex numbers, or have capturable versions.optimi Adam optimizers do not support AMSGrad and SGD does not support Nesterov momentum. Optimizers which debias updates (Adam optimizers and Adan) calculate the debias term per parameter group, not per parameter.Optimizersoptimi implements the following optimizers:Adam,AdamW,Adan,Lion,RAdam,Ranger,SGD, &StableAdamW
Package: torchort
training pytorch models with onnxruntimeto build (you need to update version number in version.txt in order to be able to upload python whl):rm dist/*
python setup.py bdist_wheelto publish:twine upload dist/*to install:stable:(uninstall onnxruntime version that do not support gpu and training)
pip install torchort onnxruntime=1.9.0+cu111_trainingnightly:(make sure you are not in the ort repo folder - otherwise torchort is taken as already installed)
pip install --pre torchort ort-gpu-nightly-training

(eventually we are aiming at: pip install --pre torchort onnxruntime=1.9.0+cu111_training)to test:python ./ort/tests/bert_for_sequence_classification.pyto use torchort within PyTorch training scripts:import onnxruntime
from torchort import ORTModule
model = ORTModule(model)
# normal PyTorch training script follows
Package: torch-ort
The torch-ort packages uses the PyTorch APIs to accelerate PyTorch models using ONNX Runtime.DependenciesThe torch-ort package depends on the onnxruntime-training package, which depends on specific versions of GPU libraries such as NVIDIA CUDA.The default commandpip install torch-ortinstalls the onnxruntime-training version that depends on CUDA 10.2.If you have a different version of CUDA installed, you can install a different version of onnxruntime-training explicitly:CUDA 11.1pip install onnxruntime-training -f https://download.onnxruntime.ai/onnxruntime_stable_cu111.htmlPost-installation stepOnce torch-ort is installed, there is a post-installation step:python -m torch_ort.configureIf this step fails, it is likely due to GPU library version mismatch between onnxruntime-training and your installation. You can check the version of onnxruntime-training by runningpip list. For example:onnxruntime-training 1.9.0+cu111Releases1.9.0Release Notes :https://github.com/pytorch/ort/releases/tag/v1.9.01.8.1Release Notes :https://github.com/pytorch/ort/releases/tag/v1.8.1
Package: torch-ort-infer
OpenVINO™ Integration with Torch-ORTaccelerates PyTorch models using OpenVINO™ Execution Provider for ONNX Runtime. This product is designed for PyTorch developers who want to get started with OpenVINO™ in their inferencing applications. It delivers OpenVINO™ inline optimizations that enhance inferencing performance with minimal code modifications.OpenVINO™ Integration with Torch-ORT accelerates inference across many AI models on a variety of Intel® hardware such as:Intel® CPUsIntel® integrated GPUsIntel® Movidius™ Vision Processing Units - referred to as VPU.InstallationRequirementsUbuntu 18.04, 20.04Python 3.7, 3.8 or 3.9This package supports:Intel® CPUsIntel® integrated GPUsIntel® Movidius™ Vision Processing Units (VPUs).The torch-ort-infer package has dependency on the onnxruntime-openvino package that will be installed by default to run inference workloads. This onnxruntime-openvino package comes with pre-built libraries of OpenVINO™ version 2022.2.0 eliminating the need to install OpenVINO™ separately. The OpenVINO™ libraries are prebuilt with CXX11_ABI flag set to 0.For more details, please refer toOpenVINO™ Execution Provider for ONNX Runtime.Post-installation stepOnce torch-ort-infer is installed, there is a post-installation step:python -m torch_ort.configureUsageBy default, Intel® CPU is used to run inference. However, you can change the default option to either Intel® integrated GPU or Intel® VPU for AI inferencing. Invoke theprovider optionsto change the hardware on which inferencing is done.For more API calls and environment variables, seeUsage.SamplesFor quick start, explore thesamplesfor few HuggingFace and TorchVision models.LicenseOpenVINO™ Integration with Torch-ORT is licensed underMIT. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.SupportPlease submit your questions, feature requests and bug reports viaGitHub Issues.How to ContributeWe welcome community contributions to OpenVINO™ Integration with Torch-ORT. If you have an idea for improvement:Share your proposal viaGitHub Issues.Submit aPull Request.
Package: torch-ort-poc
training pytorch models with onnxruntimeto build (you need to increase version minor number in version.txt in order to upload python whl):rm dist/*
python setup.py bdist_wheelto publish:twine upload dist/*to install:pip install torch-ort-pocto use torch_ort within PyTorch training scripts:import onnxruntime
from torch_ort import ORTModule
model = ORTModule(model)
# normal PyTorch training script follows
Package: torchoutil
torchoutilCollection of functions and modules to help development in PyTorch.InstallationpipinstalltorchoutilThe only requirements arepython>=3.8andtorch>=1.10.UsageBatch of padded sequencesimporttorchfromtorchoutilimportmasked_meanx=torch.as_tensor([1,2,3,4])mask=torch.as_tensor([True,True,False,False])result=masked_mean(x,mask)# result contains the mean of the values marked as True: 1.5importtorchfromtorchoutilimportlengths_to_non_pad_maskx=torch.as_tensor([3,1,2])pad_mask=lengths_to_non_pad_mask(x,max_len=4)# tensor([[True, True, True, False],#         [True, False, False, False],#         [True, True, False, False]])Multilabel conversionsimporttorchfromtorchoutilimportprobs_to_namesprobs=torch.as_tensor([[0.9,0.1],[0.6,0.9]])names=probs_to_names(probs,threshold=0.5,idx_to_name={0:"Cat",1:"Dog"})# [["Cat"], ["Cat", "Dog"]]importtorchfromtorchoutilimportmultihot_to_indicesmultihot=torch.as_tensor([[1,0,0],[0,1,1],[0,0,0]])indices=multihot_to_indices(multihot)# [[0], [1, 2], []]...and more tensor manipulations!importtorchfromtorchoutilimportinsert_at_indicesx=torch.as_tensor([1,2,3,4])result=insert_at_indices(x,indices=[0,2],values=5)# result contains tensor with inserted values: tensor([5, 1, 2, 5, 3, 4])importtorchfromtorchoutilimportget_inverse_permperm=torch.randperm(10)inv_perm=get_inverse_perm(perm)x1=torch.rand(10)x2=x1[perm]x3=x2[inv_perm]# inv_perm are indices that allow us to get x1 from x3, i.e. x1 == x3 hereContactMaintainer:Étienne Labbé "Labbeti":labbeti.pub@gmail.com
Package: torch-parameter-groups
torch-parameter-groupsGroup PyTorch Parameters according to Rules.InstallationNeed Python 3.6+.pipinstalltorch-parameter-groupsUsageimporttorchimporttorch.nnasnnimporttorch_basic_modelsimporttorch_parameter_groupsmodel=torch_basic_models.MobileNetV2.factory()optimizer=torch_parameter_groups.optimizer_factory(model=model,config={'type':'SGD','kwargs':{'momentum':0.9,'nesterov':True,'weight_decay':0.0001,},'rules':[{'param_name_list':['weight'],'kwargs':{'weight_decay':0}},{}]},)criterion=nn.CrossEntropyLoss()output=model(torch.randn(1,3,224,224))loss=criterion(output,torch.Tensor([0]).long())loss.backward()optimizer.step(closure=None)
Package: torch-pba
torch-pbaPyTorch Implementation ofPBA.AnnData-centric.InstallationFromPYPI:pipinstalltorch-pbaAlternatively, install the development version from GitHub:gitclonehttps://github.com/mvinyard/torch-pba.git;cdtorch-pba;pipinstall-e.Example use:fromtorch_pbaimportPBAfromanndataimportread_h5adpba=PBA(adata=read_h5ad("./path/to/adata.h5ad"))pba.build_kNN()pba.compute_Laplacian()pba.compute_potential()pba.compute_fate_bias()pba.compute_mean_first_passage_time()Time to calculate Mean First Passage Time for theexample hematopoiesis datasetiscut from 4+ hours to <10 mins. In this example, I used a NVIDIA T4 GPU rented from GCP.See more:notebookOriginal work:GitHub:AllonKleinLab/PBAPaper:Weinreb et al., PNAS. DOI: 10.1073/pnas.1714723115. (2018).Note:I have not contributed any methodological novelty in this library. The original implementation contains the novel application of a Laplace transform to a kNN Graph to obtain a potential value, pseudotime, etc. Here, I have simply adapted the library to PyTorch/CUDA. No formal benchmarking has been performed.Contact / questions:mvinyard@broadinstitute.org
Package: torch-pconv
Torch PconvFaster and more memory efficient implementation of the Partial Convolution 2D layer in PyTorch equivalent to the
standard Nvidia implementation.This implementation has numerous advantages:It isstrictly equivalentin computation
tothe reference implementation by Nvidia. I made unit tests to assess that all throughout development.It's commented and more readableIt's faster and more memory efficient, which means you can use more layers on smaller GPUs. It's a good thing
considering today's GPU prices.It's a PyPI-published library. You canpipinstall it instead of copy/pasting source code, and get the benefit of (
free) bugfixes when someone notice a bug in the implementation.Getting startedpip3installtorch_pconvUsageimporttorchfromtorch_pconvimportPConv2dimages=torch.rand(32,3,256,256)masks=(torch.rand(32,256,256)>0.5).to(torch.float32)pconv=PConv2d(in_channels=3,out_channels=64,kernel_size=7,stride=1,padding=2,dilation=2,bias=True)output,shrunk_masks=pconv(images,masks)Performance improvementTestYou can
findthe reference implementation by Nvidia here.I tested their implementation vs mine one the following configuration:ParameterValuein_channels64out_channels128kernel_size9stride1padding3biasTrueinput height/width256The goal here was to produce the most computationally expensive partial convolution operator so that the performance
difference is displayed better.I compute both the forward and the backward pass, in case one consumes more memory than the other.Resultstorch_pconvNvidia® (Guilin)Forward only813 466 6244 228 120 576Backward only1 588 201 4801 588 201 480Forward + Backward2 405 797 6406 084 757 512DevelopmentTo install the latest version from Github, run:git clone git@github.com:DesignStripe/torch_pconv.git torch_pconv
cd torch_pconv
pip3 install -U .
Package: torch-pdegraph
Torch_pdegraphTorch_pdegraph is a proof of concept that how one can solve PDEs (partial difference equations)  on graphs using the Message Passing class of torch_geometric and hence also benefits from the hardware acceleration. See thevideo-presentationWhat is a PDE on a graph?The basic idea is that one can define the operators like, derivatives, gradients, laplacians on graphs and construct a PDE inspired from nature on graphs.
To know more about PDEs on graph.See the publications ofElmoataz.See also theirapplications on pointcloudsClassical PDEs on images byGuillermo SapiroBe sure to play with thejupyter-notebooks in the applications/folder which presents few of their applications. Download thedataRef to operator_calculus.md for a brisk intro to calculus on graphs.InstallationFirstinstallthe torch_geometric. Then one can clone this project and install it locally:pipinstall--upgradepip
pipinstall.Or do:pipinstall--upgradepip
pipinstalltorch_pdegraphRunning the notebooksIn the notebooks I am demonstrating few applications of pdes on images and pcd by creating simple knn-graphs on gpu. One will needfaisslibrary to create the graphs.To display the pcds inside the notebook I am using jupyter visualization feature in open3d which uses a jupyter widget, notebooks must be running to for the widget to function.To do:Add an interpolation application.Add a segmentation predefined pde.
Package: torch-pesq
Loss function inspired by the PESQ scoreImplementation of the widely used Perceptual Evaluation of Speech Quality (PESQ) score as a torch loss function. The PESQ loss alone performs not good for noise suppression, instead combine with scale invariantSDR. For more information see1,2.InstallationTo install the package just run:$pipinstalltorch-pesqUsageimporttorchfromtorch_pesqimportPesqLosspesq=PesqLoss(0.5,sample_rate=44100,)mos=pesq.mos(reference,degraded)loss=pesq(reference,degraded)print(mos,loss)loss.backward()Comparison to reference implementationThe following figures uses samples from the VCTK1speech and DEMAND2noise dataset with varying mixing factors. They illustrate correlation and maximum error between the reference and torch implementation:The difference is a result from missing time alignment implementation and a level alignment done with IIR filtering instead of a frequency weighting. They are minor and should not be significant when used as a loss function. There are two outliers which may degrade results and further investigation is needed to find the source of difference.Validation improvements when used as loss functionValidation results for fullband noise suppression:Noise estimator: RecurrentSRUwith soft masking. 8 layers, width of 512 result in ~1586k parameters of the unpruned model.STFT for signal coding: 512 window length, 50% overlap, hamming windowMel filterbank with 32 Mel featuresThe baseline system uses L1 time domain loss. Combining the PESQ loss function together with scale invariantSDRgives improvement of ~0.1MOS for PESQ and slight improvements in speech distortions, as well as a more stable training progression. Horizontal lines indicate the score of noisy speech.Relevant referencesEnd-to-End Multi-Task Denoising for joint SDR and PESQ OptimizationA Deep Learning Loss Function Based on the Perceptual Evaluation of the Speech QualityP.862 : Perceptual evaluation of speech quality (PESQ)Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecsCSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning ToolkitThe Diverse Environments Multi-channel Acoustic Noise Database (DEMAND): A database of multichannel environmental noise recordings
Package: torchph
torchphThis repository containsPyTorchextensions tocomputepersistent homology and todifferentiatethrough the persistent homology computation.
The packaging structure is similar to PyTorch's structure to facilitate usage for people
familiar with PyTorch.DocumentationRead the docs!The foldertutorials(withindocs) contains some (more or less) minimalistic examples in form of Jupyter notebooks
to demonstrate how to use thePyTorchextensions.Associated publicationsIf you use any of these extensions, please cite the following works (depending on which functionality you use, obviously :)@inproceedings{Hofer17a,author={C.~Hofer,R.~Kwitt,M.~NiethammerandA.~Uhl},title={DeepLearningwithTopologicalSignatures},booktitle={NIPS},year={2017}}@inproceedings{Hofer19a,author={C.~Hofer,R.~Kwitt,M.~DixitandM.~Niethammer},title={Connectivity-OptimizedRepresentationLearningviaPersistentHomology},booktitle={ICML},year={2019}}@article{Hofer19b,author={C.~Hofer,R.~Kwitt,andM.~Niethammer},title={LearningRepresentationsofPersistenceBarcodes},booktitle={JMLR},year={2019}}@inproceedings{Hofer20a},author={C.~Hofer,F.~Graf,R.~Kwitt,B.~RieckandM.~Niethammer},title={GraphFiltrationLearning},booktitle={arXiv},year={2020}}@inproceedings{Hofer20a,author={C.~Hofer,F.~Graf,M.~NiethammerandR.~Kwitt},title={TopologicallyDensifiedDistributions},booktitle={arXiv},year={2020}}