Saravanan666/pypi-org huggingface dataset:
Package: deepmatch
DeepMatchDeepMatch is a deep matching model library for recommendations & advertising. It's easy totrain modelsand toexport representation vectorsfor user and item which can be used forANN search.You can use any complex model withmodel.fit()andmodel.predict().Let'sGet Started!orRun examples!Models ListModelPaperFM[ICDM 2010]Factorization MachinesDSSM[CIKM 2013]Deep Structured Semantic Models for Web Search using Clickthrough DataYoutubeDNN[RecSys 2016]Deep Neural Networks for YouTube RecommendationsNCF[WWW 2017]Neural Collaborative FilteringSDM[CIKM 2019]SDM: Sequential Deep Matching Model for Online Large-scale Recommender SystemMIND[CIKM 2019]Multi-interest network with dynamic routing for recommendation at TmallCOMIREC[KDD 2020]Controllable Multi-Interest Framework for RecommendationContributors(welcome to join us!)​​Shen Weichen​Alibaba Group​Wang Zhe​Baidu Inc.​​​Chen Leihui​Alibaba Group​​LeoCaiByteDance​​​Li YuanTencent​​​Yang JieyuAnt Group​​​Meng YifanDeepCTR​DisscussionGroupGithub DiscussionsWechat Discussions公众号：浅梦学习笔记微信：deepctrbot学习小组加入主题集合
Package: deepmatcher
DeepMatcher is a Python package for performing entity and text matching using deep learning.
It provides built-in neural networks and utilities that enable you to train and apply
state-of-the-art deep learning models for entity matching in less than 10 lines of code.
The models are also easily customizable - the modular design allows any subcomponent to be
altered or swapped out for a custom implementation.As an example, given labeled tuple pairs such as the following:DeepMatcher uses labeled tuple pairs and trains a neural network to perform matching, i.e., to
predict match / non-match labels. The trained network can then be used to obtain labels for
unlabeled tuple pairs.Paper and DataFor details on the architecture of the models used, take a look at our paperDeep
Learning for Entity Matching(SIGMOD ‘18). All public datasets used in
the paper can be downloaded from thedatasets page.Quick Start: DeepMatcher in 30 secondsThere are four main steps in using DeepMatcher:Data processing: Load and process labeled training, validation and test CSV data.importdeepmatcherasdmtrain,validation,test=dm.data.process(path='data_directory',train='train.csv',validation='validation.csv',test='test.csv')Model definition: Specify neural network architecture. Uses the built-in hybrid
model (as discussed in section 4.4 ofour paper) by default. Can
be customized to your heart’s desire.model=dm.MatchingModel()Model training: Train neural network.model.run_train(train,validation,best_save_path='best_model.pth')Application: Evaluate model on test set and apply to unlabeled data.model.run_eval(test)unlabeled=dm.data.process_unlabeled(path='data_directory/unlabeled.csv',trained_model=model)model.run_prediction(unlabeled)InstallationWe currently support only Python versions 3.5+. Installing using pip is recommended:pip install deepmatcherTutorialsUsing DeepMatcher:Getting Started: A more in-depth guide to help you get familiar with the basics of
using DeepMatcher.Data Processing: Advanced guide on what data processing involves and how to
customize it.Matching Models: Advanced guide on neural network architecture for entity matching
and how to customize it.Entity Matching Workflow:End to End Entity Matching: A guide to develop a complete entity
matching workflow. The tutorial discusses how to use DeepMatcher withMagellanto
perform blocking, sampling, labeling and matching to obtain matching tuple pairs from two
tables.DeepMatcher for other matching tasks:Question Answering with DeepMatcher: A tutorial on how to use DeepMatcher for question
answering. Specifically, we will look atWikiQA, a benchmark dataset for the task of
Answer Selection.API ReferenceAPI docsare here.SupportTake a look atthe FAQfor common issues. If you run into any issues or have questions not answered in the FAQ,
pleasefile GitHub issuesand we will address them asap.The TeamDeepMatcher was developed by University of Wisconsin-Madison grad students Sidharth Mudgal
and Han Li, under the supervision of Prof. AnHai Doan and Prof. Theodoros Rekatsinas.
Package: deepmatch-torch
DeepMatch-TorchDeepMatch-Torchis aPyTorchversion ofDeepMatch.DeepMatch-Torchis a PyTorch Version deep matching model library for recommendations & advertising. It's easy totrain modelsand toexport representation vectorsfor user and item which can be used forANN search. You can use any complex model withmodel.fit()andmodel.predict(). And you can keep same habit of using DeepMatch.Let'sGet Started!orRun examples!1. InstallInstalldeepmatch-torchbypippipinstalldeepmatch-torchInstalldeepmatch-torchthrough source codegitclonehttps://github.com/bbruceyuan/DeepMatch-Torch.gitcdDeepMatch-Torch
pythonsetup.pyinstall2. TutorialYou can runexampleby following steps.cdexamples
pythonrun_youtubednn.pyorrun_fm_dssm.py3. Models ListModelPaperFM[ICDM 2010]Factorization MachinesDSSM[CIKM 2013]Deep Structured Semantic Models for Web Search using Clickthrough DataYoutubeDNN[RecSys 2016]Deep Neural Networks for YouTube RecommendationsNCF[WWW 2017]Neural Collaborative FilteringMIND[CIKM 2019]Multi-interest network with dynamic routing for recommendation at TmallTODOsimplify model config. now only support kwargs, but config is a elegant choice.fixMINDonly support CPU train bug.AcknowledgmentsEspecially thanks toDeepMatch. This project relies highly on DeepMatch. Additionally, I used thePLBaseModeldesign fromtorchTS.Thanks to this awesome projects.中文 READMEDeepMatch-Torch是一个PyTorch版本的DeepMatch。DeepMatch-Torch是一个用于广告推荐的召回模型库，可以非常简单地训练模型和导致user和item的vertor 表示，你可以用这个 user/item 的表示进行 ANN 近似检索。在 DeepMatch-Torch 中，你可以保持和 DeepMatch 一样的习惯，通过model.fit()和model.predict()进行模型的训练和预测。更多的中文文档细节可以参见：中文文档1. 安装步骤一: 安装 PyTorch, 按照官网指引步骤二：通过 pip 安装：pip install deepmatch-torch通过源码安装gitclonehttps://github.com/bbruceyuan/DeepMatch-Torch.gitcdDeepMatch-Torch
pythonsetup.pyinstall2. 快速上手进入example目录查看相关代码，使用方式和 DeepMatch 几乎保持一致。cdexamples
pythonrun_youtubednn.pyorrun_fm_dssm.py
Package: deep-medical-toolkit
Deep Medical Toolkit (dmt)This repo consists of a personal code toolkit for the purpose of facilitating medical imaging research. The main components of this library consists of a neural network model zoo, image transformations (for preprocessing & augmentation), common metrics/losses, fast multiprocessed data loading, and data structures for image samples.Implementation DetailsSimilarities toTorchioSame design hierarchy where samples (dict subclass) can hold arbitrary
attributes, and library-specific data like Images (e.g. ScalarImage3D),
Labels (e.g Masks)Transforms take samples (i.e. subjects) as they contain all the abstractions
and data format conversions built-in. Also their custom attributes feature
allows for easy storage of transformation history.Improvements OverTorchioOverall objects and shift in design..Introduced data abstractions like samples (i.e. subject in torchio) and
examples (elements in a batch). This distinction is important.More extensible to allow custom behavior for data structures.Added general data structures for 2D & 3D images, and labels.3D: ScalarMask3D, ScalarImage3D2D: ScalarMask2D, ScalarImage2D, VectorImage2DClassification: CategoricalLabelExtended transformations to both 2D & 3D. Also added some 3D ones as well.Added 3D transforms:All 2D transforms:Improved existing data structures.For labels, added categorical (both multi-class & multi-label).For Images, gives you the option to permanently load data.Extensibility is improved for almost all data structures. For example, in
an Image, you can overload how a file is read, what preprocessing you want,
how to get an array/tensor from the preprocessed sitk image.Extended multiprocessing data loading for better flexibility, extensibility, and performance.Torchio has a Queue class that loades patches, DMT's equivalent of this is the PatchLoader class.This class continously loads patches rather than waiting for the queue to empty.DMT also has a DaemonLoader class that wraps the PyTorch DataLoader to continuously load samples.Added a model zoo that has both 2D & 3D neural networks.Added losses & metrics common to 2D & 3D computer vision tasks.Additional Verbose ImprovementsUniversally, numpy.ndarrays are passed around (instad of tensors like torchio)Sample images (one sample = one patient) are lazy-loaded as an sitk object
if a path is given.TODO:GeneralAdd weak references for Samples, Images, and Labels for easy access.Remove printing private attributes inreprfor images & others.For samples, and other relevant dict objects, check if reserved_attributes are not being overwritten.DataAdd mask + image overlap plotting for samplesAddTransformsAdd image shape tracking to attributes for transforms (sample transform history).Add both 2D & 3D resized crop where you can set the scale.
Package: deepmerge
A tools to handle merging of
nested data structures in python.Installationdeepmerge is available onpypi:pipinstalldeepmergeExampleGeneric Strategyfromdeepmergeimportalways_mergerbase={"foo":["bar"]}next={"foo":["baz"]}expected_result={'foo':['bar','baz']}result=always_merger.merge(base,next)assertexpected_result==resultCustom StrategyfromdeepmergeimportMergermy_merger=Merger(# pass in a list of tuple, with the# strategies you are looking to apply# to each type.[(list,["append"]),(dict,["merge"]),(set,["union"])],# next, choose the fallback strategies,# applied to all other types:["override"],# finally, choose the strategies in# the case where the types conflict:["override"])base={"foo":["bar"]}next={"bar":"baz"}my_merger.merge(base,next)assertbase=={"foo":["bar"],"bar":"baz"}You can also pass in your own merge functions, instead of a string.For more information, see thedocs
Package: deep_merge
# Deep MergeThis library contains a simple utility for deep-merging dictionaries and the
data structures they contain.Installation:pip install deep_mergeDocumentation:https://pythonhosted.org/deep_mergeRepositiory:https://github.com/halfak/deep_mergeLicense:MIT## Example>>> import deep_merge
>>>
>>> print(deep_merge.merge({'a': {'b': 10}}, {'a': {'c': 5}}))
{'a': {'b': 10, 'c': 5}}Note that the first parameter is modified and will be assigned the return value.>>> d1 = {}
>>> d2 = {'a': 1}
>>> deep_merge.merge(d1, d2)
>>> print(d1)
{'a': 1}## Author
* Aaron Halfaker –https://github.com/halfak
Package: deepmerit
DeepMeritDeepMerit is an open source package for creating novel methods that help all the stake holders of a clinical trial life cycle. It leverages research technology like Data Shapley, Multi-Accuracy and cPCA from Stanford Artificial Intelligence Labs (SAIL)Getting StartedProgramming GuidevLifeLicenseGetting StartedPlease Install DeepMerit package using!pip install deepmeritProgramming GuideComing SoonLicenseComing Soon
Package: deepml
deep-mlThis library is a wrapper around pytorch and useful for solving image classification and semantic
segmentation problems.FeaturesEasy to use wrapper around pytorch so that you can focus on training and
validating your model.Integrates with Tensorboard to use it to monitor metrics while model trains.Quickly visualize your model's predictions.Following are different types of machine learning tasks available to choose fromdeepml.tasks:ImageClassificationMultiLabelImageClassificationImageRegressionSegmentationInstallationBefore installingdeepml, it is recommended to referpytorchofficial page fortorchinstallation.PypipipinstalldeepmlUsage1. Create torch data loaders.importtorchtrain_loader=# your train loader instance of torch.utils.data.DataLoaderval_loader=# your val loader instance of torch.utils.data.DataLoader2. Create your deep neural net architecture.importtorchvision# instance of torch.nn.Modulemodel=torchvision.models.vgg.vgg19(pretrained=False)3. Choose your machine learning task.fromdeepml.tasksimportImageClassificationclassification=ImageClassification(model,model_dir="experiment1",load_saved_model=False,classes=['class1','class2','class3'])4. Define optimizer, loss function and lr scheduler.optimizer=torch.optim.Adam(model.parameters(),lr=0.0001)# loss functioncriterion=torch.nn.CrossEntropyLoss()# Choose lr_scheduler if anylr_scheduler=torch.optim.lr_scheduler.StepLR(optimizer,step_size=4,gamma=0.1)5. Quickly start training your model using deepml.train.Learner class.fromdeepml.trainimportLearner# instantiate learner classlearner=Learner(classification,optimizer,criterion)# Fit Learnerlearner.fit(train_loader,val_loader,epochs=10,lr_scheduler=lr_scheduler)6. Use tensorboard to visualize model loss and metrics.On Google Colab or Jupyter Notebook:%load_exttensorboard
%tensorboard--logdir'experiment1'On OS:tensorboard--logdir'experiment1'7. Quickly see some samples predictions from data loader.learner.show_predictions(val_loader,samples=30,cols=6,figsize=(20,20))8. Run prediction on data loader.predictions,targets=learner.predict(val_loader)ExamplesCheck out the below google colaboratory notebook examples:Image RegressionImage ClassificationBinary Semantic Segmentation (Road Segmentation on Satellite Imagery)Multiclass Semantic Segmentation (Scene Understanding on Street Imagery)Contributingdeepml is an open source project and anyone is welcome to contribute. An easy way to get started is by suggesting a new enhancement on the Issues. If you have found a bug, then either report this through Issues, or even better, make a fork of the repository, fix the bug and then create a Pull Request to get the fix into the master branch.Licensedeepml is available under the MIT License. For details see theLICENSEfile.
Package: deepmodeling-sphinx
deepmodeling_sphinxThis package should be used in all sphinx projects under the@deepmodelingorganization.FeaturesAdd the DeepModeling bannerAdd the ICP number to the footerMinify HTML, Javascript, and CSS filesSupports dark mode for both the banner and the RTD themeHow to use itSetupAdddeepmodeling_sphinxto the requirements, as well as theextensionsofconf.py:extensions=['deepmodeling_sphinx',]Projects outside DeepModeling can also use this extension but disable DeepModeling specific styles.# default: Trueenable_deepmodeling=FalseRender list of authorsThe following directive can be used to render list of authors fromgit shortlog:..git-shortlog-authors::
Package: deepmorpheus
Deep MorpheusMorphological tagger for Ancient Greek and Latin using deep learning.InstallationMake sure you are running Python 3.6 or higher. You can install the package from PyPI:pipinstalldeepmorpheusUsageImport the library:importdeepmorpheusTo tag a.txtfile::deepmorpheus.tag_from_file("input.txt","ancient-greek")To tag a string directly:deepmorpheus.tag_from_lines("τὰ γὰρ πρὸ αὐτῶν καὶ τὰ ἔτι παλαίτερα σαφῶς μὲν εὑρεῖν διὰ χρόνου πλῆθος ἀδύνατα ἦν","ancient-greek")LicenseMIT
Package: deepmultilingualpunctuation
Deep Multilingual Punctuation PredictionThis python library predicts the punctuation of English, Italian, French and German texts. We developed it to restore the punctuation of transcribed spoken language.This usesour "FullStop" modelthat we trained on theEuroparl Dataset.Please note that this dataset consists of political speeches. Therefore the model might perform differently on texts from other domains.The code restores the following punctuation markers:"." "," "?" "-" ":"Video SampleInstallTo get started install the package frompypi:pipinstalldeepmultilingualpunctuationUsageThePunctuationModelclass an process texts of any length. Note that processing of very long texts can be time consuming.Restore PunctuationfromdeepmultilingualpunctuationimportPunctuationModelmodel=PunctuationModel()text="My name is Clara and I live in Berkeley California Ist das eine Frage Frau Müller"result=model.restore_punctuation(text)print(result)outputMy name is Clara and I live in Berkeley, California. Ist das eine Frage, Frau Müller?Predict LabelsfromdeepmultilingualpunctuationimportPunctuationModelmodel=PunctuationModel()text="My name is Clara and I live in Berkeley California Ist das eine Frage Frau Müller"clean_text=model.preprocess(text)labled_words=model.predict(clean_text)print(labled_words)output[['My', '0', 0.9999887], ['name', '0', 0.99998665], ['is', '0', 0.9998579], ['Clara', '0', 0.6752215], ['and', '0', 0.99990904], ['I', '0', 0.9999877], ['live', '0', 0.9999839], ['in', '0', 0.9999515], ['Berkeley', ',', 0.99800044], ['California', '.', 0.99534047], ['Ist', '0', 0.99998784], ['das', '0', 0.99999154], ['eine', '0', 0.9999918], ['Frage', ',', 0.99622655], ['Frau', '0', 0.9999889], ['Müller', '?', 0.99863917]]ResultsThe performance differs for the single punctuation markers as hyphens and colons, in many cases, are optional and can be substituted by either a comma or a full stop. The model achieves the following F1 scores for the different languages:LabelENDEFRIT00.9910.9970.9920.989.0.9480.9610.9450.942?0.8900.8930.8710.832,0.8190.9450.8310.798:0.5750.6520.6200.588-0.4250.4350.4310.421macro average0.7750.8140.7820.762ReferencesPlease cite us if you found this useful:@article{guhr-EtAl:2021:fullstop,
  title={FullStop: Multilingual Deep Models for Punctuation Prediction},
  author    = {Guhr, Oliver  and  Schumann, Anne-Kathrin  and  Bahrmann, Frank  and  Böhme, Hans Joachim},
  booktitle      = {Proceedings of the Swiss Text Analytics Conference 2021},
  month          = {June},
  year           = {2021},
  address        = {Winterthur, Switzerland},
  publisher      = {CEUR Workshop Proceedings},  
  url       = {http://ceur-ws.org/Vol-2957/sepp_paper4.pdf}
}
Package: deepmux
deepmux-pythondeepmuxis a PaaS solution to effortlessly deploy trained machine learning models on the cloud and generate predictions without setting up any hardware.At the moment onlypytorchmodels are supported.Installationpip install deepmuxGetting tokenYou can get yourdeepmuxtoken athttps://app.deepmux.com/api_key.QuickstartCreating modelInitialize a model and upload it to deepmux serversfromdeepmuximportcreate_modelmodel=create_model(<YOURPYTORCHMODEL>,<MODEL_NAME>,<SHAPEOFINPUTDATA>,<SHAPEOFOUTPUTDATA>,<TOKEN>)Getting model by nameOn your production server you can simply get model by it's name.fromdeepmuximportget_modelmodel=get_model(<MODELNAME>,<TOKEN>)Executing model on your importAfter initializing your model withcreate_modelorget_modelyou can run the model. All computations will be performed ondeepmuxinfrastucture.output=model.run(<YOURINPUT>)Complete example on a model from PyTorch Hubimportnumpyasnpimporttorchfromdeepmuximportcreate_modeltoken="<YOUR_TOKEN>"pytorch_model=torch.hub.load('pytorch/vision:v0.5.0','squeezenet1_0',pretrained=True)deepmux_model=create_model(pytorch_model,model_name='my_model',input_shape=[1,3,227,227],output_shape=[1,1000],token=token)dummy_input=np.zeros([1,3,227,227],dtype=np.float32)output=deepmux_model.run(dummy_input)
Package: deepmux-cli
deepmux-cliDeepMux CLI allows to manage functions on DeepMux cloud platform.Pre-requisitesPythonInstall Python 3.6 or newer. Follow theinstruction here.Skip this step if you already have proper version of python installed.PipWe are going to usepipto install deepmux command line interface below. If you don't havepipavailable from the command line, followthis guide.Installing and configuring the packageInstallUsepipto installdeepmux-cli:pipinstalldeepmux-cliLoginGo to thehttps://app.deepmux.com/api_keyand copy your token.Then run:deepmuxloginAnd paste your token when prompted.Run--helpto see the full list of options:deepmux--helpDocumentationComplete documentation for deepmux-cli is availible atdeep-mux.github.io
Package: deepnccl
Deep-NCCLOptimized primitives for inter-GPU communication on Aliyun machines.IntroductionDeep-NCCL is an AI-Accelerator communication framework for NVIDIA-NCCL.
It implements optimized all-reduce, all-gather, reduce, broadcast, reduce-scatter, all-to-all, as well as any send/receive based communication pattern.
It has been optimized to achieve high bandwidth on aliyun machines using PCIe, NVLink, NVswitch, as well as networking using InfiniBand Verbs, eRDMA or TCP/IP sockets.InstallTo install Deep NCCL on the system, create a package then install it as root as follow two methods:method1: rpm/deb (Recommended)# Centos:wgethttp://mirrors.aliyun.com/aiacc/aiacc-nccl/aiacc_nccl-1.0.rpm
rpm-iaiacc-nccl-1.0.rpm# Ubuntu:wgethttp://mirrors.aliyun.com/aiacc/aiacc-nccl/aiacc_nccl-1.0.deb
dpkg-iaiacc-nccl-1.0.debmethod2: python-offlinewgethttp://mirrors.aliyun.com/aiacc/aiacc-nccl/aiacc_nccl-2.0.0.tar.gz
pipinstallaiacc_nccl-2.0.0.tar.gz# notes: must download and then pip install, cannot merge in oneline `pip install aiacc_xxx_url`# Both method1 and method2 can run concurrently.method3: python-pypipipinstallaiacc_nccl==2.0UsageAfter install aiacc-nccl package, you need do nothing to change code!EnvironmentAIACC_FASTTUNING: Enable Fasttuning for LLMs, default=1 is to enable.NCCL_AIACC_ALLREDUCE_DISABLE: Disable allreduce algo, default=0 is to enable.NCCL_AIACC_ALLGATHER_DISABLE: Disable allgather algo, default=0 is to enable.NCCL_AIACC_REDUCE_SCATTER_DISABLE: Disable reduce_scatter algo, default=0 is to enable.AIACC_UPDATE_ALGO_DISABLE: Disable update aiacc nccl algo from aiacc-sql-server, default=0 is to enable.PerformanceDeep-NCCL can speedup the nccl performance on aliyun EGS(GPU machine), for example instance type 'ecs.ebmgn7ex.32xlarge' is A100 x 8 GPU and using network eRdma.GPU(EGS)CollectiveNodesNetworkSpeedup(nccl-tests)A100 x 8all_gather2-10VPC/eRdma30%+A100 x 8reduce_scatter2-10VPC/eRdma30%+A100 x 8all_reduce2-10VPC/eRdma20%V100 x 8all_reduce2-20VPC60%+A10  x 8all_reduce1-20%CopyrightAll source code and accompanying documentation is copyright (c) 2015-2020, NVIDIA CORPORATION. All rights reserved.
All modifications are copyright (c) 2020-2024, ALIYUN CORPORATION. All rights reserved.
Package: deepncclplugin
AIACC-NCCL-PLUGINOptimized socket/rdma for inter-GPU communication on Aliyun machines.IntroductionAIACC-NCCL-PLUGIN is an AI-Accelerator communication framework plugin for NVIDIA-NCCL.
It has been optimized to achieve high bandwidth on aliyun machines using InfiniBand Verbs, eRDMA or TCP/IP sockets.InstallTo install AIACC NCCL PLUGIN on the system, create a package then install it as root as follow two methods:method1: rpm/deb (Recommended)# Centos:wgethttp://mirrors.aliyun.com/aiacc/aiacc-nccl-plugin/aiacc-nccl-plugin-1.1.0.rpm
rpm-iaiacc-nccl-plugin-1.1.0.rpm# Ubuntu:wgethttp://mirrors.aliyun.com/aiacc/aiacc-nccl-plugin/aiacc-nccl-plugin-1.1.0.deb
dpkg-iaiacc-nccl-plugin-1.1.0.debmethod2: python-pypipipinstallaiacc_nccl_pluginUsageAfter install aiacc-nccl-plugin package, you need do nothing to change code!CopyrightAll source code and accompanying documentation is copyright (c) 2015-2020, NVIDIA CORPORATION. All rights reserved.
All modifications are copyright (c) 2020-2024, ALIYUN CORPORATION. All rights reserved.