Saravanan666/pypi-org huggingface dataset:
Package: hf-hub-ctranslate2
hf_hub_ctranslate2Connecting Transformers on HuggingfaceHub with Ctranslate2 - a small utility for keeping tokenizer and model around Huggingface Hub.Read the docsUsage:Decoder-only Transformer:# download ctranslate.Generator repos from Huggingface Hub (GPT-J, ..)fromhf_hub_ctranslate2importTranslatorCT2fromHfHub,GeneratorCT2fromHfHubmodel_name_1="michaelfeil/ct2fast-pythia-160m"model=GeneratorCT2fromHfHub(# load in int8 on CPUmodel_name_or_path=model_name_1,device="cpu",compute_type="int8")outputs=model.generate(text=["How do you call a fast Flan-ingo?","User: How are you doing?"]# add arguments specifically to ctranslate2.Generator here)Encoder-Decoder:fromhf_hub_ctranslate2importTranslatorCT2fromHfHub# download ctranslate.Translator repos from Huggingface Hub (T5, ..)model_name_2="michaelfeil/ct2fast-flan-alpaca-base"model=TranslatorCT2fromHfHub(# load in int8 on CUDAmodel_name_or_path=model_name_2,device="cuda",compute_type="int8_float16")outputs=model.generate(text=["How do you call a fast Flan-ingo?","Translate to german: How are you doing?"],# use arguments specifically to ctranslate2.Translator below:min_decoding_length=8,max_decoding_length=16,max_input_length=512,beam_size=3)print(outputs)Encoder-Decoder for multilingual translations (m2m-100):fromhf_hub_ctranslate2importMultiLingualTranslatorCT2fromHfHubmodel=MultiLingualTranslatorCT2fromHfHub(model_name_or_path="michaelfeil/ct2fast-m2m100_418M",device="cpu",compute_type="int8",tokenizer=AutoTokenizer.from_pretrained(f"facebook/m2m100_418M"))outputs=model.generate(["How do you call a fast Flamingo?","Wie geht es dir?"],src_lang=["en","de"],tgt_lang=["de","fr"])Encoder-only Sentence Transformersfromhf_hub_ctranslate2importCT2SentenceTransformermodel_name_pytorch="intfloat/e5-small"model=CT2SentenceTransformer(model_name_pytorch,compute_type="int8",device="cuda",)embeddings=model.encode(["I like soccer","I like tennis","The eiffel tower is in Paris"],batch_size=32,convert_to_numpy=True,normalize_embeddings=True,)print(embeddings.shape,embeddings)scores=(embeddings@embeddings.T)*100Encoder-onlyfromhf_hub_ctranslate2importEncoderCT2fromHfHubmodel_name="michaelfeil/ct2fast-e5-small"model=EncoderCT2fromHfHub(# load in int8 on CUDAmodel_name_or_path=model_name,device="cuda",compute_type="int8_float16",)outputs=model.generate(text=["I like soccer","I like tennis","The eiffel tower is in Paris"],max_length=64,)PYPI Installpipinstallhf-hub-ctranslate2
Package: hfilesize
Parses & Formats integer file sizes to human readable file sizes.Example UsageParsing>>> from hfilesize import Format, FileSize
>>> FileSize('1k')
... 1000
>>> FileSize('1K')
... 1024
>>> FileSize('1kib')
... 1024
>>> FileSize('1K', default_binary=False, case_sensitive=False)
... 1000
>>> FileSize('1 kibibyte')
... 1024Formatting>>> '{:d}'.format(FileSize(1024))
... '1024'
>>> '{:.02fH}'.format(FileSize(1024))
... '1 KB'
>>> '{:.02fHcv}'.format(FileSize(1024))
... '1 kilobyte'
>>> '{:.02fhs}'.format(FileSize(1000))
... '1 KB'
>>> '{:.02fhs^0}'.format(FileSize(1000))
... '1000 B'
>>> '{: >10.02fH}'.format(FileSize(1024))
... '      1 KB'DocumentationParsing Optionscase_sensitive: Use 1024 for upper case and 1000 for lower case
if casing exists, as is common in unix utilities, e.g. dddefault_binary: Default base if it is not clear what the unit is
(i.e. if it is not ‘mib’ or ‘mebibytes’)Formatting Optionsformat type:[hH][size_format][^exponent]h: Base 1000H: Base 1024size_format:c | cs | cv | e | ev | s | svc: Commonly used case-sensitive suffixescs: Commonly used abbreviated case-sensitive suffixescv: Commonly used verbose case-sensitive suffixese: IEC suffixesev: IEC verbose suffixess: SI suffixessv: SI verbose suffixesexponent:integerReferencesInspired by:`hurry.filesize<https://pypi.python.org/pypi/hurry.filesize>`__Human readable file/memory sizes v2 (Python
recipe)LicenseLicensed under GPLv3.
Package: hfinger
Hfinger - fingerprinting malware HTTP requestsTool for fingerprinting malware HTTP requests. Based on Tshark and written in Python3. Working prototype stage :-)It's main objective is to provide a representation of malware requests in a shorter form than printing whole request,
but still human interpretable. This representation should be unique between malware families,
what means that any fingerprint should be seen only for one particular family.Project's website:https://github.com/CERT-Polska/hfinger.
Package: hfloader
HFLoader - Hugging Face Model LoaderThis package provides the user with one method that returns both atokenizerand amodelwhen loadingHuggingFace Models, without the need for knowing whichAutoModelto use.This package utilizes thetransformerslibrary to load a tokenizer and model, without having to know theAutoModel. This means that with one command, you can easily load the tokenizer and model of a given model onHuggingFace Model Hub. This can then be easily fed into thepipelinefunction oftransformers.InstallationThe package can be installed with the following command:pip install hfloaderHow to UseHere is a bit of code you can reference to see how to use the package.import hfloader as hfl

huggingface_model = "cardiffnlp/twitter-roberta-base-sentiment"
tokenizer, model = hfl.load_model(huggingface_model)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, torch_dtype=torch.bfloat16, device_map="auto")The return of the commandload_model()is going to be a tokenizer and a model, which can then be used in the pipeline method provided by transformers. It does so without the need to know which AutoModel to use, which can prove to be a hassle when trying out different models.Requirementspip install transformersNotesI don't have plans to upkeep this project unless it necessitates it. I was able to achieve the goal I had set out when developing the package.
Package: hf-model-s-cpu
model-smodel-s served from hf spaces for torch cpuInstall itpipinstallgit+https://github.com/ffreemt/model-s-cpu# poetry add git+https://github.com/ffreemt/model-s-cpu# git clone https://github.com/ffreemt/model-s-cpu && cd model-s-cpuInstallsentence-transformersSimplypipinstallsentence-transformersOrpipinstall--no-cache-dirtorch==1.8.0+cpu-fhttps://download.pytorch.org/whl/torch_stable.html# adjust according to os/pythonpipinstalltransformerstqdmnumpyscikit-learnnltksentencepiecepillow
pipinstall--no-depssentence-transformersUse itfromhf_model_s_cpuimportmodel_s
Package: hfoGUI
hfoGUIhfoGUI was initially a Python package designed to visualize High Frequency Oscillations (HFO's), but has grown to be an all encompassing tool to visualize LFP data recorded in the Tint format (from Axona's dacqUSB).Python Dependencies:PyQt5PillownumpyPyQtGraphscipymatplotlibpandaspyfftwRequirementsSince it utilizes PyQt5 as the GUI framework it should be available to a wide variety of Python versions and Operating Systems. However it was only tested on Python 3.7 with Windows 10. I do recommend using the latest version of Python as it makes it easy to download some of these C++ based dependencies.Project DocumentationInstallationUser GuideAuthorsGeoff Barrett-Geoffâ€™s GitHubLicenseThis project is licensed under the GNU  General  Public  License - see theLICENSE.mdfile for details
Package: hfradarpy
HFRadarPyToolbox to read in High Frequency Radar (HFR) files written in CODAR Tabular Format (CTF).Free software: MIT licenseDocumentation:https://hfradarpy.readthedocs.io.FeaturesTODOInstallationStable releaseTo install HFRadarPy, run this command in your terminal:$pipinstallhfradarpyThis is the preferred method to install HFRadarPy, as it will always install the most recent stable release.If you don’t havepipinstalled, thisPython installation guidecan guide
you through the process.We also recommend using miniconda to manage your Python environments. Download and follow theMiniconda installation guidefor the appropriate
Miniconda installer for your operating system.Make sure to add the channel,conda-forge, to your .condarc. You can
find out more about conda-forge from their website:You can do this with the following command:conda config --add channels conda-forgeFrom sourcesThe sources for HFRadarPy can be downloaded from theGithub repo.You can either clone the public repository:$gitclonegit://github.com/rucool/hfradarpyOr download thetarball:$curl-OJLhttps://github.com/rucool/hfradarpy/tarball/masterOnce you have a copy of the source, you can should create a new conda/virtual environment:Create environmentChange your current working directory to the location that you
downloaded codar_processing to.$cd~/Downloads/hfradarpy/Create conda environment from the included environment_dev.yml file:$condaenvcreate-fenvironment_dev.ymlOnce the environment is done building, you can activate the environment
by typing:$condaactivatehfradarpy# OSX/UnixOnce the environment is your active environment. You can install the toolbox to that environment.$pythonsetup.pyinstallYou can also change directory into the root hfradarpy directory and install with the following:$pipinstall.Or if you are developing new code in the toolbox, you should install this library as ‘editable’:$pipinstall--no-deps--force-reinstall--ignore-installed-e.Running testsAfter setting up your environment, you can run all of the tests, provided you install ‘pytest’:$pytestCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.4.3 (2022-02-28)Fourth release on PyPI.0.1.4.1 (2022-02-28)Third release on PyPI.0.1.3 (2022-02-28)Second release on PyPI.0.1.2 (2021-12-03)First release on PyPI.
Package: hfscanstat
hfscanstatA python implementation of the hypothesis-free multiple scan statistic with variable window for detecting clusters in point processes (L. Cucala. A hypothesis-free multiple scan statistic with variable window. Biom. J., 50 (2008), pp. 299-310).Installpip install hfscanstatHow to useUsing example data from Knox, G. (1959). Secular pattern of congenital oesophageal atresia. British Journal of Preventive Social Medecine 13, 222–226.knox_1959=[170,316,445,468,938,1034,1128,1233,1248,1249,1252,1259,1267,1305,1385,1388,1390,1446,1454,1458,1461,1491,1583,1699,1702,1787,1924,1974,2049,2051,2067,2075,2108,2151,2174,2191]clusters=get_clusters(knox_1959)print(clusters)X_i       X_j  i   j         I_H_F            ps
0  0.525977  0.653637  7  21  7.319199e+14 -6.661338e-16
Package: hfst-optimized-lookup
hfst-optimized-lookupA pip-installable library version ofhfst-optimized-lookup, originally
built foritwêwina.Installpip install hfst-optimized-lookupThis requires that the machine runningpiphave a working C++ compiler. If
enough people ask for us to uploadbinary ‘wheels’so that you don’t need a
compiler at install time, we could start doing so.UsageFull API DocumentationImport the library:>>> import hfst_optimized_lookupThen load an FST!>>> fst = hfst_optimized_lookup.TransducerFile('../crk-relaxed-analyzer-for-dictionary.hfstol')Hint: Downloadcrk-relaxed-analyzer-for-dictionary.hfstolby cloninghttps://github.com/UAlbertaALTLab/cree-intelligent-dictionary/tree/main/src/crkeng/resources/fstto follow along! The file itself is stored in Git LFS so is tricky to
link to directly.Do an ordinary lookup, to get a list ofconcatenated analysesfor a wordform:>>> fst.lookup('atim')
['atim+N+A+Sg', 'atimêw+V+TA+Imp+Imm+2Sg+3SgO']Or get eachparsed analysisfrom the wordform>>> analysis = fst.lookup_lemma_with_affixes('atim')[0]
>>> analysis.lemma
'atim'
>>> analysis.suffixes
('+N', '+A', '+Sg')You can also lookup the analyses with symbols separated:>>> fst.lookup_symbols('atim')
[['a', 't', 'i', 'm', '+N', '+A', '+Sg'], ['a', 't', 'i', 'm', 'ê', 'w', '+V', '+TA', '+Imp', '+Imm', '+2Sg', '+3SgO']]ReleasingRun./release.py --helpfor details of the release process.A typical release is:Make sure the “Unreleased” section ofCHANGELOG.mdis up-to-dateRun the script:./release.py --release-timezone=America/Edmonton --push --release
Package: hf-streamlit
Welcome to Streamlit :wave:The fastest way to build and share data apps.Streamlit lets you turn data scripts into shareable web apps in minutes, not weeks. It’s all Python, open-source, and free! And once you’ve created an app you can use ourCommunity Cloud platformto deploy, manage, and share your app!Installationpipinstallstreamlit
streamlithelloStreamlit can also be installed in a virtual environment onWindows,Mac, andLinux.A little exampleStreamlit makes it incredibly easy to build interactive apps:importstreamlitasstx=st.slider('Select a value')st.write(x,'squared is',x*x)A bigger exampleStreamlit's simple and focused API lets you build incredibly rich and powerful tools.This demo projectlets you browse the entireUdacity self-driving-car datasetand run inference in real-time using theYOLO object detection net.The complete demo is implemented in less than 300 lines of Python. In fact, the app containsonly 23 Streamlit callswhich illustrates all the major building blocks of Streamlit. You can try it right now atshare.streamlit.io/streamlit/demo-self-driving.The Streamlit GitHub badgeStreamlit's GitHub badge helps others find and play with your Streamlit app.Once you deploy your app, you can embed this badge right into your GitHub readme.md as follows:[![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://share.streamlit.io/yourGitHubName/yourRepo/yourApp/)More InformationOurlaunch postexplaining why we created StreamlitOurCommunity Cloud platform announcementOur amazingcommunitywhere Streamlit users share apps, ask questions, and help each other outStreamlitdocumentationandblogfor the latest Streamlit infoMoredemo projectsto inspire youAnd if you would like to contribute, seeinstructions hereCommunity CloudWithCommunity Cloudyou can deploy, manage, and share your apps with the world, directly from Streamlit — all for free. Sign-uphere.LicenseStreamlit is completely free and open-source and licensed under theApache 2.0license.
Package: hft
# Portfolioeffect hft Package for PythonPython API to PortfolioEffect cloud service for backtesting high frequency trading (HFT) strategies,
intraday portfolio analysis and optimization. Includes auto-calibrating model pipeline for market
microstructure noise, risk factors, price jumps/outliers, tail risk (high-order moments) and price
fractality (long memory). Constructed portfolios could use client-side market data or access HF
intraday price history for all major US Equities.## Package Installationpython setup.py install## LicenseThis package is released under the GPLv3 license. See the file LICENSE.Usage of this package with PortfolioEffect services shall be subject to the [Terms of Service][PortfolioEffect Terms].## CopyrightCopyright &copy; 2015 PortfolioEffect[PortfolioEffect Terms]:https://www.portfolioeffect.com/docs/terms
Package: hft-api
A small library for apis of huobi and ftx.Installationpip install hft-apiGet startedThere are ony 2 apis available right now Huobi, Ftxhow to work with HuobifromhuobiimportHuobishuobi=Huobi('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx','xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')# Get all accounts listaccounts=huobi.get_accounts()ifaccounts.get('status')=='ok':spot_account=list(filter(lambdai:i['type']=='spot',accounts.get('data')))[0]huobi.change_account_id(spot_account.get('id'))# Get open ordersopen_orders=huobi.get_open_orders()# Cancel order by idcancel_response=huobi.cancel_order('596827573398451')# Get symbols listsymbols=huobi.get_symbols()# Get klines listklines=huobi.get_klines('btcusdt')# Get balancesbalances=huobi.get_balances()# Place limit orderlimit_order_response=huobi.place_order(amount=101,symbol='trxusdt',price=0.05,type='buy-limit')# Place marker ordermarket_order_response=huobi.place_order(amount=1,symbol='trxusdt',type='buy-market')How to work with FTXfromftximportFTXftx=FTX('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx','xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')# Get positions listftx.get_positions()# Get account informationftx.get_account_information()# Get account balances listftx.get_balances()# Get all open ordersftx.get_open_orders()# Get all open orders for "ETH-PERP"ftx.get_open_orders("ETH-PERP")# Get orders historyftx.get_orders_history()# Get orders history for "ETH-PERP"ftx.get_orders_history("ETH-PERP")# Get open trigger ordersftx.get_trigger_orders()# Get open trigger orders for "ETH-PERP"ftx.get_trigger_orders("ETH-PERP")# Get all twap ordersftx.get_twap_orders()# Get all twap orders for "ETH-PERP"ftx.get_twap_orders("ETH-PERP")# Get all active twap orders for "ETH-PERP"ftx.get_twap_orders("ETH-PERP","running")# Place limit orderftx.place_order(market="ETH/USDT",side='buy',type='limit',size=1.2,price=1498,clientId="70bed756-228f-4325-8af4-3a75559cdf30")# Place market orderftx.place_order(market="ETH/USDT",side='buy',type='market',size=1.2,clientId="1fcf03f1-b110-46ef-9a07-be4c7c0e3a77")# Place stop loss order:ftx.place_trigger_order(market="ETH/USDT",side='sell',size=1.2,type='stop',triggerPrice=100,)# Place trailing stop order:ftx.place_trigger_order(market="ETH/USDT",side='sell',size=1.2,type='trailingStop',trailValue=-0.05)# Place take profit order:ftx.place_trigger_order(market="ETH/USDT",side='sell',size=1.2,type='takeProfit',triggerPrice=10000,)# Place twap orderftx.place_twap_order(market="ETH/USDT",durationSeconds=600,size=1,side="buy")
Package: hftbacktest
High-Frequency Trading Backtesting Tool in PythonThis Python framework is designed for developing high-frequency trading and market-making strategies. It focuses on accounting for both feed and order latencies, as well as the order queue position for order fill simulation. The framework aims to provide more accurate market replay-based backtesting, based on full order book and trade tick feed data.Key FeaturesWorking inNumbaJIT function.Complete tick-by-tick simulation with a variable time interval.Full order book reconstruction based on L2 feeds(Market-By-Price).Backtest accounting for both feed and order latency, using provided models or your own custom model.Order fill simulation that takes into account the order queue position, using provided models or your own custom model.DocumentationSeefull document here.Getting startedInstallationhftbacktest supports Python 3.8+. You can install hftbacktest usingpip:pip install hftbacktestOr you can clone the latest development version from the Git repository with:git clone https://github.com/nkaz001/hftbacktestData Source & FormatPlease seeDataorData Preparation.A Quick ExampleGet a glimpse of what backtesting with hftbacktest looks like with these code snippets:@njitdefsimple_two_sided_quote(hbt,stat):max_position=5half_spread=hbt.tick_size*20skew=1order_qty=0.1last_order_id=-1order_id=0# Checks every 0.1swhilehbt.elapse(100_000):# Clears cancelled, filled or expired orders.hbt.clear_inactive_orders()# Obtains the current mid-price and computes the reservation price.mid_price=(hbt.best_bid+hbt.best_ask)/2.0reservation_price=mid_price-skew*hbt.position*hbt.tick_sizebuy_order_price=reservation_price-half_spreadsell_order_price=reservation_price+half_spreadlast_order_id=-1# Cancel all outstanding ordersfororderinhbt.orders.values():iforder.cancellable:hbt.cancel(order.order_id)last_order_id=order.order_id# All order requests are considered to be requested at the same time.# Waits until one of the order cancellation responses is received.iflast_order_id>=0:hbt.wait_order_response(last_order_id)# Clears cancelled, filled or expired orders.hbt.clear_inactive_orders()last_order_id=-1ifhbt.position<max_position:# Submits a new post-only limit bid order.order_id+=1hbt.submit_buy_order(order_id,buy_order_price,order_qty,GTX)last_order_id=order_idifhbt.position>-max_position:# Submits a new post-only limit ask order.order_id+=1hbt.submit_sell_order(order_id,sell_order_price,order_qty,GTX)last_order_id=order_id# All order requests are considered to be requested at the same time.# Waits until one of the order responses is received.iflast_order_id>=0:hbt.wait_order_response(last_order_id)# Records the current state for stat calculation.stat.record(hbt)TutorialsData PreparationGetting StartedWorking with Market Depth and TradesIntegrating Custom DataHigh-Frequency Grid TradingImpact of Order LatencyGuéant–Lehalle–Fernandez-Tapia Market Making Model and Grid TradingMaking Multiple MarketsExamplesYou can find more examples inexamplesdirectory.
Package: hf-torrent
HF-TorrentThis repository contains an experimental torrent-style download for model weights in Hugging Face.The core implementation ismain_hub.py, which downloads files from Hugging Face and converts them into seedable torrents.How to usepip install hf-torrentBefore launching downloads, make sure you havearia2cinstalled andrun the following command in a separate terminalaria2c--enable-rpc--rpc-listen-all\--max-upload-limit=4M\--optimize-concurrent-downloads=true\--bt-detach-seed-only=true\--seed-ratio=10.0Then you can start using HF-Torrent with the following command:hf-torrent<yourtargetrepo>Or if you have your preferred torrent clients, you can pull the torrent file onlyhf-torrent--get-torrent<yourtargetrepo>How to contributeInstallationTo install, run the following command:pip install -r requirements.txtSeedingIt is easy to contribute to seeding! First, change--seed-ratio=0(from 10 to 0) when launching thearia2cRPC. Setting0means the daemon will seed infinitely.
Next, change the upload bandwidth limit to a value that suits you, for example:--max-upload-limit=20M(4M by default).After making these changes, launching seeding is as simple as downloading models. You can start seeding by running the following script and help distribute your favorite models!pythondownload.py<yourtargetrepo>CreateTo create a popular torrent, run the following command:python generate_popular_torrents.py -r efederici/retnet-120mThis script will (1) download model files from Hugging Face, (2) iterate through the folder, (3) generate torrents for each file, and then put them underhf-torrent-store/{repo}.We are looking for a wanted model list. Please open an issue if you have any favorite models to share.
Package: hf-torrents
THOP: PyTorch-OpCounterHow to installpip install thop(now continously intergrated onGithub actions)ORpip install --upgrade git+https://github.com/Lyken17/pytorch-OpCounter.gitHow to useBasic usagefromtorchvision.modelsimportresnet50fromthopimportprofilemodel=resnet50()input=torch.randn(1,3,224,224)macs,params=profile(model,inputs=(input,))Define the rule for 3rd party module.classYourModule(nn.Module):# your definitiondefcount_your_model(model,x,y):# your rule hereinput=torch.randn(1,3,224,224)macs,params=profile(model,inputs=(input,),custom_ops={YourModule:count_your_model})Improve the output readabilityCallthop.clever_formatto give a better format of the output.fromthopimportclever_formatmacs,params=clever_format([macs,params],"%.3f")Results of Recent ModelsThe implementation are adapted fromtorchvision. Following results can be obtained usingbenchmark/evaluate_famous_models.py.ModelParams(M)MACs(G)alexnet61.100.77vgg11132.867.74vgg11_bn132.877.77vgg13133.0511.44vgg13_bn133.0511.49vgg16138.3615.61vgg16_bn138.3715.66vgg19143.6719.77vgg19_bn143.6819.83resnet1811.691.82resnet3421.803.68resnet5025.564.14resnet10144.557.87resnet15260.1911.61wide_resnet101_2126.8922.84wide_resnet50_268.8811.46ModelParams(M)MACs(G)resnext50_32x4d25.034.29resnext101_32x8d88.7916.54densenet1217.982.90densenet16128.687.85densenet16914.153.44densenet20120.014.39squeezenet1_01.250.82squeezenet1_11.240.35mnasnet0_52.220.14mnasnet0_753.170.24mnasnet1_04.380.34mnasnet1_36.280.53mobilenet_v23.500.33shufflenet_v2_x0_51.370.05shufflenet_v2_x1_02.280.15shufflenet_v2_x1_53.500.31shufflenet_v2_x2_07.390.60inception_v327.165.75