Saravanan666/pypi-org huggingface dataset:
Package: texy
Texy: A conservative text processing libraryA utility library for quickly cleaning textsInstallationPython version in the dev environment:3.11.5pip install -U texyUsagePipelines with parallelization in Rust:>>>fromtexy.pipelinesimportextreme_clean,strict_clean,relaxed_clean>>>data=["hello ;/ from the other side 😊\t"]print(extreme_clean(data))>>>['hello from the other side']print(strict_clean(data))>>>['hello ;/ from the other side']print(relaxed_clean(data))>>>['hello ;/ from the other side 😊']Parallelize custom functions with Python Multiprocessing:fromtexy.pipelinesimportparallelizedefdummy(x):return[i[0]foriinx]data=["a ","b ","c ","d ","e ","f ","g ","h ?."]*100print(parallelize(dummy,data,2))ActionsPipelineActionsrelaxed_cleanremove_newlines,remove_html,remove_xml,merge_spacesstrict_cleanremove_newlines,remove_urls,remove_emails,remove_html,remove_xml,remove_emoticons,remove_emojis,remove_infrequent_punctuations,merge_spacesextreme_cleanremove_newlines,remove_urls,remove_emails,remove_html,remove_xml,remove_emoticons,remove_emojis,remove_all_punctuations,merge_spaces
Package: teyespider-sdk
TeyeSpider SDK for PythonThe SDK for Python contains two parts:CLI ToolUtility ToolsInstallation PrerequisiteYou should install the requirement below before installing the teyespider-sdk.Click==7.0
requests==2.22.0
prettytable==0.7.2
scrapy==2.2.0
pymongo==3.10.1
pymysql==0.9.3
psycopg2-binary==2.8.5
kafka-python==2.0.1
elasticsearch==7.8.0
pathspec==0.8.0CLI ToolThe CLI Tool is mainly designed for those who are more comfortable using command line tools to interact with teye-spider.The installation of the CLI Tool is simple:pipinstallteyespider-sdkThen, you can use theteye-spidercommand in the command prompt to action with teye-spider.Check the help document below, or you can refer tothe official documentation (Chinese).teye-spider --helpUsage# Hello to teye-spiderteye-spiderhello# Login on first timeteye-spiderlogin-uadmin-ahttp://xxx-master.xxxxx.10.92.xx.xx.nip.io:xxx/api# Upload spider to platformteye-spiderupload-Ndemo-d~/Downloads/tutorial/-ndemo-cres_demo# explore other commandteye-spider--helpUtility ToolsUtility tools mainly provide somehelpermethods to make it easier for you to integrate your spiders into teye-spider, e.g. saving results.Below are integration methods of Scrapy and general Python spiders with teye-spider.⚠️Note: make sure you have already installedteye-spider-sdkusing pip.Scrapy IntegrationInsettings.pyin your Scrapy project, find the variable namedITEM_PIPELINES(adictvariable). Add content below.ITEM_PIPELINES={'teyespider.pipelines.teye-spiderMongoPipeline':888,}Then, start the Scrapy spider. After it's done, you should be able to see scraped results inTask Detail -> ResultGeneral Python Spider IntegrationPlease add below content to your spider files to save results.# import result saving methodfromteyespiderimportsave_item# forexample,it your result is:result={'name':'teyespider'}# call result saving methodsave_item(result)Then, start the spider. After it's done, you should be able to see scraped results inTask Detail -> Result
Package: tez
Tez: a simple pytorch trainerNOTE: Currently, we are not accepting any pull requests! All PRs will be closed. If you want a feature or something doesn't work, please create an issue.tez (तेज़ / تیز) means sharp, fast & active. This is a simple, to-the-point, library to make your pytorch training easy.This library is in early-stage currently! So, there might be breaking changes.Idea around tez is simple:keep things as simple as possiblemake it as customizable as possibleclean codefaster prototypingproduction readyCurrently, tez supports cpu, single gpu and multi-gpu & tpu training. More coming soon!Using tez is super-easy. We don't want you to be far away from pytorch. So, you do everything on your own and just use tez to make a few things simpler.
Package: tezeta
Tezeta: A Package for Maximizing Context Window Utilization in Chatbots (Under Development):warning:Tezeta is currently under active development and is not yet functional. The features listed below are planned for future releases.Tezeta is a Python package designed to optimize memory in chatbots and Language Model (LLM) requests using relevance-based vector embeddings. This tool aims to maximize the utilization of context windows, thereby improving chatbot performance by allowing the storage and retrieval of more relevant conversation history.Supported FeaturesUsing vector embeddings to rank chats based on relevance with OpenAI embeddings and PineconeUsing ChromaDB as vector store with the default all-MiniLM-L6-v2Using ChromaDB as vector storeSupport for using Open Source Embedding Models locally (currently through all-MiniLM-L6-v2 with chromaDB)Planned FeaturesChunk up and rank sections of long text in a single chat or LLM requestSupport for using the Cohere API for EmbeddingsInstallationpipinstalltezetaBasic UsageFirst, to set the necessary environment variables in your system, you can use the following terminal commands.For macOS/Linux:exportPINECONE_API_KEY=your_api_keyexportOPENAI_API_KEY=your_api_keyexportPINECONE_ENVIRONMENT=your_pinecone_environmentFor Windows:set PINECONE_API_KEY=your_api_key
set OPENAI_API_KEY=your_api_key
set PINECONE_ENVIRONMENT=your_pinecone_environmentYou can use the package as follows:importtezetachats=[{"role":"user","content":"Wellness is an important part of wellbeing. How are you tackling that in your life"},{"role":"user","content":"Hello there Jon, I'm a less relevant text that is trying really really hard to excluded from this test."},{"role":"assistant","content":"I'm doing well, how are you?"}]tezeta.set_max_tokens(30)print(chats)llm_chats=tezeta.chats.fit_messages(chats)print(llm_chats)DocumentationDocumentation will be available in the future.LicenseThis project is licensed under the terms of the MIT license.
Package: tezos-etl
Tezos ETLFull documentation available here.QuickstartInstall Tezos ETL:pipinstalltezos-etlExport blocks, balance updates and operations (Schema,Reference):tezosetlexport--start-block1--end-block100\--provider-urihttps://mainnet.api.tez.ie--output-diroutput--output-formatjsonFor the latest version, checkout the repo and callpipinstall-e.
pythontezosetl.pyRunning Testspipinstall-e.[dev]echo"TEZOSETL_PROVIDER_URI variable is optional"exportTEZOSETL_PROVIDER_URI=https://mainnet.api.tez.ie
pytest-vvRunning Tox Testspipinstalltox
toxRunning in DockerInstall Dockerhttps://docs.docker.com/install/Build a docker image:docker build -t tezos-etl:latest .
 docker image lsStart the export using the image:docker run -v $HOME/output:/tezos-etl/output tezos-etl:latest export_partitioned \
 -s 2018-06-30 -e 2018-07-01 -p https://mainnet.api.tez.ie --output-format csv
Package: tezos-python
Tezos pythonThis is a packaged version of the Tezos python API definedhere.Currently, the main use is to use therun_nodescript, which launches a
Tezos node and baker in sandbox mode on port 18731.Installationpip3 install tezos-python
export TEZOS_HOME=~/tezos # path to your tezos binaries
run_node_baker.py --helpNoteThis should work with branchbabylonnet. It has been tested with revision
9a922180.
Package: tezz-pylsd
tezz-pylsd1. Introductiontezz-pylsd is the python bindings forLSD - Line Segment Detector.2. InstallThis package uses distutils, which is the default way of installing python modules. To install in your home directory, securely run the following:git clone https://github.com/tezz-io/tezz-pylsd.git
cd tezz-pylsd
[sudo] python setup.py installOr directly throughpipto install it:[sudo] pip install tezz-pylsd3. UsageWe can use the package by usingfrom pylsd.lsd import lsd, andlines = lsd(src)is the call format for thelsdfunction, wheresrcis a Grayscale Image (H * Wnumpy.array), and the return valuelinesis the Detected Line Segment,linesis anN * 5numpy.array, each row represents a straight line, the 5-dimensional vector is:[point1.x, point1.y, point2.x, point2.y, width]According to these presentations, we can use it just like the following code (here is the link):by using cv2 moduleimportcv2importnumpyasnpimportosfrompylsd.lsdimportlsdfullName='car.jpg'folder,imgName=os.path.split(fullName)src=cv2.imread(fullName,cv2.IMREAD_COLOR)gray=cv2.cvtColor(src,cv2.COLOR_BGR2GRAY)lines=lsd(gray)foriinxrange(lines.shape[0]):pt1=(int(lines[i,0]),int(lines[i,1]))pt2=(int(lines[i,2]),int(lines[i,3]))width=lines[i,4]cv2.line(src,pt1,pt2,(0,0,255),int(np.ceil(width/2)))cv2.imwrite(os.path.join(folder,'cv2_'+imgName.split('.')[0]+'.jpg'),src)by using PIL(Image) modulefromPILimportImage,ImageDrawimportnumpyasnpimportosfrompylsd.lsdimportlsdfullName='house.png'folder,imgName=os.path.split(fullName)img=Image.open(fullName)gray=np.asarray(img.convert('L'))lines=lsd(gray)draw=ImageDraw.Draw(img)foriinxrange(lines.shape[0]):pt1=(int(lines[i,0]),int(lines[i,1]))pt2=(int(lines[i,2]),int(lines[i,3]))width=lines[i,4]draw.line((pt1,pt2),fill=(0,0,255),width=int(np.ceil(width/2)))img.save(os.path.join(folder,'PIL_'+imgName.split('.')[0]+'.jpg'))The following is the result:car.jpg by using cv2 modulehouse.png by using PIL(Image) module
Package: tf1
tf1F1-score metrics for classification models in TensorFlow.
There are 3 average modes provided:binarymacromicroUsagefromtf1importf1_binary# use f1_binary as any other metric from tf.metrics.*Note, that due to streaming nature of metric computation process,
"macro" and "micro" average metrics should know total number of
classes. Use them as follows:fromtf1importf1_macro,f1_microdefmy_task_f1_macro(labels,predictions,num_classes,weights=None,metrics_collections=None,updates_collections=None,name=None):returnf1_macro(labels=labels,predictions=predictions,num_classes=123,# Requiredweights=weights,metrics_collections=metrics_collections,updates_collections=updates_collections,name=name)# use my_task_f1_macro as any other metric from tf.metrics.*
Package: tf14-cli
Terraform 14 CLI: KISS StyleSummaryThis is the design and implementation of providing simple and reliable terraform (tf14) commands for devs ops that build, commit and push.tffiles. Terraform has hidden functionalities that devs may not be familar with. It shouldn't be the dev's responsibility to remember multiple commands every time they make a commit to terraform servers. That's why I decided to create a bash program orchestrated by python for devs to only remember a few commands.Usage: tf14 [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  apply  Apply tf14 data to server.
  clean  Clean unnecessary tf14 files.
  edit   Edit the provided helper files.
  init   Test and fmt .tf code.
  plan   Make plans to local repo.File InformationThe main directory that requires attention is thesrc/tf14directory:api_methods.py- This files provides the users to see incoming messages by creating a global logger variable.cli_utils.py- Creates essential utilities forcli.pythat performs the system commands of the program.cli.py- Bootstrap function forcli_utils.py. This file mainly interacts with the user by running the commandtf14.How do I run the Program?NOTE : This code has only been tested on the latest Ubuntu operating system. Run on macOS at your own risk, but this program is not operational in Windows.Running on Virtual Env (venv)NOTE: You must have python3 installed to run these
commands. Infomation about python can be foundhere.Follow these instructions if you want to run the files using venv./build.sh

# run venv 
source tf14_env/bin/activate 

# use the program as intended
tf14 [OPTIONS]Cleaning Virtual Env (venv) & Dependencies./clean.sh

# exit the virtual environment 
deactivateWho do I talk to?Name- Rooklyn KlineEmail-rkline2@umbc.edu
Package: tf-1.x-rectified-adam
RAdam-TensorflowOn the Variance of the Adaptive Learning Rate and BeyondPaper|Official Pytorch codeUsagefromRAdamimportRAdamOptimizertrain_op=RAdamOptimizer(learning_rate=0.001,beta1=0.9,beta2=0.999,weight_decay=0.0).minimize(loss)AlgorithmResultAuthorJunho Kim
Package: tf-2-13-py310-cuda12
TensorFlow is an open source software library for high performance numerical
computation. Its flexible architecture allows easy deployment of computation
across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters
of servers to mobile and edge devices.Originally developed by researchers and engineers from the Google Brain team
within Google's AI organization, it comes with strong support for machine
learning and deep learning and the flexible numerical computation core is used
across many other scientific domains. TensorFlow is licensed underApache
2.0.
Package: tf2calc
PricesTF Python APIThis modul is a simple python package to calcualte tf2 prices.InstallationThis module requiresPythonv3+ to run.For install the module type this to your command line...pipinstalltf2calcSyntaxcalculate(<amount>, <"type">)
- amount (intriger)
- type (string):
  - "scrap"
  - "rec"
  - "ref"
  - "key"
  - "earbuds"Quick Examplefromtf2clacimport*calculation=calculate(1,"ref")It will return a dictionary with the calculated values.{"scrap":9,"rec":3,"ref":1,"key":0.02,"earbuds":0.0}If you write an unknown type, it will return0.LicenseMIT
Package: tf2crf
tf2crfa simple CRF layer for tensorflow 2 kerassupport keras maskingInstall$pipinstalltf2crfFeatureseasy to use CRF layer with tensorflowsupport mixed precision trainingsupport the ModelWithCRFLossDSCLoss with DSC loss, which increases f1 score with unbalanced data (refer the paperDice Loss for Data-imbalanced NLP Tasks)AttentionAdd internal kernel like CRF in keras_contrib, so now there is no need to stack a Dense layer before the CRF layer.I have changed the previous way that putting loss function and accuracy function in the CRF layer. Instead I choose to use ModelWappers (refered to jaspersjsun), which is more clean and flexible.Tipstensorflow >= 2.1.0
Recommmend use the latest tensorflow-addons which is compatiable with your tf version.Exampleimporttensorflowastffromtf2CRFimportCRFfromtensorflow.keras.layersimportInput,Embedding,Bidirectional,GRU,Densefromtensorflow.keras.modelsimportModelfromtf2crfimportCRF,ModelWithCRFLossinputs=Input(shape=(None,),dtype='int32')output=Embedding(100,40,trainable=True,mask_zero=True)(inputs)output=Bidirectional(GRU(64,return_sequences=True))(output)crf=CRF(units=9,type='float32')output=crf(output)base_model=Model(inputs,output)model=ModelWithCRFLoss(base_model,sparse_target=True)model.compile(optimizer='adam')x=[[5,2,3]*3]*10y=[[1,2,3]*3]*10model.fit(x=x,y=y,epochs=2,batch_size=2)model.save('tests/1')
Package: tf2-data
tf2-dataItem schemas, unusual effects, SKUs and more which can be useful for TF2 trading. Implemented bytf2-utils.DonateBTC:bc1qntlxs7v76j0zpgkwm62f6z0spsvyezhcmsp0z2Steam Trade OfferSetupInstallpipinstalltf2-data# orpython-mpipinstalltf2-dataUpdatingpipinstall--upgradetf2-data# orpython-mpipinstall--upgradetf2-dataTesting# tf2-data/python-munittest
Package: tf2-fm-zoo
tf2-fm-zooPython package for the factorization machine implementations fromtensorflow2_model_zoo.AcknowledgementThe original implementation for the methods in this repo were done byRen Zhangwho kindly granted permission to use his code for the creation of the package.Installationpipinstalltf2_fm_zooBasic Exampleimporttensorflowastfimportnumpyasnpimportpandasaspdfromsklearn.preprocessingimportKBinsDiscretizerfromsklearn.datasetsimportload_bostonfromfm_zoo.fmimportFactorizationMachineX,y=load_boston(return_X_y=True)X=X[:,:3]y=tf.cast(y,dtype=tf.float32)kbd=KBinsDiscretizer(n_bins=15,encode="ordinal")nunique_vals=pd.DataFrame(X).nunique()X=tf.cast(kbd.fit_transform(X),dtype=tf.int64)fm=FactorizationMachine(feature_cards=tf.cast(nunique_vals,tf.int32),factor_dim=3)fm.compile(loss=tf.keras.losses.mean_squared_error,optimizer="Adam")hist=fm.fit(X,y,validation_split=0.15,batch_size=16,epochs=100,callbacks=[tf.keras.callbacks.EarlyStopping(patience=10,restore_best_weights=True)])pd.DataFrame(hist.history).plot(figsize=(15,10))Supported ModelsModelReferenceYearFMFactorization Machines2010FFMField-aware factorization machines for CTR prediction2016FNNDeep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction2016AFMAttentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks2017DeepFMDeepFM: A Factorization-Machine based Neural Network for CTR Prediction2017NFMNerual Factorization Machines for Sparse Predictive Analytics2017xDeepFMxDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems2018AutoIntAutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks2018FNFMField-aware Neural Factorization Machine for Click-Through Rate Prediction2019
Package: tf2helm
tf2helmt2helmis a simple Python utility that converts a Terraform module to a Helm Chart. The Helm Chart contains a Kubernetes Custom Resource understood and managed by one of the following Kubernetes Operators:Isaaguilar Terraform OperatorOpen Application Model Terraform ControllerAppvia Terraform Controllertf2helmreads a Terraform module from a local or remote path and converts it into a Helm chart in a specified directory. It reads the Terraform variables and writes them as.Values.requiredand.Values.optionalvalues in the Helmvalues.yamlfile depending on whether they have been assigned values or not.This gives infrastructure operators e.g. DevOps or Platform engineers the flexibility to set sensible default parameters for cloud resources and make them visible and configurable to application developers who may or may not override them afterwards.Installationpip install tf2helmUsagetf2helm --help
Package: tf2-initializer
tf2-initializerProject sturcture to initialize tf2 projectsInstallTo install the current version run the following:pip install tf2-initializerThis package currently only supports python versions >= 3.7.UsageRunning tf2-initializer will generate a project structure for building a custom subclassed model in tensorflow 2.0.
This project includes support for pre-processing data, defining a model, and training a model.To run the initializer enter the following into your terminal:tf2-initializerRunning the above will prompt you to enter the name of the directory where you would like the tf2-initializer to be spawned.If you would like to use the current directory as the root of your project, simply hit return
Package: tf2-keras-surgeon
Tf2-Keras-SurgeonNote: This is a modification of keras-Surgeon:https://github.com/BenWhetton/keras-surgeonandhttps://github.com/Raukk/tf-keras-surgeon/thats been adopted for specific use. If you are interested try adopting the
original keras-surgeon first, to see if it works for you.
This version specifically changes some of the issues with instalation of keras-surgeon, but also has less features,
and is in general not well tested. For additional information, please look at the repositories above.
Package: tf2md
TF2MDA readme generator for terraform code.AboutHave you ever wanted to generate markdown documentation based onvariables.tforoutputs.tffiles?
This tool aims to solve that.UsageInstall the programpip install tf2mdRun the program <Whereterraform/outputs.tfis the path to my file>:tf2md gen-docs terraform/outputs.tfNote: This does not work ontffiles that have a mix ofoutputsandvariables.See the./examplesdir for examples of generated code.Dev NotesHere are things I want to do:TestsI am not happy with thecreate_variables_filefunction, I want to clean it up,
but right now I just want to use the tool.I want to be able to add an output arg to the cli so the created files do not
just get dummped in the./Verbose logging and options are required
Package: tf2project
Terraform Test FrameworkUnified Test Framework to test Terraform codes and Terraform-provisioned infrastructures.The new version, v0.2.0 (Freedom) is now available.Documentation:https://tf2project.ioSource Code:https://github.com/tf2project/tf2projectChangelog:https://tf2project.io/changelog.htmlTF2is a unified test framework to test Terraform codes and Terraform-provisioned infrastructures. With TF2, you can implement PaC(Policy as Code),Compliancetests and e2e(End to End) tests just in a unified framework.RequirementsTheTF2 coredoesn't have any external or third-party requirements, and we will keep this approach as the main development policy to reduce the attack surface. This policy is created because you run the TF2 in your critical environments, CI/CD systems, and cloud/on-premises infrastructures, and we want to keep you completely secure. The TF2 core is just using standard and official Python libraries.InstallationOnproductionenvironment:pipinstalltf2projectOndevelopmentenvironment:exportENV=development
pipinstallgit+https://github.com/tf2project/tf2projectDocker ImageYou can also use our official Docker image to run the framework. The imagetf2project/tf2projectconsists of the latest version of Terraform and the TF2 framework itself.It's a ready-to-go image to integrate the TF2 within CI/CD pipelines.dockerruntf2project/tf2project:latestLicenseThis project is licensed under the terms of the Apache 2.0 license.Copyright © 2022 Saeid Bostandoust
Package: tf2-resnets
TensorFlow-ResNetsThis repository contains TensorFlow Keras ResNet models.
Below, you will find the supported variants of ResNet and what weights are supported.The codebase takes inspiration fromTensorFlow ResNetsandPyTorch ResNets.This repository is compatible with TF 2.2.0 even though grouped convolutions are only
supported in TF Nightly.
Grouped convolutions will be imported fromconvolutions.pywhich is the TF Nightly version of convolution layers.
This allows the use of TPUs since they don't work with TF Nightly.These modelswill not work without a GPU or TPUdue to the use of grouped convolutions.InstallationTo install, you can usepip install tf2-resnets.To use a model, you can dofromtf2_resnetsimportmodels# Weights here are ImageNet.# They can also be 'ssl' (semi-supervised)# or 'swsl' (semi-weakly supervised)# for a selection of models.model=models.ResNeXt50(input_shape=(224,224,3),weights='imagenet')ModelsArchitectureWeightsTop-1 Acc.Top-5 Acc.ResNet-18ImageNet69.889.1ResNet-34ImageNet73.391.4ResNet-50ImageNet76.292.9ResNet-101ImageNet77.493.6ResNet-152ImageNet78.394.1ResNeXt-50 32x4dImageNet77.693.7ResNeXt-101 32x8dImageNet79.394.5Wide ResNet-50 2ImageNet78.594.1Wide ResNet-101 2ImageNet78.894.3ResNeSt-50ImageNet81.0*N/AResNeSt-101ImageNet82.8*N/AResNeSt-200ImageNet83.8*N/AResNeSt-269ImageNet84.5*N/AResNet-18semi-supervised72.891.5ResNet-50semi-supervised79.394.9ResNet-18semi-weakly supervised73.491.9ResNet-50semi-weakly supervised81.296.0ResNeXt-50 32x4dsemi-supervised80.395.4ResNeXt-101 32x8dsemi-supervised81.796.1ResNeXt-50 32x4dsemi-weakly supervised82.296.3ResNeXt-101 32x8dsemi-weakly supervised84.397.2* ResNeSt models' Top-1 Accuracies were reported using different crop sizes.
Crop sizes in order -224, 256, 320, and 416.PreprocessingThere are two different preprocessing functions which are meant to be used
according to the task at hand. The first one, located inimagenet_preprocessing.py,
is meant to be used for reproducing the ImageNet results. The second function
is located insideresnet.py. This function is meant to be used for transfer learning.
The first function includes an additional resize to 256 by 256 and a central crop to 224 by 224.Original ImplementationsThe original implementions of these models are listed below.ResNet(PyTorch)ResNeSt(PyTorch)ResNeXt(PyTorch)Wide ResNet(PyTorch)