Saravanan666/pypi-org huggingface dataset:
Package: longleding-static-file-service-sdk
Longleding Static File Service SDKSupported Python VersionsPython >= 3.6Installationlongleding-static-file-service-sdk is available for Linux, macOS, and Windows.$pipinstalllongleding-static-file-service-sdkBasic Usage# -*- coding: utf-8 -*-importstatic_file_servicefromstaticFileService_pb2importFileACLEnumstatic_file_service_endpoint="localhost:80"static_file_service.init_service(endpoint=static_file_service_endpoint)if__name__=='__main__':test_bytes=b'test file'print(static_file_service.get_oss_bucket_info())print(static_file_service.get_upload_credentials())static_file_service.put_file('upload/t.txt',test_bytes)static_file_service.put_file_acl('upload/t.txt',FileACLEnum.PRIVATE)print(static_file_service.get_file('upload/t.txt'))print(static_file_service.get_file_md5('upload/t.txt'))static_file_service.copy_file('upload/t.txt','upload/t1.txt')print(static_file_service.get_file_md5('upload/t1.txt'))static_file_service.move_file('upload/t.txt','upload/t2.txt')print(static_file_service.get_file_md5('upload/t2.txt'))TroubleshootIf you encounter error messages similar to the following:...
TypeError:Couldn'tbuildprotofileintodescriptorpool!
Invalidprotodescriptorforfile"common.proto":common.proto:Afilewiththisnameisalreadyinthepool.Setting an environment variable the following before running:exportPROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python'See also[Python] A file with this name is already in the pool.Python Generated Code
Package: longleding-system-settings-service-sdk
Longleding System Settings Service SDKSupported Python VersionsPython >= 3.6Installationlongleding-system-settings-service-sdk is available for Linux, macOS, and Windows.$pipinstalllongleding-system-settings-service-sdkBasic Usage# -*- coding: utf-8 -*-fromsdkimportsystem_settings_servicesystem_settings_service_endpoint="localhost:80"source_name="demo"system_settings_service.init_service(endpoint=system_settings_service_endpoint,src=source_name)if__name__=='__main__':system_settings_service.get_version_doc_shot(1)system_settings_service.get_last_version_doc_shot()TroubleshootIf you encounter error messages similar to the following:...
TypeError:Couldn'tbuildprotofileintodescriptorpool!
Invalidprotodescriptorforfile"common.proto":common.proto:Afilewiththisnameisalreadyinthepool.Setting an environment variable the following before running:exportPROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python'See also[Python] A file with this name is already in the pool.Python Generated Code
Package: longmao-open-sdk-python
longmao-open-sdk-pythonThe official LongMao SDK for Python.访问龙猫数据开放平台的官方SDK。LinksWebsite:https://www.longmaosoft.comExample.. code-block:: python#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
import traceback

from longmao.api.project.ApiProjectTaskCreate import ApiProjectTaskCreate
from longmao.core.DefaultLongMaoClient import DefaultLongMaoClient
from longmao.core.LongMaoClientConfig import LongMaoClientConfig

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s %(levelname)s %(message)s',
    filemode='a',)
logger = logging.getLogger('')


if __name__ == '__main__':
    """
    设置配置。
    """
    longmao_client_config = LongMaoClientConfig()
    longmao_client_config.access_key_id = '58bf489978ed44c49a4b9c64e96d9d6f'
    longmao_client_config.access_key_secret = 'bacb00c2e0200ccd7e997b53c9efb62d795b70e9'

    """
    得到客户端对象。
    """
    client = DefaultLongMaoClient(longmao_client_config, logger)

    """
    系统接口示例：批量添加任务
    """
    # 对照接口文档，构造请求对象
    api = ApiProjectTaskCreate()
    api.project_id = 'd36ec84f-ea18-4d1d-ae33-33bbad816f11'
    api.file = {'file': open('/longmao/data/demo.csv', 'rb')}

    result = None
    try:
        result = client.execute(api)
    except Exception as e:
        print(traceback.format_exc())
    if not result:
        print("failed execute")
    else:
        if result['code'] == '200':
            # 成功
            print("get response job_id:" + result['object']['job_id'])
        else:
            # 失败
            print(result['code'] + "," + result['message'])
Package: longnet
LongNet: Scaling Transformers to 1,000,000,000 TokensThis is an open source implementation for the paperLongNet: Scaling Transformers to 1,000,000,000 Tokensby Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Furu Wei. The LongNet is a Transformer variant designed to scale sequence length up to more than 1 billion tokens without sacrificing performance on shorter sequences.InstallationpipinstalllongnetUsageOnce you have installed LongNet, you can use theDilatedAttentionclass as follows:importtorchfromlong_netimportDilatedAttention# model configdim=512heads=8dilation_rate=2segment_size=64# input databatch_size=32seq_len=8192# create model and datamodel=DilatedAttention(dim,heads,dilation_rate,segment_size,qk_norm=True)x=torch.randn((batch_size,seq_len,dim))output=model(x)print(output)LongNetTransformerA fully ready to train transformer model with dilated transformer blocks with Feedforwards with layernorm, SWIGLU, and a parallel transformer blockimporttorchfromlong_net.modelimportLongNetTransformerlongnet=LongNetTransformer(num_tokens=20000,dim=512,depth=6,dim_head=64,heads=8,ff_mult=4,)tokens=torch.randint(0,20000,(1,512))logits=longnet(tokens)print(logits)TrainTo run a simple training run on the enwiki8 dataset, gitclone, install the requirements.txt, and then runpython3 train.pyLongNet SummarizedScaling sequence length has become a critical bottleneck in the era of large language models. However, existing methods struggle with either computational complexity or model expressivity, rendering the maximum sequence length restricted. In this paper, they introduce LongNet, a Transformer variant that can scale sequence length to more than 1 billion tokens, without sacrificing the performance on shorter sequences. Specifically, they propose dilated attention, which expands the attentive field exponentially as the distance grows.FeaturesLongNet has significant advantages:It has a linear computation complexity and a logarithm dependency between tokens.It can be served as a distributed trainer for extremely long sequences.Its dilated attention is a drop-in replacement for standard attention, which can be seamlessly integrated with the existing Transformer-based optimization.Experiment results demonstrate that LongNet yields strong performance on both long-sequence modeling and general language tasks. Their work opens up new possibilities for modeling very long sequences, e.g., treating a whole corpus or even the entire Internet as a sequence.Citation@inproceedings{ding2023longnet,title={LongNet: Scaling Transformers to 1,000,000,000 Tokens},author={Ding, Jiayu and Ma, Shuming and Dong, Li and Zhang, Xingxing and Huang, Shaohan and Wang, Wenhui and Wei, Furu},booktitle={Proceedings of the 10th International Conference on Learning Representations},year={2023}}TodoFix the ParallelTransformer Block's forward pass with dilated attnTrain on enwiki 8 and testCreate multihead iteration
Package: longPolling
pipinstalllongPolling客户端单向传输importlongPollingexample=longPolling.client.Client(url,callback)example.login(uername)双向传输importlongPollingexample=longPolling.client.BothwayClient(url,callback)example.login(uername)example.send(data)服务端单向传输importlongPollingexample=longPolling.server.Server(host,port)example.send(username,message)双向传输importlongPollingexample=longPolling.server.BothwayServer(host,port,callback)example.send(username,message)关于作者作者主页宽宽2007pypilongPolling · PyPI本项目在苟浩铭/长轮询
(gitee.com)上开源帮助文档参见宽宽的帮助文档
Package: longport
LongPort OpenAPI SDK for Pythonlongportprovides an easy-to-use interface for invokesLongPort OpenAPI.QuickstartInstall LongPort OpenAPI SDKpipinstalllongportSetting environment variables(MacOS/Linux)exportLONGPORT_APP_KEY="App Key get from user center"exportLONGPORT_APP_SECRET="App Secret get from user center"exportLONGPORT_ACCESS_TOKEN="Access Token get from user center"Setting environment variables(Windows)setxLONGPORT_APP_KEY"App Key get from user center"setxLONGPORT_APP_SECRET"App Secret get from user center"setxLONGPORT_ACCESS_TOKEN"Access Token get from user center"Quote API(Get basic information of securities)fromlongpport.openapiimportConfig,QuoteContext# Load configuration from environment variablesconfig=Config.from_env()# Create a context for quote APIsctx=QuoteContext(config)# Get basic information of securitiesresp=ctx.quote(["700.HK","AAPL.US","TSLA.US","NFLX.US"])print(resp)Quote API(Subscribe quotes)fromtimeimportsleepfromlongport.openapiimportConfig,QuoteContext,SubType,PushQuote# Load configuration from environment variablesconfig=Config.from_env()# A callback to receive quote datadefon_quote(symbol:str,event:PushQuote):print(symbol,event)# Create a context for quote APIsctx=QuoteContext(config)ctx.set_on_quote(on_quote)# Subscriberesp=ctx.subscribe(["700.HK"],[SubType.Quote],is_first_push=True)# Receive push duration to 30 secondssleep(30)Trade API(Submit order)fromdecimalimportDecimalfromlongport.openapiimportTradeContext,Config,OrderType,OrderSide,TimeInForceType# Load configuration from environment variablesconfig=Config.from_env()# Create a context for trade APIsctx=TradeContext(config)# Submit orderresp=ctx.submit_order("700.HK",OrderType.LO,OrderSide.Buy,Decimal("500"),TimeInForceType.Day,submitted_price=Decimal("50"),remark="Hello from Python SDK")print(resp)LicenseLicensed under either ofApache License, Version 2.0,(LICENSE-APACHEorhttp://www.apache.org/licenses/LICENSE-2.0)MIT license (LICENSE-MITorhttp://opensource.org/licenses/MIT) at your option.
Package: longpython
longpythonlooooooooooooooooooooong python!Usage$ longpython
  _
(" ヽ
   \ \
   / /
   \ \ _ ,
    \___/

$ longpython -h
usage: longpython [-h] [-l INT]

CLI tool to print long python

optional arguments:
  -h, --help            show this help message and exit
  -l INT, --length INT  length of python (default: 1)Installationlongpython requires python 3.9 or later.Via pipx (recommended)I recommend usingpipxto install this project.pipxinstalllongpythonVia pippipinstalllongpythonDownload Single FileThe project does not have any dependencies, you can downloadthis fileand use as soon as.Derived Projectssyumai/longify: A command to output
longified any ascii artarrow2nd/longdeno:
LooooooooooooooooooooooooooooooooooooooooooooooongDenoikanago/longferris: LongFerriswritten in Rustsheepla/longgopher: ʕ◉ϖ◉ʔ
loooooooooooooooooooooong gopherLicenseMIT
Package: long-range-models
Long Range ModelsA collection of simple implementations of long-range sequence models, includingLRU,S5, andS4.
More implementations to come.Install$pipinstalllong_range_modelsUsageThis library offers detailed documentation for every module and layer implemented.
Models are created by composing different pieces together.
Check out the examples below.Discrete sequence dataConsider a language model built with an LRU sequence layer and the architecture proposed in the S4 paper:fromfunctoolsimportpartialimportjax.randomasjrandomfromlong_range_modelsimportSequenceModel,S4Module,LRULayerrng=jrandom.PRNGKey(0)model=SequenceModel(num_tokens=1000,module=S4Module(sequence_layer=partial(LRULayer,state_dim=256),dim=128,depth=6,),)x=jrandom.randint(rng,(1,1024),0,1000)variables=model.init(rng,x)model.apply(variables,x)# (1, 1024, 1000)Continuous sequence dataFor sequences with continuous values, the setup looks as follows:fromfunctoolsimportpartialimportjax.randomasjrandomfromlong_range_modelsimportContinuousSequenceModel,S4Module,LRULayerrng=jrandom.PRNGKey(0)model=ContinuousSequenceModel(out_dim=10,module=S4Module(sequence_layer=partial(LRULayer,state_dim=256),dim=128,depth=6,),)x=jrandom.normal(rng,(1,1024,32))variables=model.init(rng,x)model.apply(variables,x)# (1, 1024, 10)Note:both model types offer several customization options. Make sure to check outtheir documentation.Upcoming featuresMore implementations:Extend the library with models like S4D, S4Liquid, BiGS, Hyena, RetNet, SGConv, H3, and others.Customization:Allow users to better customize currently implemented layers and architectures (e.g., activation functions, initialization, etc.).Sequential API:Allow recurrent models to run sequentially, allowing for efficient inference.
Package: longstaff-schwartz
Longstaff-Schwartz AlgorithmA Python implementation of the Longstaff-Schwartz linear regression algorithm for the evaluation of call rights and American options.Seminal paper:Francis A. Longstaff, Eduardo S. Schwartz,Valuing American Options by Simulation: A Simple Least-Squares Approach(The Review of Financial Studies) (2001) Vol 14, No 1, pp. 113-147Documentation:https://longstaff-schwartz.readthedocs.ioFree software: MIT licenseTalksPyConDE 2019-10-10:Slides,Jupyter NotebookPyData Meetup 2019-09-18:Slides,Jupyter NotebookUsagefromlongstaff_schwartz.algorithmimportlongstaff_schwartzfromlongstaff_schwartz.stochastic_processimportGeometricBrownianMotionimportnumpyasnp# Model parameterst=np.linspace(0,5,100)# timegrid for simulationr=0.01# riskless ratesigma=0.15# annual volatility of underlyingn=100# number of simulated paths# Simulate the underlyinggbm=GeometricBrownianMotion(mu=r,sigma=sigma)rnd=np.random.RandomState(1234)x=gbm.simulate(t,n,rnd)# x.shape == (t.size, n)# Payoff (exercise) functionstrike=0.95defput_payoff(spot):returnnp.maximum(strike-spot,0.0)# Discount factor functiondefconstant_rate_df(t_from,t_to):returnnp.exp(-r*(t_to-t_from))# Approximation of continuation valuedeffit_quadratic(x,y):returnnp.polynomial.Polynomial.fit(x,y,2,rcond=None)# Selection of paths to consider for exercise# (and continuation value approxmation)defitm(payoff,spot):returnpayoff>0# Run valuation of American put optionnpv_american=longstaff_schwartz(x,t,constant_rate_df,fit_quadratic,put_payoff,itm)# European put option for comparisonnpv_european=constant_rate_df(t[0],t[-1])*put_payoff(x[-1]).mean()# Check resultsassertnp.round(npv_american,4)==0.0734assertnp.round(npv_european,4)==0.0626assertnpv_american>npv_europeanPlotsFor details seePyData Meetup Jupyter Notebook.Approximation of continuation valueFavourable exerciseCreditsMain developer isluphord.Primary source for the algorithm isFrancis A. Longstaff, Eduardo S. Schwartz,Valuing American Options by Simulation: A Simple Least-Squares Approach(The Review of Financial Studies) (2001) Vol 14, No 1, pp. 113-147.
There is no affiliation between the authors of the paper and this code.This package was prepared withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.2.0 (2022-11-10)Drop support for Python 3.7Add support for Python 3.9 and 3.10Upgrade dependenciesReformat code with blackMigrate from travis-ci.com to GitHub actionsUpgrade development status to alpha0.1.1 (2020-12-01)Support Python 3.8Migrate to travis-ci.comIncrease number of simulated paths in example to prevent poor conditioning warning0.1.0 (2019-10-03)First release on PyPI.
Package: longtang
What’s Longtang?Longtang is a command-line software based onactors modelcapable of processing any sloppy mp3 collection that you have into a well-organized, easy-to-locate tagged media library.Why should I use it?Because it will save you aHUGEamount of time by:Tagging all yourmp3files with the right ID3Tag information.Downloading album cover art(New feature!!)Decompressing and tagging anyzip,raror7zfile that you have with music on it transparently.Creating a hierarchical media tree organized by artists and albums.Naming all files in a simple and yet clear syntax.Detecting tag issues that prevent files from being accurately classified.Avoiding any stranded or unknown music files inside any portable music player (which usually happens in Creative Zen players, just to name an example) which take up storage space and are not easily detected.Installation guide:Before proceeding with the installation, please check if your system already has the following packages installed:Python 2.7.x and headers (python-dev)Chromaprint tools (libchromaprint-tools)Libevent headers (libevent-dev)Libxml (libxml2-dev and libxslt-dev)unrar and 7z commands available within your systemPATHAfter that, you can proceed to install in either two manners: from the source code or fromPypiby usingpiputility.1. Using the source codeCheckout the source code:git clone https://bitbucket.org/gszeliga/longtang.gitExecute the setup utility as follows:python setup.py install2. UsingpipJust type on your command line:pip install longtangCommand guide:The main binary is calledlongtang(as you can imagine) and it accepts the following parameters:--sourceSource path where all the to-be-processed files are located.--targetTarget path where the hierarchical structure will be created.--verbosityLevel of debug information to be printed. Accepted values are:DEBUG,INFO,WARNorERROR. Default value isNONE.--override-tags(Optional) Whether you wantLongtangto override any id3tag information on the source music files.Bear in mind that if the amount of files is high it will turn the process to be real slow.--offline(Optional) Since any missing id3tag information will be retrieved usingAcoustidservice, maybe you might be interested in not performing this action and just handle your media collection with the already existing id3 information and point out any failure during the process.--disable-cover-art(Optional) Disables resolution of all cover art from all matched artists and albums.--helpPrints out help information.What’s coming next?Well, i currently have a lot of ideas but, in a short term, i’m thinking in:Supporting more music formats: ogg, mpc, flac and so on.Porting source code into Python 3.Replacinggeventwithmultiprocessinglibrary so that we gain executionparallelism(take a look at branchfeatures/multiprocessingfor further details).Bugs and FeedbackYou can contact me atgszeliga@gmail.comor follow me on twitter:@gszeliga
Package: longtask
Long task runner with few nice features.longtaskis a simple framework to run long tasks (eg. data migration, data processing) in console environment.
During process execution you can track changes on progress bar. If something goes wrong or you don’t have time now - you
can stop task and rerun it later.longtaskcollects all your task stats and store it in JSON file.How to create task?If you want to create task you need to implement derived class fromlongtask.Task. You also need to override few
things:name- task name,get_items()- this method should return you items collection,process_item(item)- in this method you can process your item.Simple example - resize many pictures (resize.py):import longtask
import glob
from PIL import Image


class ResizeTask(longtask.Task):
    name = 'resize'

    def get_items(self):
        return glob.glob('*.png')

    def process_item(self, item):
        original_image = Image.open(item)
        resized_image = original_image.resize((100, 100))
        resized_image.save('output/' + item)


if __name__ == '__main__':
    ResizeTask().run()Now in directory with*.pngimages you can run:$ python resize.py
[*] Starting task: resize [2013-03-03 18:36:29.953306]
[*] Processed: 100% (1000/1000) errors: 10% (100/1000) |###################| Elapsed Time: 1:00:00 ETA:  0:00:00
[*] Finished task: resize [2013-03-03 18:36:30.080421]
[*] Stats:
  - processed: 100% (1000/1000)
  - success: 90% (900/1000)
  - errors: 10% (100/1000)
    - 100 x IOError with traceback:

Traceback (most recent call last):
  File "/Users/konradhalas/dev/workspace/personal/longtask/longtask/task.py", line 59, in run
    self.process_item(item)
  File "resize.py", line 13, in process_item
    original_image = Image.open(item)
  File "/Users/konradhalas/dev/virtualenvs/longtask/lib/python2.7/site-packages/PIL/Image.py", line 1980, in open
    raise IOError("cannot identify image file")
IOError: cannot identify image fileIn our example we resized 1000 files - 100 of them have corrupted data (IOError). This task took 1h.You can always stop task withCtrl+Cand rerun it later with flag-c. If some errors occured durring execution
you can rerun it with flag-e(it works only with flag-c).LicenseCopyright [2013] [Konrad Hałas]Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Package: longtrainer
LongTrainer - Production-Ready LangChainFeatures 🌟✅Long Memory:Retains context effectively for extended interactions.✅Unique Bots/Chat Management:Sophisticated management of multiple chatbots.✅Enhanced Customization:Tailor the behavior to fit specific needs.✅Memory Management:Efficient handling of chat histories and contexts.✅GPT Vision Support:Integration Context Aware GPT-powered visual models.✅Different Data Formats:Supports various data input formats.✅VectorStore Management:Advanced management of vector storage for efficient retrieval.Works for All Langchain Supported LLM and Embeddings✅ OpenAI (default)✅ VertexAI✅ HuggingFaceExampleVertexAI LLMsfromlongtrainer.trainerimportLongTrainerfromlangchain_community.llmsimportVertexAIllm=VertexAI()trainer=LongTrainer(mongo_endpoint='mongodb://localhost:27017/',llm=llm)TogetherAI LLMsfromlongtrainer.trainerimportLongTrainerfromlangchain_community.llmsimportTogetherllm=Together(model="togethercomputer/RedPajama-INCITE-7B-Base",temperature=0.7,max_tokens=128,top_k=1,# together_api_key="...")trainer=LongTrainer(mongo_endpoint='mongodb://localhost:27017/',llm=llm)Usage Example 🚀pipinstalllongtrainerHere's a quick start guide on how to use LongTrainer:fromlongtrainer.trainerimportLongTrainerimportos# Set your OpenAI API keyos.environ["OPENAI_API_KEY"]="sk-"# Initialize LongTrainertrainer=LongTrainer(mongo_endpoint='mongodb://localhost:27017/',encrypt_chats=True)bot_id=trainer.initialize_bot_id()print('Bot ID: ',bot_id)# Add Datapath='path/to/your/data'trainer.add_document_from_path(path,bot_id)# Initialize Bottrainer.create_bot(bot_id)# Start a New Chatchat_id=trainer.new_chat(bot_id)# Send a Query and Get a Responsequery='Your query here'response=trainer._get_response(query,bot_id,chat_id)print('Response: ',response)Here's a guide on how to use Vision Chat:chat_id=trainer.new_vision_chat(bot_id)query='Your query here'image_paths=['nvidia.jpg']response=trainer._get_vision_response(query,image_paths,str(bot_id),str(vision_id))print('Response: ',response)List Chats and Display Chat History:trainer.list_chats(bot_id)trainer.get_chat_by_id(chat_id=chat_id)This project is still under active development. Community feedback and contributions are highly appreciated.