Saravanan666/pypi-org huggingface dataset:
Package: sftp-to-s3
Welcome tosftp_to_s3Documentationsftp_to_s3is an Open Source solution to ingest large amount of data from SFTP server to AWS S3.How it WorkThis solution assume that the SFTP server folder structure has to have partition strategy. For example, the SFTP server folder structure is like this:/data/2022-01-01/many-folder-many-files...
/data/2022-01-02/many-folder-many-files...
/data/2022-01-03/many-folder-many-files...
/data/...Once a partition of files are created on SFTP server, it should not be changed to frequently. Files in each partition are grouped into many batches to ensure that the total amount of files and total file size has a reasonable boundary. A coordinator will be responsible to detect the delta of partitions on SFTP and AWS S3, and then group files into batches. Then the batch data will be sent to a cluster of worker to download in parallel if it fits the SFTP server’s network bandwidth.The coordinator and worker can be deployed as either AWS Lambda function or AWS Batch job. Also this solution uses DynamoDB to track the status of each partition and each batch to ensure the data integrity.Please read the following links to learn more:Architecture designSample project using this librarySample script to generate fake data for integration testSample script to run the downloaderPartition Tracker Data:{
    "n_batches": 2, # this partition has 2 batches to download
    "n_succeeded_batches": 2, # number of succeeded batch
    "sftp_dir": "/home/username/part0001", # SFTP directory of this partition
    "s3_dir": "s3://bucket/projects/sftp_to_s3/download/part1", # where you store the downloads
    "n_files": 6, # total number of files in this partition
    "total_size": 5430500 # total size of files in this partition
}Request Tracker Data:# this is the first batch
{
    "partition_key": "part1", # partition key
    "request_id": "request1", # request id / batch id
    "sftp_dir": "/home/username/part1", # SFTP directory of this partition
    "s3_dir": "s3://bucket/projects/sftp_to_s3/download/part1", # where you store the downloads
    "s3uri_batch": "s3://bucket/projects/sftp_to_s3/requests/part1/request1.json", # where you store the batch data
    "n_files": "3", # total number of files in this batch
    "total_size": 3442228 # 3.4 MB # total size of files in this batch
}Batch data JSON file:# this is the sample content of the batch data
# it is the "s3uri_batch" field in Request Tracker Data
{
    "dir_root": "/home/username/part1", # SFTP directory of this partition
    "s3uri_root": "s3://bucket/projects/sftp_to_s3/download/part1", # where you store the downloads
    "files": [ # list of files to download
        "/home/username/part1/file0001.txt",
        "/home/username/part1/file0002.txt",
        "/home/username/part1/file0003.txt"
    ],
    "n_files": "3", # total number of files in this batch
    "total_size": 3442228 # 3.4 MB # total size of files in this batch
}Installsftp_to_s3is released on PyPI, so all you need is:$pipinstallsftp_to_s3To upgrade to latest version:$pipinstall--upgradesftp_to_s3
Package: sftp-uploader
Table of contentWhat's this, what it's need it?- description of packageInstallation- how to install packageSetup- how to setup packageHow script is work- description of workingTODO- what i need todo in nearby future (task list for me)🧐What's this, what it's need it?This script (library) need for upload to sftp before commit files which was changed. It's very comfortable script for me, but if you need it you can use it ! :)📥InstallationWith pip:pip install sftp-uploaderWith poetry:poetry add sftp-uploader⚙️ SetupFor setup sftp config, you need to execure commandsftp_setup. After this, you need to inputhost,port,user,passwordof your sftp server, and after that input a path to remote directory which contain your project,
for example:Your project in /path/to/my/project
Your input `/path/to/my/project` <b>WITHOUT</b> slash in the end (in future a fix it, but now - it's work how it's work), and this directory need to be <b>EXIST</b>After that script add config file to.gitignoreand add script to load files inpre-commithook file of git. And that's it! Your perfect!!!You can change in any time config (config name -sftp_config.jsonin root project directory), if you need it, or if any step is broken you can re-execute command and get success result, if
any step do not need it, script just skip it and all.💪How script is workScript after you execure commandsftp_setupask you a questions about sftp configuration, exactlyhost,port,username,passwordof sftp connection and after that you remote directory of project root.
After you fill this mini-form, a script create a filesftp_config.jsonwhich contain all of this information (not encrypted, it's just ajsonformat), and with configuration of sftp and filling form is end.Next step it's creategit(if not exists) andgitignorefile (if not exist), after that, script addsftp_config.jsonfile togitignore(i think you don't need save your sftp data in your github 😁).In the next step, script create or use existedpre-commitfile, which containbashscript which start before you make a commit, you can see a little logs about proccess, how it start,
which files upload which not upload, and after that you can make a commit.🤔TODO[x] Make title, description, which problem is solve, metter of this project description[ ] Add moreclassifiers[ ] Add tests[x] Add additional links to project conf[x] Add poetry scripts and pre-commit scripts[x] Add poetry post install, if git exists and to gitignore and setup pre commit hook, and setup config module, if not exists, create and setup it
Package: sftputil
sftputilPython High-level SFTP client libraryDocumentation is available onreadthedocs,
or indocs/index.md.Descriptionsftputilis Python library to transfer files using SFTP. At this point only
the API is available, but a command line will probably be added in the future.Why this package?Paramiko provides aSFTPclient but it does not contain many methods. It is alright if one only needs
simple get/put/list commands. But it is not enough for more complex operations.pySFTPwould have been the solution, but
it has not been updated since 2016 (at the time of this writing). It is a dead
project and cannot be improved. It does not manage the last SSH key
algorithms.Thus this new project. The initial reason was also that I needed arsync-like
command through SFTP in Python scripts.InstallationAvailable onpypi.pip install sftputilUsageTODOSupportIf you have any question or suggestion, you can open a newissue.RoadmapTODO for future releases:Add synchronisation on the other direction (push)Add a command lineAdd some unit testsContributingYou are free to fork this repository and to do Merge Requests that I will
review.Authors and acknowledgmentMain author: Romain TAPRESTromain@taprest.frLicenseLicensed underMozilla Public License v2.Project statusAlive! 💓
Package: sftpwrapper
OverviewA Python library that facilitates uploading and downloading of files to/from an SFTP server using either username/password or username/sshkey.Install ItFrom PyPI$ pip install sftpwrapperNew in v0.2.1Defaults to port 22Logging works with the new version of ParamikoHostKey and Private Key support for connecting to SFTP server (have to provide the file path and encryption)Code Examplefrom sftpwrapper import SftpWrapperremote_connection = SftpWrapper(host, port)
remote_connection.host_connect(user, password)
upload_good = remote_connection.upload_stuffs(csv_name, remote + csv_name)
remote_connection.host_disconnect()DependenciesparamikoLicenseMIT
Package: sfunc
sfuncSingle lambda function for AWS Api GatewayThis is a very simple package which adds a few decorators to simplify the use of a single lambda function for multiple
endpoints for AWS Api Gateway. It uses theoperationIdattribute in the OAS3 specification to identify the correct
handler and then passes the event and context to the correct method.Basic usageimportjsonfromsfuncimportSFuncsfunc=SFunc()@sfunc.operation('getPetsByName')defget_pets_by_name(event,context):"""The methods need to have the same signature as a regular lambda handler entrypoint"""return{'statusCode':200,'body':json.dumps({'names':['fido','catty']})}@sfunc.operation('getPetsByAge')defget_pets_by_age(event,context):"""This will be invoked when the endpoint with the operationId 'getPetsByAge' is invoked"""return{'statusCode':204}deflambda_handler(event,context):"""We only need to specify one entry point and can ignore any other """rsp=sfunc.execute(event,context)# rsp will contain the return value from the invoked methodreturnrspUsing sfunc classesFor request/response which doesn't require any additional and are json encoded, sfunc provides two convenience classes
for request and response which can simplify the code further when it comes to parsing and getting headers and path
params. The following code will result in the same outcome as the example above, except for the extra headers.fromsfuncimportApiGatewayRequest,ApiGatewayResponse,SFuncsfunc=SFunc()@sfunc.operation('getPetsByName')defpets_by_name(req:ApiGatewayRequest,context)->ApiGatewayResponse:_=req.headers().get('x-request-id')returnApiGatewayResponse(200,{'names':['fido','catty']})@sfunc.operation('getPetsByAge')defpets_by_age(req:ApiGatewayRequest,context)->ApiGatewayResponse:rsp=ApiGatewayResponse(204)rsp.add_header('content-type','application/json')returnrspdeflambda_handler(event,context):returnsfunc.sfunc_execute(ApiGatewayRequest(event),context).response()Specifying default handlersIt might be useful during development or if some endpoints are missing the operationId attribute in the specification.fromsfuncimportSFuncsfunc=SFunc()@sfunc.default()defdefault_handler(event,context):return{'statusCode':405}deflambda_handler(event,context):returnsfunc.execute(event,context)# or sfunc.sfunc_execute(ApiGatewayRequest) -> ApiGatewayResponse as above
Package: sfz
SFZA simple Package to deal with Mainland ID Card Number.
一个用来处理大陆身份证号码的小包。用法先安装啦~pip3installsfz然后引入这个包：fromsfzimport*获得户籍归属地正常情况下会返回地址，否则返回 "Unknown"。>>>s="432503197505028819">>>object=Sfz(s)>>>object.loc()'湖南省娄底地区涟源市'获得性别>>>object.gender()'男'地址列表來自：https://gist.github.com/mayufo/4207ed3fa925e6b3df7559832af85165。
Package: sfzlint
Linter and parser for .sfz filesCLI programs are mostly done.Includes thesfzlintandsfzlistcommand line utilitiessfzlintwill parse and validate sfzfiles. If a directory is passed it will be recursivly searched for sfz files.$ sfzlint path/to/file.sfz
path/to/file.sfz:60:11:W continuous not one of ['no_loop', 'one_shot', 'loop_continuous', 'loop_sustain'] (loop_mode)
path/to/file.sfz:98:18:W 8400 not in range -1 to 1 (fileg_depthccN)
path/to/file.sfz:107:12:E expected integer got 0.1 (lfoN_freq)
path/to/file.sfz:240:1:W unknown opcode (ampeg_sustain_curveccN)sfzlistwill print a list of known opcodes and metadata to stdout. Callig with--pathwill cause it to print opcodes found in that path$ sfzlist --path /sfz/instra/Scarypiano/
amplitude_onccN aria Range(0,100) modulates=amplitude
lokey v1 Range(0,127)
ampeg_release_onccN v2 Alias(ampeg_releaseccN)
label_ccN aria Any()
bend_up v1 Range(-9600,9600)Opcode data is fromsfzformat.com. If you see a bug insyntax.ymlconsider putting you PR
againstthe sourceFeaturessyntax validationchecks opcodes against known opcodes on sfzformat.comvalidates opcode values when min or max or type are defined in the specvalidates*_curveccvalues above 7 have a corresponding<curve>headerchecks that sample files exists, also checks that case matches for portability with case-sensitive filesystemspulls in #includes and replaces vars from #definesvalidation based on aria .xml filesHowToIf you have a project that is seperated into several.sfzfiles using#includemacros
Example:instra.sfz
samples/
   a#1.wav
   b1.wav
   ...
includes/
   piano.sfz
   forte.sfz
   ...To validate the whole project you can usesfzlint --check-includes instra.sfz.
Running sfzlint against a program.xmlfile will check includes by default.
If you runsfzlint includes/piano.sfzandpiano.sfzhas some sample opcodes you may get file not found errors.
To fix this run with--rel-pathsfzlint includes/piano.sfz --rel-path .InstallingI've not put this on pypi yet. You can install with pippip install pyyaml
pip install git+git://github.com/jisaacstone/sfzlint.gitOr clone the repo and usepython setup.py installBoth methods require python version >= 3.6To use with vim/neomake:(This is what I built this thing for)put the following in your .vimrc:au BufNewFile,BufRead *.sfz set filetype=sfz
let g:neomake_sfz_enabled_makers=['sfzlint']
let g:neomake_sfzlint_maker = {'exe': 'sfzlint', 'errorformat': '%f:%l:%c:%t %m'}
Package: sg
AWSのセキュリティグループを管理するツールです。セキュリティグループの設定をcsvでダウンロード、コメントをつけてバージョン管理、追加したものをAPI経由で保存という使い方ができます。## インストール```$ pip install sg```OR```$ python setup.py install```## 使い方適切なディレクトリを作成し`init`コマンドを実行してください。設定が保存されます。```$ cd yourpath$ sgsg.py initenter your region[us-east-1]:us-west-2save to sg.cfgenter your aws_access_key_id:xxxenter your aws_secret_access_key:xxxsave to aws_key```### リモートからローカルへの同期以下AWS上に保存された設定を「リモート設定」、csvとして保存されている設定を「ローカル設定」と呼びます。リモート設定を保存するには`fetch`コマンドを利用します。```$ sgsg.py fetchmkdir security_groupsGROUP: groupsave to security_groups/group.csv```ダウンロードされたcsvファイルにはコメントなどを追加することができます。エクセルなどで編集してください(文字コードはutf-8を想定しています)。なお、新たなリモート設定が追加された場合も`fetch`で取得できます。更新時もコメントなどを消すことはありません。### 差分の表示ローカルとリモートの差分を表示するには`diff`コマンドを使用します。`L:`とついているのはローカル設定だけにあるもの、`R:`とついているのはリモート設定だけにあるものです。```$ sgsg.py diffGROUP: somegroupL:tcp   22      22      192.168.11.1/32R:tcp   22      22      0.0.0.0/32```### ローカルからリモートへの同期更新を反映するには`commit`コマンドを使用します。csvに追加、または削除を行なったあと、以下のコマンドを実行します。差分が表示されるので確認後yを選択してください。```sgsg.py commit security_groups/somegroup.csvGROUP: somegrouppost this setting?[y/N]+tcp    80      80      192.168.0.1```
Package: sg1
This repo is a stupid, simple, static site generator. It is very
opinionated, but you can easily create static sites with jinja templates
and json files. This basically cuts the framework out of rendering.Getting StartedInstallTo install fromPyPi:pip3 install sg1Setupsg1 start projectnameThis creates 3 folders in a directoryprojectname:templates: These are thejinja2templates that will be
rendered.content: JSON files that specify the key value pairs used to
render the templates.html: Output directory of the rendering. This is where your
beautifully* rendered html files are generated.*Beauty not guaranteed.TemplatesSee thejinja2 documentationContentExample:#index.json{"template":"index.html","title":"I'm a title""p":"I'm a paragraph.""extras":["about/about.json","posts"]...}The only required field istemplatewhich is the relative path from
thetemplatesfolder.The rest is whatever you want, just make sure the variables are in the
template ({{ content.variable }}).Simple URLssg1 urls projectnamecreates aurlsfolder inprojectnamewith aurls.jsonfile containing all the project urls for ease of
use in the templates.Example:Let’s say you have a content fileposts/post.json. The above command
will add relative reference in theurls.jsonfile like:{...,"posts__post":"/posts/post.html",...}You can then create a link in the template with<ahref="{{urls.posts__post}}">Post</a>.HTMLTo generate the HTML files, runsg1 render projectname.To Do☒ Addextrasfield to context files that automagically includes
context of a specified subfolder (to easily use context fromexample_post.jsonto render the image and title inindex.htmlfor example).☐ Pagination of extra☐ Refactor and general cleanup☐ Update paths for Windows
Package: sg2
Solar Geometry 2 (SG2) is the second generation of library for computing the relative position of the sun and the earth. Valid over the time period 1980-2100, the algorithm is 20 times faster than the well-know SPA algorithm, with an accuracy order of approx. 0.005°. Reference article: Blanc P. and L. Wald,  The SG2 algorithm for a fast and accurate computation of the position of the sun for multi-decadal time period. Solar Energy 88, 3072-3083, 2012, doi: 10.1016/j.solener.2012.07.018.
Package: sgap
sgapShawn Presser's Python utilitiesWARNING: This repo is in development. It was automatically generated withmkpylib. If you're reading this message, it means that I use this repo for my own purposes right now. It might not do anything at all; the default functionality isprint('TODO').If you really want to try it out, feel free. I recommend reading through the code and commit history to see if it does what you need, orask mefor status updates.Stay tuned!Installpython3 -m pip install -U sgapUsageimportsgapprint('TODO')LicenseMITContactA library byShawn Presser. If you found it useful, please considerjoining my patreon!My Twitter DMs are always open; you shouldsend me one! It's the best way to reach me, and I'm always happy to hear from you.Twitter:@theshawwnPatreon:https://www.patreon.com/shawwnHN:sillysaurusxWebsite:shawwn.com
Package: sgb-ftso-contracts
Songbird FTSO Contract FactoryA small library to quickly instantiate Flare Time Series Oracle (FTSO) contracts on the Songbird network.Installationpip install sgb-ftso-contractsGet StartedHow to get prices of crypto assets tracked by the Songbird network:fromsgb_ftso_contractsimportFtsofromweb3importWeb3# Songbird network RPC endpoint# This is a free, rate-limited API node.rpcurl="https://songbird-api.flare.network/ext/bc/C/rpc"# Init web3 with REST HTTP provider.web3=Web3(Web3.HTTPProvider(rpcurl))# Create an FTSO contract instance with factory library.btcFtso=Ftso("BTC").contract(web3)# Fetch the latest price for Bitcoin from the FTSO.btcDecimals=btcFtso.functions.ASSET_PRICE_USD_DECIMALS().call()btcPriceData=btcFtso.functions.getCurrentPrice().call()# Prices are recorded as integers. Convert to decimal format.print(btcPriceData[0]/pow(10,btcDecimals))MoreDocumentationDocumentation can be foundhere.Source codeThe source code for the FTSO smart contract system, for the Songbird network can be foundhere.White paperMore information about the Flare Network, concepts, etc. can be foundhere.
Package: sgci-resource
Floatview StatsLatest ReleasePyPI Downloads#Introduction to SCGI InventoryVersionThis is version 1.0.0 of the SGCI Resource Description Specification schema. This work is released under an Apache 2.0license <https://raw.githubusercontent.com/SGCI/sgci-resource-inventory/master/LICENSE>_ .IntroductionThe user-facing components of the Cyberinfrastructure (CI) ecosystem, science gateways and scientific workflow systems,
share a common need of interfacing with physical resources (storage systems and execution environments) to manage data and execute codes (applications).However, there is no uniform, platform-independent way to describe either the resources or the applications. To address this, we propose uniform semantics for describing resources and applications that will be relevant to a diverse set of stakeholders.The SGCI Resource Description Specification provides a standard way for institutions and service providers to describe storage and computing infrastructure broadly available to the research computing and science gateway community. SGCI Resource descriptions provide a foundation for interoperability across gateway components and other cyberinfrastructure software.The current, initial version of the resource description language focuses on “traditional” HPC and high-throughput storage and computing resourcesInstallationpipinstallsgci-resourceUsagefromsgci_resourceimportcoregh=core.SCGICatalogXsede()#gh = core.SCGICatalogLocal(folder="REPO_PATH")#gh = core.SCGICatalogGithub(token="GITHUB_TOKEN")#print available resourcesl=gh.listResources()print(l)# Print json descriptionresource=gh.getResource('RESOURCE.IDENTIFICATION.KEY')# e.g. stampede2.tacc.xsedeprint(resource)# Print json descriptionnamehost=gh.searchPath("[name, host]",resource)print(namehost)#IntegrationThe SCGI Inventory is currently been integrated with Airavata, HUBzero |reg|  , and Tapis. We expect the inventory to be adapted by others soon.Links:https://github.com/SGCI/sgci-resource-inventoryhttps://github.com/SGCI/sgci-resource-inventory-cache-serviceGet Involved!Issues, Comments, PRs all welcome!SGCI:help@sciencegateways.orgEmail: jstubbs at tacc.utexas.edu, smarru at iu.edu, dmejiapa at purdue.edu
Package: sgcli
What is it?SGCLI is a command line interface for SeatGeek - the most popular
destination for finding tickets to live events.Why?Making the SeatGeek experience available to everyone is key to our
mission. By catering specifically to users without graphics-capable
computers, we expose a whole new segment of people to the wonders of
finding tickets with SeatGeek.Are you serious?I try not to be.How do I get it?Trypip install sgclioreasy_install sgcli, and then run the commandsgcli.Alternatively, just download thesgcliscript from this repo and run it
directly.What are the dependencies?SGCLI depends onrequestsfor doing internet-ey things andPILfor
rendering images.Did you do a double-take when I said rendering images?How do I use it?Like any good command line app, SGCLI is full of obscure keyboard
shortcuts that aren’t really documented. Here are some of them that
work on most pages:h: Go to the home screen
s: Start a new search
q: Quit (with confirmation)
ESC: Quit (without confirmation)
BKSP: Go back to the previous pageOn the search page, enter text and hit ENTER to run the search. Hit
ESC to go home.On pages with data tables, you can navigate using emacs/vim style
navigation or the arrow keys:^p / j / UP ARROW: Go up a row
^n / k / DOWN ARROW: Go down a row
^b / h / LEFT ARROW: Go to the previous page
^f / l / RIGHT ARROW: Go to the next pageWhen you’ve found a result you like, hit the ENTER key to advance.Use the number keys to select a specific quantity of tickets you’re
searching for (‘0’ for all listings).When you get to a listing page, you can hit ‘b’ or ‘w’ to buy the
ticket(s). ‘b’ will attempt to open the market page in a text-mode
browser, while ‘w’ will use the default browser.Are there easter eggs?Of course.Can I contribute?Pull requests are welcome.
Package: sgdhfdhfjjfk
A sample project that exists as an aid to thePython Packaging User Guide’sTutorial on Packaging and Distributing
Projects.This projects does not aim to cover best practices for Python project
development as a whole. For example, it does not provide guidance or tool
recommendations for version control, documentation, or testing.This is the README file for the project.The file should use UTF-8 encoding and be written usingreStructuredText. It
will be used to generate the project webpage on PyPI and will be displayed as
the project homepage on common code-hosting services, and should be written for
that purpose.Typical contents for this file would include an overview of the project, basic
usage examples, etc. Generally, including the project changelog in here is not
a good idea, although a simple “What’s New” section for the most recent version
may be appropriate.