Saravanan666/pypi-org huggingface dataset:
Package: cc12703-pychrome
pychromeA Python Package for the Google Chrome Dev Protocol,more documentTable of ContentsInstallationSetup ChromeGetting StartedTab managementDebugExamplesPublishRefInstallationTo install pychrome, simply:$ pip install -U pychromeor from GitHub:$ pip install -U git+https://github.com/fate0/pychrome.gitor from source:$ python setup.py installSetup Chromesimply:$ google-chrome --remote-debugging-port=9222or headless mode (chrome version >= 59):$ google-chrome --headless --disable-gpu --remote-debugging-port=9222or use docker:$ docker pull fate0/headless-chrome
$ docker run -it --rm --cap-add=SYS_ADMIN -p9222:9222 fate0/headless-chromeGetting Startedimportpychrome# create a browser instancebrowser=pychrome.Browser(url="http://127.0.0.1:9222")# create a tabtab=browser.new_tab()# register callback if you wantdefrequest_will_be_sent(**kwargs):print("loading:%s"%kwargs.get('request').get('url'))tab.Network.requestWillBeSent=request_will_be_sent# start the tabtab.start()# call methodtab.Network.enable()# call method with timeouttab.Page.navigate(url="https://github.com/fate0/pychrome",_timeout=5)# wait for loadingtab.wait(5)# stop the tab (stop handle events and stop recv message from chrome)tab.stop()# close tabbrowser.close_tab(tab)or (alternate syntax)importpychromebrowser=pychrome.Browser(url="http://127.0.0.1:9222")tab=browser.new_tab()defrequest_will_be_sent(**kwargs):print("loading:%s"%kwargs.get('request').get('url'))tab.set_listener("Network.requestWillBeSent",request_will_be_sent)tab.start()tab.call_method("Network.enable")tab.call_method("Page.navigate",url="https://github.com/fate0/pychrome",_timeout=5)tab.wait(5)tab.stop()browser.close_tab(tab)more methods or events could be found inChrome DevTools ProtocolDebugset DEBUG env variable:Tab managementrunpychrome -hfor more infoexample:Examplesplease see theexamplesdirectory for more examplesPublishinstall depspip install setuptools wheel twinepacakgepython setup.py sdist bdist_wheeluploadtwine upload dist/*Refchrome-remote-interfaceChrome DevTools Protocol
Package: cc2imgcap
cc2imgcapEasily convert common crawl to image caption set using pyspark.Common crawl has5M wat files. They provide links of the web.
This simple tool allows you to process one warc in about 50s and get image link along with the alt text.It also runs deduplication against url+text in order to save on output space and speed up the process.This makes it possible to do the first step of building a dataset likelaion5Bin 70k cpu core hours. (5*10^6*50/(3600))
That's$2.8kusing aws EC2 (0.04$/core hour)What hardware to pick ?cpu128-dy-c6i-32xlargeinstances are advised. Spark stores the non duplicated first stage in local disk. They should be nvme drive for speed during deduplication. At this first stage, one wat takes about 20MB, so the total (over all workers) space must be more than 20MB times wat count. So for example for the whole CC, that means 100TB. So for example that can fit in 150 instances with 1TB nvme drive each. 150 instances of 128 cores is 19200 cores so the whole processing takes 2h. Less instances with bigger hard drives can work too. It's also a possibility to do the processing in multiple pieces if temporary disk space is an issue by specifying--multipart.Document typeThis tool support extracting several documents from CC:image/text: about 300B after dedupaudio/text: about 3B after dedupThey can be selected with eg--document_type audio.
You may experiment with more document kinds by runningpython example single_warc_example.pyand exploring the resulting output.parquet.Installpip install cc2imgcapPython examplesCheckout these examples:run_on_spark.pyit shows how to bring your own spark sessionIf you have a slurm cluster, refer tohttps://gist.github.com/rom1504/67ada3dedbecc113ae2dbdfd9c642d83to start a spark cluster there.APIThis module exposes a single functioncc2imgcapwhich takes the same arguments as the command line tool:output_paththe output path, should probably start with s3://. The output will be written to this path sufixed by the date (required)wat_index_countthe number of wat index files to read, can be None for all. (default 1)wat_countthe number of wat files to read, can be None for all, will randomly subsample if present. (default 100)masterthe spark master url. (default local)num_coresthe number of cores of each spark executor. (default 128)mem_gbthe memory of each spark executor. (default 256)multipartruns the processing of the specified number of parts, merge at the end (default None)shufflerandomly shuffle the output right before saving (default True)resumethe specific path of the output to resume (default None)spark_buildera function that create a spark session, None will default to the built-in methods (default None)document_typethe kind of document to extract (default image)source_cc_protocolget common crawl from http or s3 (default s3)For developmentEither locally, or ingitpod(doexport PIP_USER=falsethere)Setup a virtualenv:python3 -m venv .env
source .env/bin/activate
pip install -e .to run tests:pip install -r requirements-test.txtthenmake lint
make testYou can usemake blackto reformat the codepython -m pytest -x -s -v tests -k "dummy"to run a specific test
Package: cc42
the following is the usage:1.Paste file to 42qu.cchi@Mars~$ 42qucc < foo.txthttp://42qu.cc/xa47qt4712.Custom urlhi@Mars~$ 42qucc hi < foo.txthttp://42qu.cc/hi3.Save web page to local filehi@Mars~$ 42qucchttp://42qu.cc/xa47qt471>  foo.txt
Package: ccaaws
ccaawsPython objects wrapped aroundboto3botosession.pyBase class inherited by all the others.  Manages connections to AWS via
access keys or profiles.cfnclient.pyCloudFormation clientcwclient.pyCloudWatch clientiamclient.pyAccess to IAM functions - mainly for user access key management.Can raise anAccessKeyErrorexception on failure.from ccaaws.iamclient import IamClient

iam = IamClient("iam.user")
newkey = iam.rotateKeys()paramstore.pySSM ParameterStore client
Package: ccaawscreds
ccaawscredsCommand line commands,awskeyandawspw, to change your AWS credentials.Commandgenpwto generate arbitray passwords using the password generator.These tools will only work with credentials stored in the ~/.aws/config file,
not in the ~/.aws/credentials file.AWS Config format is:[profile profile-name]
aws_access_key_id = AKIA....
aws_secret_access_key = ....These tools use 2 extra keys for each section in the config file:userandpasswordi.e.[profile profile-name]
aws_access_key_id = AKIA....
aws_secret_access_key = ....
user = chris.allison
password = anythIng you W@antBoth of these commands rely on the entries in the config file, if one of the
items is missing they will fail.Note: though the AWS Config file format requires that each profile section hasprofileprepended to the name, there is no need to type that when referencing
it, both these tools, boto3 and the AWS CLI look for just the bare profile name
without the wordprofileprepended (see the examples below).awspwThis command will change the users password in the named profile, storing the
new password back into the config file and also displaying it to the screen
along with the old password.  The password generator can be optionally
configured to return passwords of a certain length.  It also splits the
generated password into blocks of 4 characters seperated by spaces, as AWS
allows spaces in passwords.Example awspw$ awspw -l 32 sadmin-static
awscreds 0.2.3
current: 98b8 8d89 7e82 9bec 2)05 1e14 a8Fb a0e9
new: d96d 21fd A816 2762 a[19 81e3 cc52 2af9
$awskeyThis command rotates the users access key for the named profile.  It will first
delete any inactive keys, then generate a new one, storing it in the config
file. Lastly, it deactivates the old, active key.If you have 2 active aws keys, this command will fail, delete or deactivate one
of them and try again.Example awskey$ awskey sadmin-static
awscreds 0.2.3
New access key id AKIAXxxxxxxxxxxxxxxx created.
$genpwThis command will generate an arbitrary password and display it.  It uses the
password generator with the same limits as above (between 8 and 32 characters,
default being 16).  It can optionally remove all the spaces from the password.The password generator works by selecting 4 random words from a list, hashing
them with the sha256 function, randomly selecting characters from the resulting
string up to the required length. It will then insert a random non-alphabetic
character into a random position in the string, and then switch one of the
alphabetic characters to upper-case.  This should be enough for most password
policies.There is also a list of characters that will not appear in the password to avoid
misinterpretation, they arebBiIlLoOso a0in the output is definitely a
zero not a capitalo, simarlarly, a '1' in the output is definitely a one not
a lower case L.Update: as the password is now run through a hashing function, the only letters in the password will bea-f, so the excluded
letter check has now been removed.Example genpw$ genpw -l 32
ca4b 4dac 74bb d^1c 2b2F 0d2e 8b4d 264c

$ genpw -l 10 -n
149388^34A
Package: ccacb
ytcbWatches the clipboard for youtube urls and downloads them.Will now auto-update youtube-dl on startupinstallfrom pypi:python3 -m pip install ccacb --user --upgradefrom the repo (requires poetry):git clone https://github.com/ccdale/ccacb.git
cd ccacb
poetry install
poetry build
python3 -m pip install dist/ccacb-0.6.6-py3-none-any.whl --user --upgradeconfigedit~/.config/ytcb.yaml---
youtubedl: /home/chris/bin/youtube-dl
incoming: /home/chris/Videosexample$ ytcb
08/05/2020 13:47:55 [INFO ]  ytcb - youtube-dl clipboard queue processor 0.6.3
08/05/2020 13:47:55 [INFO ]  reading /home/chris/.config/ytcb.yaml
08/05/2020 13:47:55 [INFO ]  Using /home/chris/bin/youtube-dl
08/05/2020 13:47:55 [INFO ]  youtube-dl will store files in /home/chris/Videos/kmedia/incoming

<CTRL>-c

08/05/2020 13:48:11 [INFO ]  Interrupted: Will finish off the Q, then exit
08/05/2020 13:48:12 [INFO ]  ytcb has finished
Package: ccache-download-redis
ccache-upload-redis / ccache-download-redis GitHub ActionAn GitHub Action for using ccache-upload-redis / ccache-download-redis
to uploads / downloads the contents of the local ccache cache from a Redis remote storage.Usagename:Test ccache-download-upload-redison:workflow_dispatch:jobs:# Label of the container jobcontainer-job:# Containers must run in Linux based operating systemsruns-on:ubuntu-latest# Service containers to run with `container-job`services:# Label used to access the service containerredis:# Docker Hub imageimage:redis:7.0.10-alpine3.17# Set health checks to wait until redis has startedoptions:>---interactive--hostname redis--add-host=host.docker.internal:host-gateway--health-cmd "redis-cli ping"--health-interval 10s--health-timeout 5s--health-retries 5--restart alwaysports:-6379/tcp# get the rendom port via ${{ job.services.redis.ports['6379'] }}# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservices# https://docs.github.com/en/actions/learn-github-actions/contexts#job-context# Maps port 6379 on service container to the host#- 6379:6379steps:-name:"Setsomeredissettings"run:|docker network lsdocker network ls --format='{{.ID }} {{.Name}}'docker inspect ${{ job.services.redis.id }}docker exec ${{ job.services.redis.id }} /bin/sh -c 'echo "cat /etc/redis/redis.conf ||:" 'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'mkdir -p /etc/redis ||: 'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "save 60 100" >> /etc/redis/redis.conf'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "loglevel verbose" >> /etc/redis/redis.conf'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "# see https://github.community/t/how-do-i-properly-override-a-service-entrypoint/17435/8" >> /etc/redis/redis.conf'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "# https://hub.docker.com/_/redis" >> /etc/redis/redis.conf'docker kill --signal=SIGHUP ${{ job.services.redis-y9g98g58d.id }} ||:-name:Checkout repositoryuses:actions/checkout@v3-upload-test:"upload-test"run:|ccache-upload-redisenv:|REDIS_CONF='localhost:${{ job.services.redis.ports['6379'] }}'CCACHE_DIR='~/.cache/ccache'-download-test:"download-test"run:|ccache-download-redisenv:|REDIS_CONF='localhost:${{ job.services.redis.ports['6379'] }}'CCACHE_DIR='~/.cache/ccache'-name:Connect REDISuses:./with:host:${{ variable.REDIS_SERVER }}username:${{ variable.USERNAME }}password:${{ variable.PASSWORD }}ccachedir:${{ variable.CCACHEDIR }}
Package: ccache-upload-redis
ccache-upload-redis / ccache-download-redis GitHub ActionAn GitHub Action for using ccache-upload-redis / ccache-download-redis
to uploads / downloads the contents of the local ccache cache from a Redis remote storage.Usagename:Test ccache-download-upload-redison:workflow_dispatch:jobs:# Label of the container jobcontainer-job:# Containers must run in Linux based operating systemsruns-on:ubuntu-latest# Service containers to run with `container-job`services:# Label used to access the service containerredis:# Docker Hub imageimage:redis:7.0.10-alpine3.17# Set health checks to wait until redis has startedoptions:>---interactive--hostname redis--add-host=host.docker.internal:host-gateway--health-cmd "redis-cli ping"--health-interval 10s--health-timeout 5s--health-retries 5--restart alwaysports:-6379/tcp# get the rendom port via ${{ job.services.redis.ports['6379'] }}# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservices# https://docs.github.com/en/actions/learn-github-actions/contexts#job-context# Maps port 6379 on service container to the host#- 6379:6379steps:-name:"Setsomeredissettings"run:|docker network lsdocker network ls --format='{{.ID }} {{.Name}}'docker inspect ${{ job.services.redis.id }}docker exec ${{ job.services.redis.id }} /bin/sh -c 'echo "cat /etc/redis/redis.conf ||:" 'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'mkdir -p /etc/redis ||: 'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "save 60 100" >> /etc/redis/redis.conf'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "loglevel verbose" >> /etc/redis/redis.conf'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "# see https://github.community/t/how-do-i-properly-override-a-service-entrypoint/17435/8" >> /etc/redis/redis.conf'docker exec ${{ job.services.redis.id }} /bin/sh -x -c 'echo "# https://hub.docker.com/_/redis" >> /etc/redis/redis.conf'docker kill --signal=SIGHUP ${{ job.services.redis-y9g98g58d.id }} ||:-name:Checkout repositoryuses:actions/checkout@v3-upload-test:"upload-test"run:|ccache-upload-redisenv:|REDIS_CONF='localhost:${{ job.services.redis.ports['6379'] }}'CCACHE_DIR='~/.cache/ccache'-download-test:"download-test"run:|ccache-download-redisenv:|REDIS_CONF='localhost:${{ job.services.redis.ports['6379'] }}'CCACHE_DIR='~/.cache/ccache'-name:Connect REDISuses:./with:host:${{ variable.REDIS_SERVER }}username:${{ variable.USERNAME }}password:${{ variable.PASSWORD }}ccachedir:${{ variable.CCACHEDIR }}
Package: ccaconfig
ccaConfiga config file utility. Will read yaml formatted config files from various
locations in the following order, so that the 'nearer' files override the
further ones.  Finally, it checks the environment for variables and
overrides any set in the config file.The order of files to read is/etc/appname.yaml
/etc/appname/appname.yaml
$HOME/.config/appname.yaml
$HOME/Library/Preferences/appname.yaml
$HOME/.appname.yaml
$(pwd)/appname.yamlAny environment variables of the formAPPNAME_VARIABLENAMEwill be found, chopped at the underscore, lower cased and set into the
final configuration i.e:config[variablename]will exist if there is an
environment variableAPPNAME_VARIABLENAME.Usagefrom ccaconfig.config import ccaConfig

cf = ccaConfig(appname="appname")
config = cf.envOverride()or, to not take environment variables into account:from ccaconfig.config import ccaConfig

cf = ccaConfig(appname="appname")
config = cf.findConfig()Two additional dictionaries can be supplied, the firstdefaultdcan be
used to set a default config, and the 2nd,overridedcan be used for
config variables that you do not want overridden by any config file found
or from the environment.from ccaconfig.config import ccaConfig

defd = {"environment": "dev"}
overd = {"product": "myapp"}
cf = ccaConfig(appname="appname", defaultd=defd, overrided=overd)
config = cf.envOverride()
# config["environment"] == "dev" if it is not overridden by a subsequent
# config file or from an environment variable
#
# config["product"] == "myapp" and will not be overridden, at all
Package: ccad
Aboutccad is the backend Python module for ChemCAD
(https://gitlab.com/croningroup/reactionware/chemcad). It's its API and performs
all the calculations/rendering. ChemCAD is just a GUI for this module, and you
can use ccad without the GUI.Installationccad is now installed automatically as part of ChemCAD. Please refer to the installation instructions on the ChemCAD repository for further information -https://gitlab.com/croningroup/reactionware/chemcad
Package: ccaerrors
ccaerrorsUniversal error handling for python functions.Usageimport sys

from ccaerrors import errorNotify

def somefunction():
    try:
        pass
    except Exception as e:
        errorNotify(sys.exc_info()[2], e)errorNotifyprint the function name and the Exception details to stdout.errorRaiseprint the function name and the Exception details to stdout and also re-raise the ExceptionerrorExitprint the function name and the Exception details to stdout. Exit the script by callingsys.exit().
Package: ccAF
ccAF:  cell cycle ASU-Fred Hutch neural network based scRNA-seq cell cycle classifierThe ability to accurately assign a cell cycle phase based on a transcriptome profile has many potential uses in single cell studies and beyond. We have developed a cell cycle classifier based on a scRNA-seq optimized Neural Network (NN) based machine learning algorithmACTINN. The ACTINN code was adapted from:https://github.com/mafeiyang/ACTINNDependenciesThere are four dependencies that must be met for ccAF to classify cell cycle states:numpy- (install)scipy- (install)scanpy- (install)tensorflow- (install)Python dependency installation commands:NOTE!pip may need to be replaced with pip3 depending upon your setup.pip3installnumpyscipyscanpytensorflowInstallation of ccAF classifierThe ccAF classifier can be installed with the following command:pipinstallccAFAlternatively use the ccAF Docker containerWe facilitate the use of ccAF by providing a Docker Hub containercplaisier/ccAFwhich has all the dependencies and libraries required to run the ccAF classifier. To see how the Docker container is configured please refer to theDockerfile. Pleaseinstall Dockerand then from the command line run:dockerpullcplaisier/ccafThen run the Docker container using the following command (replace  with the directory where you have the scRNA-seq data to be classified):dockerrun-it-v'<path to scRNA-seq profiles directory>:/files'cplaisier/ccafThis will start the Docker container in interactive mode and will leave you at a command prompt. You will then want to change directory to where you have your scRNA-seq or transcriptome profiling data.Running ccAF against your scRNA-seq dataThe first step in using ccAF is to import your scRNA-seq profiling data into scanpy. A scanpy data object is the expected input into the ccAF classifier:importscanpyimportccAF# Load WT U5 hNSC data used to train classifier as a loom fileset1_scanpy=sc.read_loom('data/WT.loom')# Predict cell cycle phase labelspredictedLabels=ccAF.predict_labels(set1_scanpy)More complete example is available astest.pyon the GitHub page.ContactFor issues or comments please contact:Chris PlaisierCitationNeural G0: a quiescent-like state found in neuroepithelial-derived cells and glioma.Samantha A. O'Connor, Heather M. Feldman, Chad M. Toledo, Sonali Arora, Pia Hoellerbauer, Philip Corrin, Lucas Carter, Megan Kufeld, Hamid Bolouri, Ryan Basom, Jeffrey Delrow, Jose L. McFaline-Figueroa, Cole Trapnell, Steven M. Pollard, Anoop Patel, Patrick J. Paddison, Christopher L. Plaisier. bioRxiv 446344; doi:https://doi.org/10.1101/446344
Package: cc-agency
CC-AgencyCC-Agency is part of the Curious Containers project. It connects to a cluster of docker-engines for the distributed execution of reproducible data-driven experiments defined in the RED format.For more information please refer to the Curious Containersdocumentation.AcknowledgementsThe Curious Containers software is developed atCBMI(HTW Berlin - University of Applied Sciences). The work is supported by the German Federal Ministry of Economic Affairs and Energy (ZIM project BeCRF, grant number KF3470401BZ4), the German Federal Ministry of Education and Research (project deep.TEACHING, grant number 01IS17056 and project deep.HEALTH, grant number 13FH770IX6) and HTW Berlin Booster.
Package: ccai-client
Cryptocurrencies.AI API python clientHow to generate api key and keyId?Go to your profile api page:https://app.cryptocurrencies.ai/profile/apiClick on "GENERATE API KEY"How to copy KeyId:Go to your profile accounts page:https://app.cryptocurrencies.ai/profile/accountsClick on copy icon -> your key token will be copied in clipboardReferenceSome methods require 'params' dictparam = {
    "strategyId": "5ded5307240b81f3012372de"
}
status = client.get_order_status(params=param)get_order_statusArgs: ('params' dict)strategyId (str): order idReturns:string: order status [Waiting, Canceled]create_orderArgs:marketType (int): 0 for spot, 1 for futures marketpair (str): currency pair with underscore, e.g. "BTC_USDT"stopLoss (float): stop loss percent, e.g. 10.0stopLossType (str): stop loss type [limit, market]leverage (int): leverage [1 - 125]entryOrder (dict):side (str): [buy, sell]orderType (str): [limit, market]type (int): not using yet, just place 0amount (float): coin amount, e.g. 0.01exitLevels (dict array): one or more exit levels {type (int): 1 - amount and price is in percentage, 0 - in absolute values [0, 1]price (float): percent of profit, e.g. 15, or absolute price (depending of type)amount (int): percent of entry, e.g. 70, or absolute amount (depending of type)orderType (string): order type, e.g. "limit"}Returns:dict: order infocancel_orderArgs: ('params' dict)strategyId (str): order idReturns:string: order stateget_active_smart_ordersArgs:marketType (int): 0 for spot, 1 for futures marketReturns:dict array: array of active smart ordersget_active_smart_orders_idsConvenient wrapper for get_active_smart_orders which returns only smart order idsArgs:marketType (int): 0 for spot, 1 for futures marketReturns:string array: array of active smart orders idsget_balancesArgs:noneReturns:dict array: balances for all assetsexample: [{'assetType': 0, 'locked': 0, 'free': 31.86915117, 'asset': {'symbol': 'USDT', 'priceUSD': 1}}]get_futures_free_balanceConvenient wrapper for get_balances.Args:symbol (str): coin symbol, e.g. USDTReturns:float: coin balance on futures wallet