Saravanan666/pypi-org huggingface dataset:
Package: npblind
npblindA package to manage BigQuery policy tag and GCP IAM related tasksReference:GCP rolesGCP rolesGCP roles predefinedhttps://github.com/googleapis/python-dlp/tree/main/samples/generated_sampleshttps://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/dlp/snippetshttps://github.com/googleapis/python-bigquery-datapolicieshttps://cloud.google.com/python/docs/reference/datacatalog/latest/google.cloud.datacatalog_v1.services.policy_tag_manager.PolicyTagManagerClientWorkaround:https://cloud.google.com/sdk/gcloud/reference/data-catalog/taxonomiesUntested:gcloud data-catalog taxonomies policy-tags set-iam-policy POLICY_TAG --location=LOCATION --taxonomy=TAXONOMY policy.jsonTested & Works:# one by one: works !!!gcloudbetadata-catalogtaxonomiespolicy-tags\add-iam-policy-bindingprojects/nplearn/locations/asia-southeast1/taxonomies/7979849403702726703/policyTags/6110705995628818296\--location='asia-southeast1'\--taxonomy='7979849403702726703'\--member='user:nplearn.channel@gmail.com'\--role='roles/datacatalog.categoryFineGrainedReader'# using file: not workinggclouddata-catalogtaxonomiespolicy-tags\set-iam-policyprojects/nplearn/locations/asia-southeast1/taxonomies/7979849403702726703/policyTags/6110705995628818296\--location=asia-southeast1\--taxonomy=7979849403702726703policytag.yaml# get policygcloudbetadata-catalogtaxonomiespolicy-tags\get-iam-policyprojects/nplearn/locations/asia-southeast1/taxonomies/7979849403702726703/policyTags/6110705995628818296\--location='asia-southeast1'\--taxonomy='7979849403702726703'# describegcloudbetadata-catalogtaxonomiespolicy-tags\describeprojects/nplearn/locations/asia-southeast1/taxonomies/7979849403702726703/policyTags/6110705995628818296\--location='asia-southeast1'\--taxonomy='7979849403702726703'# listgcloudbetadata-catalogtaxonomiespolicy-tags\list\--location='asia-southeast1'\--taxonomy='7979849403702726703'
Package: npc-analyze-image
NPC Image AnalysisA Python package to assist in performing image analysis on Neural Progenitor Cells (NPCS).  In particular, it was made to answer the specific question of analyzing the radial process of the cells. This application aims to allow you to use a default implementation to get you started, but also allows the user to customize the analysis process.NPC Analysisaims to bridge the gap between those who want to perform image analysis but are less technical and are blocked by complicated and lengthy python code documentation.  This provides a simple GUI for users to perform image analysis, specifically on the radial process of Neural Progenitor Cells. However, this application can likely be used for any analysis that requires the inital use of a mask to limit the analysis to a specific region.NPC Image Analysis utilizes skimage's image processing tools as well as napari for viewing the images in 3D.
Package: npccp
longmao-point-cloud-converter点云文件转换工具类(pcd,pot)安装.. code-block:: Pythonpip install npccp使用.. code-block:: pythonimport npccp

############ 工具类
## 1. pcd格式转换(ascii,binary,binary_compressed)
npccp.pcdtopcd(sourcePcd, targetPcd, targetDataType)
## 2. pcd转pot(只转换x,y,z,intensity)
npccp.pcdtopot(pcd, pot)

############ 自行封装
## 1. pcd读
with PcdReader(pcd) as reader:
    header = reader.getHeader()
    while True:
        point = reader.getPoint()
        if point is None:
            break
        print(point)
## 2. pcd写
header = PcdXyziHeader()
header.data = PcdHeader.DATA_TYPE_ASCII
with PcdWriter(target, header) as writer:
    point = [x,y,z,i]
    writer.write(point) ## 循环写点

## 3. pot写
with PotreeWriter(pot) as writer:
    point = npccp.new_potree_point(x, y, z, intensity, r, g, b)
    writer.write(point) ## 循环写点
Package: npc-engine
npc-engineNPC-Engine is a deep learning and NLP toolkit for designing NPC AI with natural language.FeaturesChat-bot dialogue system.SoTA tools like text semantic similarity and text to speech.Easy, open source deep learning model standard (ONNX with YAML configs).GPU accelerated inference with onnxruntime.Engine agnostic API through ZMQ server viaJSONRPC 2.0.Getting startedThe easiest way to get started is to use NPC Engine through theUnity integrationYou can also use it directly through ZMQ or HTTP. SeeDocumentationfor more details.RoadmapDone:Real-time end-to-end chatbot dialogue systemSemantic similarityReal-time speech to text (experimental)Unity integrationCLI tool for importing models fromHuggingfaceAsynchronous API featuresIn progress:Actions and planningUnreal integrationImporting models from popular TTS librariesEmotion featuresMultiple languages supportMuch moreBuild on WindowsCreate virtualenv and activate it:> python3 -m venv npc-engine-venv
> .\npc-engine-venv\activate.batInstall dependencies> pip install -e .[dev,dml](Optional) Compile, build and install your custom ONNX python runtimeBuild instructions herehttps://onnxruntime.ai/(Optional) Run tests> toxCompile to exe with:> pyinstaller --hidden-import="sklearn.utils._cython_blas" --hidden-import="sklearn.neighbors.typedefs" ^
--hidden-import="sklearn.neighbors.quad_tree" --hidden-import="sklearn.tree._utils" ^
--hidden-import="sklearn.neighbors._typedefs" --hidden-import="sklearn.utils._typedefs" ^
--hidden-import="sklearn.neighbors._partition_nodes" --additional-hooks-dir hooks ^
--exclude-module tkinter --exclude-module matplotlib .\npc_engine\cli.py --onedirDockerIf you wish to host NPC Engine somewhere you can use our the docker image. It's Linux image with TensorRT ONNX Runtime provider.You can build it yourself with:dockerbuild-tnpc-engine.To run the image you must mount the models directory to/app/modelse.g.dockerrun--gpusall-it--mounttype=bind,source=%cd%\tests\resources\models,target=/app/models-p5000:5000npc-engine/inference-engine:latestnpc-enginerun--port5000Where--gpus allwill give access to the GPU,-itwill output logs and let you use the container interactively,--mountwill mount the models directory to the container,-p 5000:5000will expose the port 5000 on the host machine.CommunityWe have aDiscordserver where you can get support, ask questions and show off your creations.If you would like to donate, you can check out ourPatreon.Our PatronsMarrech GamesAuthorseublefar-Python, Neural Nets-githubigorzmitrovich-Python, CI/CD-githubSee also the list ofcontributorswho participated in this project.
Package: npc-engine-import-wizard
NPC Engine Import Wizard CLIPython CLI tool for converting models into npc-engine server format.InstallationYou can install it via pip. Import wizards might have their own extra requirements.e.g.pip install npc-engine-import-wizard[transformers]UsageYou can use the CLI tool to convert models from popular libraries into npc-engine services.To start the wizard, run the following command:npc-engine-import-wizard import --models-path <path-to-models> <model-path-or-id>It will prompt you to select the import wizard for the model.
Each service has its own set of import wizards for each library.It will also omit import wizards that lack their extras installed
(i.e. withnpc-engine-import-wizard[transformers]you will only see
import wizards fortransformerslibrary).1. HfChatbotImportWizard
        ImportWizard for converting HF transformer models into text gen services.
2. HfClassifierImportWizard
        ImportWizard for converting HF classification transformer models.
3. HfSimilarityImportWizard
        ImportWizard for converting HF semantic similarity transformer models.You can also list all available import wizards which will also give you detailed readme for each one.npc-engine-import-wizard list-wizardsSupported libraries🤗 TransformersThere are three import wizards for transformers available for different tasks:1. HfChatbotImportWizard
        ImportWizard for converting HF transformer models into text gen services.
2. HfClassifierImportWizard
        ImportWizard for converting HF classification transformer models.
3. HfSimilarityImportWizard
        ImportWizard for converting HF semantic similarity transformer models.NVIDIA'sFlowtron+Waveglowimport wizard for TTS.ESPNet2import wizard thanks to theespnet_onnxproject.
Package: npc-ephys
npc_ephysTools for accessing and processing raw ephys data, compatible with data in the cloud.Usagecondacreate-nnpc_ephyspython>=3.9
condaactivatenpc_ephys
pipinstallnpc_ephysWindowswavpack-numcodecsis used to read compressed ephys data from S3 (stored in Zarr format). On Windows, that requires C++
build tools to be installed: ifpip install npc_ephysfails you'll likely need to download itfrom here.Python>>>importnpc_ephys# get device timing on sync clock using barcodes:>>>recording_path='s3://aind-ephys-data/ecephys_670248_2023-08-03_12-04-15/ecephys_clipped/Record Node 102/experiment1/recording1'>>>sync_path='s3://aind-ephys-data/ecephys_670248_2023-08-03_12-04-15/behavior/20230803T120415.h5'>>>timing_data=next(npc_ephys.get_ephys_timing_on_sync(sync_path,recording_path))>>>timing_data.device.name,timing_data.sampling_rate,timing_data.start_time('Neuropix-PXI-100.ProbeA-AP',30000.070518634246,20.080209634424037)# get a dataclass that reads SpikeInterface sorted data from the cloud# - from a path:>>>si=npc_ephys.get_spikeinterface_data('s3://codeocean-s3datasetsbucket-1u41qdg42ur9/4797cab2-9ea2-4747-8d15-5ba064837c1c')# - or from a subject ID + date + session-index-on-date (separators are optional):>>>si=npc_ephys.get_spikeinterface_data('670248_2023-08-03_0')>>>siSpikeInterfaceKS25Data(session='670248_2023-08-03_0',root=S3Path('s3://codeocean-s3datasetsbucket-1u41qdg42ur9/4797cab2-9ea2-4747-8d15-5ba064837c1c'))# various bits of data are available for use:>>>si.version'0.97.1'>>>''.join(si.probes)'ABCEF'>>>si.quality_metrics_df('probeA').columnsIndex(['num_spikes','firing_rate','presence_ratio','snr','isi_violations_ratio','isi_violations_count','rp_contamination','rp_violations','sliding_rp_violation','amplitude_cutoff','drift_ptp','drift_std','drift_mad','isolation_distance','l_ratio','d_prime'],dtype='object')>>>si.spike_indexes('probeA')array([491,738,835,...,143124925,143125165,143125201])>>>si.unit_indexes('probeA')array([56,61,161,...,151,72,59])DevelopmentSee instructions inhttps://github.com/AllenInstitute/npc_ephys/CONTRIBUTING.mdand the original template:https://github.com/AllenInstitute/copier-pdm-npc/blob/main/README.md
Package: npc-gzip
Code for Paper: “Low-Resource” Text Classification: A Parameter-Free Classification Method with CompressorsThis paper is accepted to Findings ofACL2023.Getting StartedThis codebase isavailable on pypi.org via:pipinstallnpc-gzipUsageSee theexamplesdirectory for example usage.TestingThis package utilizespoetryto maintain its dependencies andpytestto execute tests. To get started running the tests:poetryshell
poetryinstall
pytestOriginal CodebaseRequireSeerequirements.txt.Install requirements in a clean environment:condacreate-nnpcpython=3.7
condaactivatenpc
pipinstall-rrequirements.txtRunpythonmain_text.pyBy default, this will only use 100 test and training samples per class as a quick demo. They can be changed by--num_test,--num_train.--compressor <gzip, lzma, bz2>
--dataset <AG_NEWS, SogouNews, DBpedia, YahooAnswers, 20News, Ohsumed_single, R8, R52, kinnews, kirnews, swahili, filipino> [Note that for small datasets like kinnews, default 100-shot is too big, need to set --num_test and --num_train.]
--num_train <INT>
--num_test <INT>
--data_dir <DIR> [This needs to be specified for R8, R52 and Ohsumed.]
--all_test [This will use the whole test dataset.]
--all_train
--record [This will record the distance matrix in order to save for the future use. It's helpful when you when to run on the whole dataset.]
--test_idx_start <INT>
--test_idx_end <INT> [These two args help us to run on a certain range of test set. Also helpful for calculating the distance matrix on the whole dataset.]
--para [This will use multiprocessing to accelerate.]
--output_dir <DIR> [The output directory to save information of tested indices or distance matrix.]Calculate Accuracy (Optional)If we want to calculate accuracy from recorded distance file<DISTANCE DIR>, usepythonmain_text.py--record--score--distance_fn<DISTANCEDIR>to calculate accuracy. Otherwise, the accuracy will be calculated automatically using the command in the last section.Use Custom DatasetYou can use your own custom dataset by passingcustomto--dataset; pass the data directory that containstrain.txtandtest.txtto--data_dir; pass the class number to the--class_num.Bothtrain.txtandtest.txtare expected to have the format{label}\t{text}per line.You can change the delimiter according to you dataset by changingdelimiterinload_custom_dataset()indata.py.
Package: np-chaonay
NP-chaonay object collection Python package (np_chaonay)For GitHub viewer/user, please see ./GITHUB-NOTICESPackage/Module Table of Contents (with versioning)np_chaonay (1.3.0_stable)main: Module contains object for general usage (2.5.0_stable)math: Module contains object for mathematics usage (1.1.0_stable)humour (package): Humour-related objects collection package (1.0.3_stable)main: Main module for humour-related objects (1.0.3_stable)Tricks and TipsGenerally, there not need to use this package unless you're going to use
my software that depends on this.However, this package may be useful for you.Feel much stress? Try np_chaonay.humour (For more good experience:
try executingfrom np_chaonay.humour.main import *.Bug reportingsThis is currently recommended way to notice me :Issues page of Github repositoryEnding passageHope anyone enjoy using this package.Start doing something useful for yourself, your family+friends, your nearest-neighborhood, your circles, and your society+world.
Package: npchessimage
Creates a chessboard pattern as an image.pip install npchessimageTested against Windows 10 / Python 3.10 / AnacondaArgs:block_size(int,optional):Thesizeofeachsquareblockonthechessboard.Defaultsto10.width(int,optional):Thetotalwidthofthechessboardimage.Defaultsto1000.height(int,optional):Thetotalheightofthechessboardimage.Defaultsto1000.colors(tupleorlist,optional):TheRGBcolorvaluesforthechessboard.Defaultsto((0,0,0),(255,0,0),(255,255,0)).Returns:numpy.ndarray:ThegeneratedchessboardimageasaNumPyarray.Example:fromnpchessimageimportcreate_chessboardchessboard=create_chessboard(block_size=20,width=800,height=600,colors=((255,255,255),(0,0,0)))print(chessboard.shape)# Output:# (600, 800, 3)img=create_chessboard(block_size=10,width=200,height=300,colors=((0,0,0),(255,0,0),(255,255,0)),)importcv2cv2.imwrite('c:\\testimage.png',img)
Package: npc-io
npc_ioPath, IO and caching tools for MindScope Neuropixels projects, compatible with data in the
cloud.Usagecondacreate-nnpc_iopython>=3.9
condaactivatenpc_io
pipinstallnpc_ioimportnpc_ioDevelopmentSee instructions inhttps://github.com/AllenInstitute/npc_io/CONTRIBUTING.mdand the original template:https://github.com/AllenInstitute/copier-pdm-npc/blob/main/README.md
Package: npcirclepack
Circle PackingImage generated bypython3 example/small.pyThis library provides a tool for circle packing in python, using the front-chain packing algorithm byWanget al.and enclosed by a circle computed using theMatoušek-Sharir-Welzl algorithm.Adapted From D3The circle pack code is adapted fromd3-hierarchy, which comes with the ISC license found in the LICENSE file.Why?I was doing some circle-packing visualizations with in the order of 1 million total leaf nodes. Performing the layout in the front-end each time was not only wasteful, but taking too long. I didn't find a Python equivalent that met my needs, so I made this. The output of x, y, and r ar ein a numpy array so it's easier to output as a binary dump. This library could be faster, but it is enough for my purposes.InstallationInstall throughpip. E.g.python3 -m pip install npcirclepack.Note: if you are getting errors, try installingnumpyfirst, thennpcirclepack.
Package: npc_lims
npc_limsneuropixelscloudlabinformationmanagementsystem
Tools to fetch and update paths, metadata and state for Mindscope Neuropixels sessions, in the cloud.quickstartmake a new Python >=3.9 virtual environment with conda or venv (lighter option, since this package does not require pandas, numpy etc.):python-mvenv.venvactivate the virtual environment:Windows.venv\scripts\activateUnixsource.venv/bin/activate.shinstall the package:python-mpipinstallnpc_limssetup credentialsrequired environment variables:AWS S3AWS_DEFAULT_REGIONAWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYto find and read files on S3must have read access on relevant aind bucketscan be in a standard~/.awslocation, as used by AWS CLI or boto3CodeOcean APICODE_OCEAN_API_TOKENCODE_OCEAN_DOMAINto find processed data in "data assets" via the Codeocean APIgenerated in CodeOcean:right click onAccount(bottom left, person icon)clickUser Secrets- these are secrets than can be made available as environment variables in CodeOcean capsulesgo toAccess Tokensand clickGenerate new token- this is for programatically querying CodeOcean's databasesinToken NameenterCodeocean API (read)and checkreadon capsules and datasetsa token will be generated: click copy (storing it in a password manager, if you use one)head back toUser Secretswhere we'll paste it into a new secret viaAdd secret > API credentials- indescriptionenterCodeocean API (read)- inAPI keyenterCODE_OCEAN_API_KEY- inAPI secretpaste the copied secret from before (should start withcop_...)CODE_OCEAN_DOMAINis the codeocean https address, up to and including.orgenvironment variables can also be specified in a file named.envin the current working directoryexample:https://www.dotenv.org/docs/security/env.htmlbe very careful that this file does not get pushed to public locations, e.g. githubif using git, add it to a.gitignorefile in your project's root directory:.env*now in Python we can find sessions that are available to work with:>>>importnpc_lims;# get a sequence of `SessionInfo` dataclass instances, one per session:>>>tracked_sessions:tuple[npc_lims.SessionInfo,...]=npc_lims.get_session_info()# each `SessionInfo` instance has minimal metadata about its session:>>>tracked_sessions[0]# doctest: +SKIPnpc_lims.SessionInfo(id='626791_2022-08-15',subject=626791,date='2022-08-15',idx=0,project='DRPilotSession',is_ephys=True,is_sync=True,allen_path=PosixUPath('//allen/programs/mindscope/workgroups/dynamicrouting/PilotEphys/Task 2 pilot/DRpilot_626791_20220815'))>>>tracked_sessions[0].is_ephys# doctest: +SKIPFalse# currently, we're only tracking behavior and ephys sessions that use variants of https://github.com/samgale/DynamicRoutingTask/blob/main/TaskControl.py:>>>all(s.date.year>=2022forsintracked_sessions)True"tracked sessions" are discovered via 3 routes:https://github.com/AllenInstitute/npc_lims/blob/main/tracked_sessions.yaml\\allen\programs\mindscope\workgroups\dynamicrouting\DynamicRoutingTask\DynamicRoutingTraining.xlsx\\allen\programs\mindscope\workgroups\dynamicrouting\DynamicRoutingTask\DynamicRoutingTrainingNSB.xlsx
Package: npc-mvr
npc_mvrTools for reading raw video data from MindScope Neuropixels experiments, compatible with data in the cloud.Usagecondacreate-nnpc_mvrpython>=3.9
condaactivatenpc_mvr
pipinstallnpc_mvrPython>>>importnpc_mvr>>>d=npc_mvr.MVRDataset('s3://aind-ephys-data/ecephys_670248_2023-08-03_12-04-15/behavior')# get paths>>>d.video_paths['behavior']S3Path('s3://aind-ephys-data/ecephys_670248_2023-08-03_12-04-15/behavior/Behavior_20230803T120430.mp4')>>>d.info_paths['behavior']S3Path('s3://aind-ephys-data/ecephys_670248_2023-08-03_12-04-15/behavior/Behavior_20230803T120430.json')>>>d.sync_pathS3Path('s3://aind-ephys-data/ecephys_670248_2023-08-03_12-04-15/behavior/20230803T120415.h5')# get data>>>type(d.video_data['behavior'])<class'cv2.VideoCapture'>>>>type(d.info_data['behavior'])<class'dict'>>>>type(d.sync_data)<class'npc_sync.sync.SyncDataset'># get frame times for each camera on sync clock# - nans correspond to frames not recorded on sync# - first nan is metadata frame>>>d.frame_times['behavior']array([nan,14.08409,14.10075,...,5084.4582,5084.47487,5084.49153])DevelopmentSee instructions inhttps://github.com/AllenInstitute/npc_mvr/CONTRIBUTING.mdand the original template:https://github.com/AllenInstitute/copier-pdm-npc/blob/main/README.md
Package: npcnn
DeepCTRDeepCTR is aEasy-to-use,ModularandExtendiblepackage of deep-learning based CTR models along with lots of core components layers which can be used to easily build custom models.You can use any complex model withmodel.fit()，andmodel.predict().Providetf.keras.Modellike interface forquick experiment.exampleProvidetensorflow estimatorinterface forlarge scale dataanddistributed training.exampleIt is compatible with bothtf 1.xandtf 2.x.Let'sGet Started!(Chinese Introduction) andwelcome to join us!Models ListModelPaperConvolutional Click Prediction Model[CIKM 2015]A Convolutional Click Prediction ModelFactorization-supported Neural Network[ECIR 2016]Deep Learning over Multi-field Categorical Data: A Case Study on User Response PredictionProduct-based Neural Network[ICDM 2016]Product-based neural networks for user response predictionWide & Deep[DLRS 2016]Wide & Deep Learning for Recommender SystemsDeepFM[IJCAI 2017]DeepFM: A Factorization-Machine based Neural Network for CTR PredictionPiece-wise Linear Model[arxiv 2017]Learning Piece-wise Linear Models from Large Scale Data for Ad Click PredictionDeep & Cross Network[ADKDD 2017]Deep & Cross Network for Ad Click PredictionsAttentional Factorization Machine[IJCAI 2017]Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention NetworksNeural Factorization Machine[SIGIR 2017]Neural Factorization Machines for Sparse Predictive AnalyticsxDeepFM[KDD 2018]xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender SystemsAutoInt[arxiv 2018]AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural NetworksDeep Interest Network[KDD 2018]Deep Interest Network for Click-Through Rate PredictionDeep Interest Evolution Network[AAAI 2019]Deep Interest Evolution Network for Click-Through Rate PredictionFwFM[WWW 2018]Field-weighted Factorization Machines for Click-Through Rate Prediction in Display AdvertisingONN[arxiv 2019]Operation-aware Neural Networks for User Response PredictionFGCNN[WWW 2019]Feature Generation by Convolutional Neural Network for Click-Through Rate PredictionDeep Session Interest Network[IJCAI 2019]Deep Session Interest Network for Click-Through Rate PredictionFiBiNET[RecSys 2019]FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate PredictionFLEN[arxiv 2019]FLEN: Leveraging Field for Scalable CTR PredictionCitationWeichen Shen. (2018). DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models.https://github.com/shenweichen/deepctr.If you find this code useful in your research, please cite it using the following BibTeX:@misc{shen2018deepctr,author={Weichen Shen},title={DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models},year={2018},publisher={GitHub},journal={GitHub Repository},howpublished={\url{https://github.com/shenweichen/deepctr}},}DisscussionGroup 交流群Please follow our wechat to join group:公众号：浅梦的学习笔记wechat ID:deepctrbot
Package: npconf
npconfis a flexible configuration system whose configuration files are plain python modules. The identifiers
(left-hand side) are declared by the administrator/developer and are hierarchical. A typical configuration file might
be:myapp.subcomponent.value=1myapp.enabled=TrueThe user needn’t define “myapp”. It’s automatically included inglobals().The administrator/developer can disallow user-defined attributes. This is mostly intended to prevent typos in the
configuration files. In other words, ifmyapp.foois not defined before reading the configuration files, no file
will be allowed to setmyapp.foo.The configuration attributes are normally declared by a program before it reads any configuration files. They can be
declared at init time:importnpconfroot=npconf.ConfigValue(name='myapp',data={'enabled':'false'})# and/or updated after init:root.update(data={'thing':None})# to add another level to the hierarchy:subcomponent=npconf.ConfigValue(name='subcomponent',data={'foo':'banana',})root.update({'subcomponent':subcomponent})