Saravanan666/pypi-org huggingface dataset:
Package: shardedstore
shardedstoreProvides a sharded Zarr store.FeaturesFor large Zarr stores, avoid an excessive number of objects or extremely large objects, which bypasses filesystem inode usage and object store limitations.Performance-sensitive implementation.Use existing Zarr v2 stores.Mix and match shard store types.Serialize and deserialize the ShardedStore in JSON.Shard groups or array chunks.Easily run transformations on store shards.InstallationpipinstallshardedstoreExamplefromshardedstoreimportShardedStore,array_shard_directory_store,to_zip_store_with_prefixfromzarr.storageimportDirectoryStore# xarray example, but works with zarr in generalimportxarrayasxrfromdatatreeimportDataTree,open_datatreeimportjsonimportnumpyasnpimportosCreate component shard storesbase_store=DirectoryStore("base.zarr")shard1=DirectoryStore("shard1.zarr")shard2=DirectoryStore("shard2.zarr")array_shards1=array_shard_directory_store("array_shards1")array_shards2=array_shard_directory_store("array_shards2")Generate data for the example# xarray-datatree Quick Overviewdata=xr.DataArray(np.random.randn(2,3),dims=("x","y"),coords={"x":[10,20]})# Sharded array dimensions must have a chunk shape of 1.data=data.chunk([1,2])ds=xr.Dataset(dict(foo=data,bar=("x",[1,2]),baz=np.pi))ds2=ds.interp(coords={"x":[10,12,14,16,18,20]})ds2=ds2.chunk({'x':1,'y':2})ds3=xr.Dataset(dict(people=["alice","bob"],heights=("people",[1.57,1.82])),coords={"species":"human"},)dt=DataTree.from_dict({"simulation/coarse":ds,"simulation/fine":ds2,"/":ds3})A monolithic storesingle_store=DirectoryStore("single.zarr")dt.to_zarr(single_store)A sharded store demonstrating sharding on groups and arrays.Arrays are sharded over 1 dimension.sharded_store=ShardedStore(base_store,{'people':shard1,'species':shard2},{'simulation/coarse/foo':(1,array_shards1),'simulation/fine/foo':(1,array_shards2)})dt.to_zarr(sharded_store)Serialize / deserializeconfig=sharded_store.get_config()config_str=json.dumps(config)config=json.loads(config_str)sharded_store=ShardedStore.from_config(config)Validatefrom_single=open_datatree(single_store,engine='zarr').compute()from_sharded=open_datatree(sharded_store,engine='zarr').compute()assertfrom_single.identical(from_sharded)Run transformations over component shards withmap_shardsto_zip_stores=to_zip_store_with_prefix("zip_stores")zip_sharded_stores=sharded_store.map_shards(to_zip_stores)DevelopmentContributions are welcome and appreciated.git clone https://github.com/thewtex/shardedstore
cd shardedstore
pip install -e ".[test]"
pytest
Package: shardgather
===============================shardgather===============================.. image:: https://badge.fury.io/py/shardgather.png:target: http://badge.fury.io/py/shardgather.. image:: https://travis-ci.org/kevinjqiu/shardgather.png?branch=master:target: https://travis-ci.org/kevinjqiu/shardgather.. image:: https://pypip.in/d/shardgather/badge.png:target: https://crate.io/packages/shardgather?version=latest* Free software: BSD license* Documentation: http://shardgather.rtfd.org.Features--------* Executing SQL query against sharded databases (databases of of the same schema set)* Render the result in different formats:* table* csv* human* Interactive mode allows you manipulate the gathered data in a Python shellInstallation------------See :doc:`installation.rst`Usage-----See :doc:`usage.rst`History-------1.0.0 (2014-09-01)++++++++++++++++++* First release on PyPI.
Package: shareable
shareableDynamic python object access & manipulation across threads/processesInstallation:
requires python3.8+pip install shareableExample:# make a test class:classTest:def__init__(self,name,age):self.name=nameself.age=agedefget_details(self):returnself.name,self.age# in terminal 1>>>fromshareableimportShareable>>>test=Test('DB Cooper',50)>>>s=Shareable(test)>>>s'Shareable(DB Cooper, 50)'# in terminal 2:>>>fromshareableimportShareable>>>s=Shareable()'Connection established'>>>print(s['name'])'DB Cooper'>>>s['name']='new name'# back in terminal 1:'Connection established'>>>print(s['name'])'new name'>>>print(s.methods())'["age", "get_details", "name"]'Support for complex objects:>>>importpandasaspd>>>importnumpyasnp>>>df=pd.DataFrame(np.random.randint(0,100,size=(100,4)),columns=list('ABCD'))>>>s=Sharedable(df)>>>s['info']()<class'pandas.core.frame.DataFrame'>RangeIndex:100entries,0to99Datacolumns(total4columns):#   Column  Non-Null Count  Dtype----------------------------0A100non-nullint641B100non-nullint642C100non-nullint643D100non-nullint64dtypes:int64(4)memoryusage:3.2KB# terminal 2:>>>s=Shareable()'Connection established'>>>s['columns']Index(['A','B','C','D'],dtype='object')Gracefully handles resources on keyboard or explicit exit:>>>s=Shareable()>>>exit()'Destroyed shared resources''Killed all child processes'
Package: sharebear
ShareBearThis project provides a simple api for setting and getting variables and objects from a text file.Free software: BSD licenseDocumentation:https://sharebear.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2019-12-06)First release on PyPI.
Package: shareboard
Shareboard is a local HTTP server which works like clipboard.
After you start the server, you can simply set/get the text data to that
server with HTTP communication. It is useful to connect two different process.Shareboard also give you a builtin HTTP viewer to display the data. You can
use this HTTP viewer to visualize your HTML text.InstallUse [easy_install][] or [pip][] to install. Follow the command below$ easy_install shareboardor$ pip install shareboard[easy_install]:http://packages.python.org/distribute/easy_install.html[pip]:http://pypi.python.org/pypi/pipUsage### ClipboardThe basic mechanisms of Shareboard is described below. Shareboard use HTTP
connection to set/get text data.Sender === POST:http://localhost:8081/==> Shareboard
Getter <== GET :http://localhost:8081/=== ShareboardAs I described, Shareboard use local HTTP server so you must start the server
before use. You can start the server with the command below$ shareboard startShareboard provide you a small script to set/get text data so you can simply
set/get data to/from Shareboard with following commands$ shareboard set “Hello World”
$ shareboard get
Hello World#### Clipboard advanceIf you specify, Shareboard automatically run a text modification program
everytime when you set text data. For example, you can usesedto modify
the text data with the commands below$ shareboard start -c ‘sed “s/Hello/Hi/g”’With this shareboard server, data will be modified with the command like below$ shareboard set “Hello World”
$ shareboard get
Hi World### Builtin HTML viewerShareboard is developed to help a program such as a Markup viewer for vim.
That’s why it has builtin HTML viewer written in Qt.
To enable this feature, you have to install the following libraries[Qt](http://qt-project.org/)[PySide](http://qt-project.org/wiki/PySide)After you install these libraries, simply start the Shareboard server with-voption like$ shareboard start -vThe builtin HTML viewer will automatically be refreshed everytime when you set
new text data.Special thanksShareboard is originally inspired by
[mkdpreview](https://github.com/mattn/mkdpreview-vim).
Package: shared
Pyrustic SharedData exchange and persistenceThis project is part of thePyrustic Open Ecosystem.Installation.Latest.ModulesTable of contentsOverviewExampleAPIInstallationOverviewSharedis a Python package created to be the programmer's companion when it comes to storing unstructured application data, managing configuration files, caching data, and exchanging data with other programs.Under the hood, Shared usesParadictto encodestrings,scalars(integer, float, decimal float, complex, booleans),dateandtime,nullvalue,binarydata and nestedcollections(list and dictionary).Note:This library isn't intended to replace a proper relational database such as SQLite !ExamplefromsharedimportDossier,HOMEfromdatetimeimportdatetimefrompathlibimportPath# load a picturewithopen("/home/alex/image.png","rb")asfile:photo=file.read()# create a user profile dictionary embedding the picturenow=datetime.now()profile={"name":"alex","access_datetime":now,"photo":photo,"pi":3.14,"books":["Seul sur Mars","The Fall"],"is_author":True,"fingerprint":None}# create a dossier (or access an existing one)path=Path(HOME,"my_dossier")dossier=Dossier(path)# save profile dictionary in the dossierdossier.set("my_profile",profile)# retrieve profile dictionaryprofile_bis=dossier.get("my_profile")# let's compare the two profile objects !assertprofile==profile_bisAPIExplore the source code.InstallationSharediscross platform. It is built onUbuntuwithPython 3.8and should work onPython 3.5ornewer.Create and activate a virtual environmentpython-mvenvvenvsourcevenv/bin/activateInstall for the first timepipinstallsharedUpgrade the packagepipinstallshared--upgrade--upgrade-strategyeagerDeactivate the virtual environmentdeactivateAbout the authorHi, I'm Alex, a tech enthusiast ! Get in touch withme!Back to top
Package: shared-atomic-enterprise
This package provided a second way to access those shared data types with atomic operations.
can be used onmacOS x86_64,Microsoft Windows 10 x64, Windows Server 2019 x64Centos7,9/RHEL7,9 x86_64,Ubuntu Linux 18, 20, 22 x86_64 andSuse Linux Enterprise Server 12, 15 x86_84.This is copyright software, you should buy it athttps://sharedatomic.top/enterprise/paymentto get a licence to use it. For the price and details, please checkhttps://sharedatomic.top/enterprise/payment.Included Datatypesatomic_intatomic_uintatomic_floatatomic_boolatomic_bytearrayatomic_stringatomic_shared_memoryatomic_set, package bitarray >= 2.4.0 is needed.atomic_list, package bitarray >= 2.4.0 is needed.LINUX/macOSCPython 3.6 - 3.11, if compiled by the user, for CPython3.6 –enable-shared should be added in the configuration parameterPypy 3.8 - 3.10(numpy should be installed to use the atomic_shared_memory)The package requires libatomic and libgomp installed on the Linux platformsbitarray if you need to use atomic_set or atomic_listurwid==2.1.2 to activate the packageWindowsCPython 3.6 - 3.11Pypy 3.8 - 3.10(numpy should be installed to use the atomic_shared_memory)Microsoft Visual C++ 2019 Redistributable(X64)bitarray if you need to use atomic_set or atomic_listfor CPython, windows-curses==2.3.1 and urwid==2.1.2 to activate the package on windowsInstallationTo install shared_atomic, use pip:The binary distribution of the software is compiled and delivered in different epochs.macOS only x86_64 platform supported, silicon not supportedepoch 1: version 1!3.3.2Redhat or Centos Linux, only centos7,9/rhel7,9 on x86_64 platform supported.epoch 2: version 2!3.3.2Ubuntu Linux 18, 20, 22 on x86_64 platform supportedepoch 3: version 3!3.3.2Suse Linux Enterprise Server 12, 15 on x86_64 platform supportedepoch 4: version 4!3.3.2Microsoft Windows x64, Windows 10 and Windows server 2019 supportedepoch 5: version 5!3.3.2On Linux, you need assistance from the root user,If you are a root user, it is ok, then only root can run this package.
If you are not root user, in the activation process you will need assistance from the root user.cd (the directory where the package is installed )
chown root shared_atomic/atomic_python_suid
chmod u+s shared_atomic/atomic_python_suidthen the user who installed the package can run the package.For documentation, please go to:https://sharedatomic.top/
Package: sharedbuiltinmutables
Shared list / set / dict across processes / environmentspip install sharedbuiltinmutablesTested against Windows 10 / Python 3.11 / AnacondaFILE 1fromsharedbuiltinmutablesimportMemSharedDict,MemSharedList,MemSharedSet,cfg# dill/pickle protocolcfg.protocol=5d=MemSharedDict({3:323},name="d1",size=1024)l=MemSharedList([3,323],name="l1",size=1024)s=MemSharedSet({3,6,5},name="s1",size=1024)d[111]=444d.pop(3)d[9]=lambdah:h*3# to clean up: d.cleanup()FILE 2fromsharedbuiltinmutablesimportMemSharedDict,MemSharedList,MemSharedSet,cfg# dill/pickle protocolcfg.protocol=5d=MemSharedDict(name="d1",size=1024)# passing a value ( d = MemSharedDict({33:11,3:3} name="d1", size=1024) )# won't do anything if# the dict has already been created by another proc## use instead:# d = MemSharedDict(name="d1", size=1024)# d.clear()# d.update({33:11,3:3})l=MemSharedList(name="l1",size=1024)s=MemSharedSet(name="s1",size=1024)# to clean up: d.cleanup()
Package: shareddata
Instalation Instructionscreate virtual environmententer command: pip -m venv venvactivate virtual environnment (venv\Scripts\activate.bat)install requirementsenter command: pip install -r requirements.txtCreate .env file in root folder Ie:AWS_PROFILE=DATABASE_FOLDER=SOURCE_FOLDER=DOWNLOADS_FOLDER=S3_BUCKET=DOWNLOADS_S3BUCKET=USERNAME=COMPUTERNAME=PYTHONHASHSEED=0GIT_USER=GIT_TOKEN=GIT_ACRONYM=
Package: shared-ds
Shared Memory Data StructureThis package allows you to use your data structures like numpy arrays in the
shared memory environment between two or more python processes. This library
simplifies the use of shared memory data structures as you don't need to manually
manage shared memory.SharedArray example:Process #1fromshared_dsimportSharedArray# Create shared memory and put you numpy array into that memory segment.shared_np_array=SharedArray.from_array(np_array)shm_descriptor=shared_np_array.to_json()Process #2fromshared_dsimportSharedArray# Attaches to existing shared memory and reads numpy array representation.shared_np_array=SharedArray.from_json(shm_descriptor)shm_descriptor=shared_np_array.to_json()io.BytesIO example:Process #1fromshared_dsimportSharedBytesIOimportio# Create shared memory and put content of passed BytesIO into that memory segment.data_to_store=io.BytesIO(b'data which we want to store')shared_memory=SharedBytesIO.from_bytes_io(data_to_store)shm_descriptor=shared_memory.to_json()Process #2fromshared_dsimportSharedBytesIO# Attaches to existing shared memory and gets io.BytesIO content.shared_memory=SharedBytesIO.from_json(shm_descriptor)data=shared_memory.get_data()Currently supported data structures:Numpy Arrayio.BytesIO
Package: shared_lock
A python package that provides useful locks.It includes the following functionality:Reader-writer locksMultiple readers (at the same time).Single writers (blocking any readers).Helpfulread_lockedandwrite_lockeddecorators.Inter-process locksSingle writer using file based locking (these automatically
release on process exit, even if__release__or__exit__is never called).Helpfulinterprocess_lockeddecorator.
Package: sharedmap
安装补充上传到pypi了，传的sdist，所以也支持python2。pip3 install sharedmap示例import sharedmap

def t1():
	wmap = sharedmap.rbtree();
	wmap.set("aaa","323");
	wmap.set("zb","1233");
	wmap.set("tes","32z");
	wmap.set("红","32是");
	print(wmap.get("tes"));
	wmap.share("test:3325",force=True);


print("测试写入");
t1();



def t2():
	rmap = sharedmap.sharedmap("test:3325");
	print(rmap.get("红"));

print("测试读取");
t2();工具类rbtree这个类用c++代码模拟了一个简单的字符串字典。该字典可以通过share方法向共享内存写入一个sharedmap。set参数类型说明keystrvalstr返回值bool成功True该方法失败的时候会抛出异常。get参数类型说明keystr返回值strshare把字典当前状态制作成一个快照(基于索引的内存池和avl树)保存到共享内存中，这个快照可以通过sharedmap读取。参数类型说明namestr共享字典的名字，sharedmap在初始化的时候会用到forcebool如果该名字的共享字典已经存在，是否删掉旧的重建。返回值bool成功True该方法失败的时候会抛出异常。dump把字典当前状态制作成一个快照返回成bytes这个二进制数组其实就是sharedmap的内存结构。(目前没提供读取的办法，所以这个方法其实没什么意义)参数类型说明namestr共享字典的名字，sharedmap在初始化的时候会用到返回值bytes成功Trueremove这是一个静态方法，用户删除由share方法构造的共享内存sharedmap，删除操作不要求和写入操作在同一个进程中。删除后，已经获取的sharedmap依然有效，无法通过该名字获得sharedmap对象。删除后可以通过share方法建立新的sharedmap。更多细节和清理内存的时机参考共享内存的原理。参数类型说明namestr共享字典的名字，sharedmap在初始化的时候会用到返回值bool成功Truesharedmap__init__通过给定的名字获得一个共享内存字典，该字典数据必须是由上面的rbtree对象通过share方法写入的。参数类型说明namestrget参数类型说明keystr返回值str实现原理avl树是完美平衡树，在内存中可以用连续的内存块表示，同时avl树的读取操作并不需要在其内存结构中写入任何辅助的信息。avl树的插入操作成本非常高（用给定的有序定长数组构造avl树很快），但读取速度非常快。红黑树插入方便。共享内存中的数据修改需要在多个不同的进程中用锁来控制，是比较困难的，所以本文的方案是禁用了共享字典的持续写入能力。用红黑树初始化一个字典，然后制作成avl树快照供后续读取。一个共享内存字典的生命周期内，只允许一次性的写入操作，因此具有极好的无锁并发读取特性。但缺点也是存在的，想要更新共享字典中的数据，需要用红黑树加载所有数据修改后再全量写入共享内存。而且重新写入后，已经被加载的共享字典变成了那些进程的私有数据脱离了共享内存，占用一块内存不说，还不会更新，只能通过销毁对象重新创建或者重启进程来更新数据。可以说修改成本是非常非常高的。
Package: sharedmock
A multiprocessing-friendly Python mock objectGetting startedTheSharedMockobject has an interface similar to Python’s ownunittest.mock.Mock. The main difference is that the state of aSharedMockobject is shared among subprocesses. This allows you to
easily test interactions of subprocesses with your mock instance.fromsharedmock.mockimportSharedMocksharedmock=SharedMock()subprocess=mp.Process(target=sharedmock,args=('fancyArg',))subprocess.start()subprocess.join()subprocess.terminate()expected_calls=[call('fancyArg')]sharedmock.assert_has_calls(expected_calls,same_order=False)If theSharedMockwere to be replaced by aunittest.mock.Mockin
the example above, the assertion would fail. The interaction with theunittest.mock.Mockobject would not get propagated to your test
code:EAssertionError:Callsnotfound.EExpected:[call('fancyArg')]EActual:[]SourceThe entire source code is available onGitHub.DocumentationDocumentation can be found onRead the Docs.
Package: shared-ndarray
A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX
shared memory.SharedNDArrays are designed to be sent over multiprocessing.Pipe and Queue
without serializing or transmitting the underlying ndarray or buffer. While the
associated file descriptor is closed when the SharedNDArray is garbage
collected, the underlying buffer is not released when the process ends: you
must manually call theunlink()method from the last process to use it.Usagefrom__future__importprint_functionimportmultiprocessingasmpimportnumpyasnpfromshared_ndarrayimportSharedNDArraytry:shm=SharedNDArray((4,4))shm.array[0,0]=1p=mp.Process(target=lambdashm:print(shm.array),args=(shm,))p.start()p.join()finally:shm.unlink()This should print:[[ 1.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]There are also convenience methods to create a new SharedNDArray from an
existing NumPy array:arr=np.array([0,0])shm1=SharedNDArray.copy(arr)shm2=SharedNDArray.zeros_like(arr)shm1.unlink()shm2.unlink()Dependenciesnumpyposix_ipc
Package: shared-nearest-neighbors
Shared Nearest NeighborsA Shared Nearest Neighbors (Jarvis-patrick) clustering implementation. This code is basically a wrapper of sklearn DBSCAN, implementing the neighborhood similarity as a metric.
The implementation follows the syntax of scikit-learn clustering classes.Free software: MIT licenseDocumentation:https://shared-nearest-neighbors.readthedocs.io.History0.1.0 (2022-01-06)First release on PyPI.
Package: shared-python-library
airtech-shared-python-libraryDescriçãoO airtech-shared-python-library é um projeto de biblioteca compartilhada que oferece funcionalidades específicas, com suporte para PostgreSQL por meio de um adaptador dedicado.RequisitosPythonPostgreSQLInstalaçãoCertifique-se de ter o Python e o PostgreSQL instalados no seu ambiente. Em seguida, execute os seguintes comandos:pipinstallairtech-shared-python-libraryConfiguração do Adaptador PostgreSQLPara utilizar o adaptador PostgreSQL, é necessário configurar as informações de conexão. Aqui está um exemplo de como configurar o adaptador no seu código Python:fromairtech_shared_python_library.adaptadoresimportPostgreSQLConnection# Configuração do adaptador PostgreSQLconexao_postgres=PostgreSQLConnection(host='localhost',port=5432,database='seu_banco_de_dados',user='seu_usuario',password='sua_senha')Lembre-se de substituir as informações acima pelos detalhes específicos do seu ambiente.Uso do Adaptador PostgreSQLAgora que o adaptador está configurado, você pode usá-lo para realizar consultas no banco de dados PostgreSQL. Aqui está um exemplo básico:# Exemplo de uso do adaptador PostgreSQLwithconexao_postgresasconexao:cursor=conexao.create_cursor()resultado=conexao.execute_query("SELECT * FROM sua_tabela;")forlinhainresultado.fetchall():print(linha)ContribuiçãoSe você quiser contribuir para este projeto, siga estas etapas:Fork do projetoCrie uma nova branch (git checkout -b feature/nova-feature)Faça commit das suas alterações (git commit -m 'Adicionar nova feature')Faça push para a branch (git push origin feature/nova-feature)Abra um pull request
Package: sharedshell
Shared shell for launching processes in PythonThe purpose of this package is to provide a simple way to launch processes in Python but in the same shell. Only one shell is allocated and all the processes are launched in order. There is no need to allocate a different shell on each execution, and therefore it benefits from faster execution and a smaller memory footprint.Getting startedInstallation via pip:pipinstallsharedshellUsing it:fromsharedshellimportSharedShellwithSharedShell()asshell:print(shell.run('echo 1').stdout)print(shell.run('echo 2').stdout)print(shell.run('echo 3').stdout)Output:123Benchmarking:┌---------------------|os.system
└--------------------
command:Mean+-stddev:9.22sec+-2.33sec
┌---------------------|subprocess
└---------------------
command:Mean+-stddev:7.48sec+-1.92sec
┌---------------------|sharedshell
└---------------------
command:Mean+-stddev:2.11sec+-0.03sec
Package: sharedvault
SharedVaultSharedVault is a small application that allows you to define a secret that will require multiple people to unlock.How does it work ?At the core, SharedVault uses Shamir's Secret Sharing algorythm (some code is directly copied from the wikipedia page). This allows us to define a key(M, N) and N parts where the key can be recovered from M parts.
On top of that we add a layer where each user has a public/private key that encrypts the shares they hold.
This allows the following features:A user change chose and change their password.A user has a single password to remember (instead of one per share).We can grant users access without having to communicate the share over untrusted network.Security considerationPrivate keysPrivate keys are stored in the main database and encrypted with the user's password. A weak/leaked password will compromise the private key and therefore all the shares it can decrypt.Secret encryptionThe secret itself is encrypted using the Fernet algorithm. The key is derived with Scrypt from the secret number generated by Shamir's Secret Sharing algorithm.
Note that I have no idea what I am doing so it is likely that there is a flaw related to the scrypt configuration that could compromise a secret's safety.Revoking accessEverytime a secret is updated we generate a brand new set of keys for the new encryption. This allows us to deny future access to a user that has their key removed from the secret.DeletionSharedVault uses the database you provide, therefore it is impossible for us to protect against deletion since the database itself could be deleted.Past password/keysFor the same reason, we cannot enforce that previous version of the database have been permanently deleted. This means that if an old password or key leaks, and the past database is available, then that key is compromized.How to use it ?I will add more ways to use this tool but right now there is a cli that you can run with:$ pipenv run python -m cli --db=$DB_CONN_STR --helpDepending on the type of database you use, you might need additional dependencies. You can read more about the connection string format, available backends and additional dependencies athttps://docs.sqlalchemy.org/en/13/core/engines.html.