Saravanan666/pypi-org huggingface dataset:
Package: unstable
UnstableBecause sometime documenting a function is unstable is not enough.Because often, reaching to the documentation to know if something is
stable takes too much time.Because once it’s stable, you’ll forget to re-check again.Installation$ pip install unstableUtilityThis module provides a decorator to mark function asunstable.
Unstable function willraiseanUnstableWarningunless used in
anunstablecontext. Thus preventing you from using an unstable
function by mistake.Provide anwith unstable()context manger which silence (or log) the
exceptions raised by unstable functions. By default anunstablecontext manager that does not detect any use ofunstablefunctions
will log aStableWarning.See Readme for more information.UsageMark a function as unstablefromunstableimportunstable@unstabledeffrobulate(frumious,bandersnatch):galumphing(frumious)+bandersnatchMark a branch as being unstablefromunstableimportunstabledefgalumphing(frumious):iffrumious:unstable.here_be_dragons(reason='frumious=True')# ...else:# ...Using unstable codeBy default unstable code will raise anUnstableWarning, wrap
unstable code in anunstablecontext manager.fromunstableimportunstablewithunstable:fumious(True,25)# Will Behavefrumious(False,18)# will Fail with UnstableWarninggalumphing(False)# finewithunstable:galumphing(True)# fineContext manager or decorator ?Q : Is unstable a context manager or decorator ? A : Both. It’s magic.Advance usage:I need to write the doc, PR welcome.Why ?Inspired by Rust’s#[Unstable], which allows you to use it only
explicitly (and I’m not talking about safety here). This mimick it (as
much as it can).I want to be able to “ship” unstable code and N.x version of a library
and tell users “Use at your own risk”, though no-one read the doc. Here
it make things obvious, don’t use it unless youmeantto. Also the
“instability” in contagious as function using unstable functions will
become (partially) unstable. The instability boundary is obvious, it’s
the context manager.What about try/except ?It will/can catch the exception, though the called function will not
run.TodoAdd a plugin for sphinx which automatically mark the function in the doc
as unstable.
Package: unstable-populations
Population stabilityA package to calculate theUnstable Population Indicator(see below for definition) or thePopulation Stability Index, see e.g.this paperorthis paper, with a (slightly flawed, as explained in the accompanying paper) Python implementationhere.Installation instructionsClone the repository, thenpipinstallunstable-populationsThe Unstable Population IndicatorThe UPI is explained in much more detail in an accompanying paper, but the main idea is that it is an indicator that can be used when comparing two populations, to see if they are similar. Dissimilarity can be interpreted as data drift, for example, and may indicate a difference between a population some algorithm was trained on and a new population, for which the algorithm will be used.The UPI works for continuous, ordinal and nominal data (continuous data will be binned) and is formally defined as (LaTeX probably not rendered on Github, see your local version or the accompanying paper):$$
\textrm{UPI} =
\sum_{\textrm{bins, }i} (f_{1,i} - f_{0,i}) \cdot \log \Big( \frac{f_{1,i} + 1/n_\textrm{tot} }{f_{0,i} + 1/n_\textrm{tot} } \Big)$$
where $f_{0,i}$ and $f_{1,i}$ are the fraction of the entities in bin or category $i$ in the original and new population, respectively,  $n_\textrm{tot}$ is the number of entities in the original and new populations together and $N_\textrm{bins}$ is the number of bins or categories (over which the sum is taken).UsageThe UPI and PSI can be calculated for two populations as followsfromunstable_populationsimportupi,psiUPI=upi(pop1,pop2)PSI=psi(pop1,pop2)There are optional keyword parameters that can let you do binning of the populations. The data can be provided in lists, numpy arrays, dictionaries, pandas Series or pandas DataFrames, see the documentation for more details.As a rule of thumb (but test for yourself!), values as low as 0.1 for UPI or PSI indicate a similar population, and the larger the values, the more dissimilar the populations are.MiscDeveloped and written by Marcel Haas, Joris Huese and Lisette Sibbald at the Data Science Center of the University of Amsterdam.
Hosted athttps://github.com/harcel/unstable_populations/
Package: unstac
unstacCLI tool for making GeoPackages from staticSTACcatalogs.Installationpip install unstacorgit clone https://gitlab.com/multione/unstac.git
pip install .Usage$ unstac --help
                                                            
 Usage: unstac [OPTIONS] URL                                
                                                            
 Crawl through a static SpatioTemporal Asset Catalog from   
 URL and dump assets into GeoPackage                        
 --out-file defaults to a file named after the catalog      
 Title, in the working directory                            
 --workers defaults to number of available CPU threads      
                                                            
╭─ Arguments ──────────────────────────────────────────────╮
│ *    url      TEXT  [default: None] [required]           │
╰──────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────╮
│ --out-file        TEXT     [default: None]               │
│ --workers         INTEGER  [default: 16]                 │
│ --help                     Show this message and exit.   │
╰──────────────────────────────────────────────────────────╯For example:$ unstac https://s3.eu-central-1.wasabisys.com/stac/odse/catalog.json
parsing Open Environmental Data Cube EuropeLicenseThis project is licensed under the terms of theMIT license.
Package: unstar-pipfile
unstar-pipfileIf you have stars in your Pipfile, this project is for you!unstar-pipfileis a tool to scanPipfile.lockand replace any stars inPipfilewith precise versions from the lock file.Installationpip install unstar-pipfileUsagepython -m unstar_pipfileRun as a python module in the directory withPipfileandPipfile.lock.Helppython -m unstar_pipfile --help
Package: unstdio
github|PyPi|ReadTheDocsunstdiomanipulate stdiofromunstdio.unstdioimportUnstdiofromunstdio.handlersimportOutfromunstdio.formatsimportDictFormatterimportiowithUnstdio(stdout=Out(frmt=DictFormatter()))asunstdio:print("{Hello World!}")# out to Unstdio instanceprint(isinstance(unstdio.stdout,io.StringIO))print(unstdio.stdout.getvalue())# out to system stdout
Package: unstract-adapters
Unstract AdaptersThis is Unstract's python package which helps to configure to a number of different LLMs, Embeddings and VectorDBs.LLMsThe following LLMs are supported:LLMVersionOpenAI1.3.9Azure OpenAI1.3.9Anthropic0.7.8PaLM0.3.1Replicate0.22.0AnyScale0.5.165Mistral0.0.8EmbeddingsThe following Embeddings are supported:EmbeddingVersionOpenAI1.3.9Azure OpenAI1.3.9Qdrant FastEmbed0.1.3HuggingFace0.0.1PaLM0.3.1VectorDBsThe following VectorDBs are supported:Vector DBVersionMilvus2.3.4Pinecone2.2.4Postgres0.2.4Qdrant1.7.0Supabase2.2.1Weaviate3.25.3InstallationLocal DevelopmentTo get started with local development,Create and source a virtual environment if you haven't already followingthese steps.If you're using Mac, install the below library needed for PyMSSQLbrew install pkg-config freetdsInstall the required dependencies withpdminstall
Package: unstract-connectors
Unstract ConnectorsThis is Unstract's python package which helps connect to a number of different filesystems and databases.FilesystemsFilesystems are supported with the help offsspeclibraries that provide a uniform interface to these connectors.The following filesystems are supportedGoogle DriveS3/MinioUnstract Cloud StorageBoxDropbox (issues exist around file discovery/listing)HTTP(S)DatabasesThe following databases are supportedSnowflakePostgreSQLMySQLMSSQLRedshiftMariaDBBigQueryInstallationLocal DevelopmentTo get started with local development,Create and source a virtual environment if you haven't already followingthese steps.If you're using Mac, install the below library needed for PyMSSQLbrew install pkg-config freetdsInstall the required dependencies withpdminstallEnvironment variablesIf theGCSHelperis used, the following environment variables need to be setGOOGLE_SERVICE_ACCOUNT : The service account JSON to perform authentication with Google Cloud Storage account.GOOGLE_PROJECT_ID : The project ID associated with the Google Cloud Storage account.Running testsTODO: Use a test framework and document way to run tests
Package: unstract-dropboxdrivefs
DescriptionThis package is one of the implementation available in the fsspec module.It can be used alone, or dask and intake by indicating the  protocol in the path file:dropbox://file_pathSmall nicety : file_path in dropbox needs to start at the root "/folder1/folder2/etc." which means your path when using the
protocol identifier should look like this :dropbox:///folder1/folder2/etcYes, with three /// ! What happen if not, for some reasons the dropbox api will remove everything before the first /
in the path keep only what is after.It can also be used directly from the fsspec module to download and upload files in the github account.The upload part is using the dropbox API.The download part is using the dropbox API to create a temporary link and then used the already existing webhdfs implementation.
Caching (see fsspec module) is available in that case.Documentation of the fsspec module :https://github.com/intake/filesystem_spec/blob/master/docs/source/features.rstInstallpip install dropboxdrivefsHow to obtain your access token?https://blogs.dropbox.com/developers/2014/05/generate-an-access-token-for-your-own-account/ThanksThanks to martindurant and TomAugspurger for the help to developping this implementation
see info about the development:https://github.com/intake/filesystem_spec/pull/207
Package: unstract-sdk
Unstract SDKTheunstract-sdkpackage helps with developing tools that are meant to be run on the Unstract platform. This includes
modules to help with tool development and execution, caching, making calls to LLMs / vectorDBs / embeddings .etc.
They also contain helper methods/classes to aid with other tasks such as indexing and auditing the LLM calls.InstallationThe below libraries need to be installed to run the SDKLinuxsudo apt install build-essential pkg-config libmagic-dev tesseract-ocr pandocMacbrew install pkg-config libmagic pandoc tesseract-ocrToolsCreate a scaffolding for a new toolExampleunstract-tool-gen--commandNEW--tool-name<nameoftool>\--location~/path_to_repository/unstract-tools/--overwritefalseSupported commands:NEW- Create a new toolEnvironment variables required for all ToolsVariableDescriptionPLATFORM_SERVICE_HOSTThe host in which the platform service is runningPLATFORM_SERVICE_PORTThe port in which the service is listeningPLATFORM_SERVICE_API_KEYThe API key for the platformTOOL_DATA_DIRThe directory in the filesystem which has contents for tool executionLlama Index supportUnstract SDK 0.3.2 uses the following version of Llama
Index Version0.9.28as on January 14th, 2024Environment variables required for various LLMs (deprecated)Azure OpenAIOPENAI_API_KEYOPENAI_API_BASEOPENAI_API_VERSIONOPENAI_API_ENGINEOPENAI_API_MODELDocumentation generationFollowthis README.mdfor generating documentation.
Package: unstructured-client
Python SDK for the Unstructured APIThis is a Python client for theUnstructured API.SDK Installationpipinstallunstructured-clientUsageOnly thefilesparameter is required. See thegeneral partitionpage for all available parameters.fromunstructured_clientimportUnstructuredClientfromunstructured_client.modelsimportsharedfromunstructured_client.models.errorsimportSDKErrors=UnstructuredClient(api_key_auth="YOUR_API_KEY")filename="sample-docs/layout-parser-paper-fast.pdf"withopen(filename,"rb")asf:# Note that this currently only supports a single filefiles=shared.Files(content=f.read(),file_name=filename,)req=shared.PartitionParameters(files=files,# Other partition paramsstrategy='ocr_only',languages=["eng"],)try:resp=s.general.partition(req)print(resp.elements[0])exceptSDKErrorase:print(e)# {# 'type': 'UncategorizedText',# 'element_id': 'fc550084fda1e008e07a0356894f5816',# 'metadata': {#   'filename': 'layout-parser-paper-fast.pdf',#   'filetype': 'application/pdf',#   'languages': ['eng'],#   'page_number': 1#   }# }Change the base URLIf you are self hosting the API, or developing locally, you can change the server URL when setting up the client.# Using a local servers=unstructured_client.UnstructuredClient(server_url="http://localhost:8000",api_key_auth=api_key,)# Using your own servers=unstructured_client.UnstructuredClient(server_url="https://your-server",api_key_auth=api_key,)Custom HTTP ClientThe Python SDK makes API calls using therequestsHTTP library.  In order to provide a convenient way to configure timeouts, cookies, proxies, custom headers, and other low-level configuration, you can initialize the SDK client with a customrequests.Sessionobject.For example, you could specify a header for every request that this sdk makes as follows:importunstructured_clientimportrequestshttp_client=requests.Session()http_client.headers.update({'x-custom-header':'someValue'})s=unstructured_client.UnstructuredClient(client:http_client)MaturityThis SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning usage
to a specific package version. This way, you can install the same version each time without breaking changes unless you are intentionally
looking for the latest version.ContributionsWhile we value open-source contributions to this SDK, this library is generated programmatically.
Feel free to open a PR or a Github issue as a proof of concept and we'll do our best to include it in a future release!SDK Created bySpeakeasy
Package: unstructured-fileconverter-haystack
unstructured-fileconverter-haystackTable of Contentsunstructured-fileconverter-haystackInstallationLicenseTestingInstallationpip install unstructured-fileconverter-haystackLicenseunstructured-fileconverter-haystackis distributed under the terms of theApache-2.0license.TestingTo run tests, first start a Docker container running the Unstructured API:docker run -p 8000:8000 -d --rm --name unstructured-api quay.io/unstructured-io/unstructured-api:latest --port 8000 --host 0.0.0.0Then run tests:hatch run test
Package: unstructured-haystack
Unstructured HaystackUnstructured Connectors for HaystackThis is an example Haystack 2.0 integration. It's an integration for Unstructured.io connectors. Please contribute 🚀The current version has 2 available Unstructured connectors:Discord:UnstructuredDiscordConnectorGitHub:UnstructuredGitHubConnectorGoogle Drive:UnstructuredGoogleDriveConnectorHow to use in a Haystack 2.0 PipelineFor example, you can write documents fetched from Discord using theUnstructuredDiscordConnector:fromhaystack.previewimportPipelinefromhaystack.preview.components.writersimportDocumentWriterfromunstructured_haystackimportUnstructuredDiscordConnectorfromchroma_haystackimportChromaDocumentStore# Chroma is used in-memory so we use the same instances in the two pipelines belowdocument_store=ChromaDocumentStore()connector=UnstructuredDiscordConnector(api_key="UNSTRUCTURED_API_KEY",discord_token="DISCORD_TOKEN")indexing=Pipeline()indexing.add_component("connector",connector)indexing.add_component("writer",DocumentWriter(document_store))indexing.connect("connector.documents","writer.documents")indexing.run({"connector":{"channels":"993539071815200889","period":3,"output_dir":"discord-example"}})
Package: unsupervised-on-policy
Unsupervised On-Policy Reinforcement LearningThis work combinesActive Pre-Trainingwith an On-Policy algorithm,Phasic Policy Gradient.Active Pre-TrainingIs used to pre-train a model free algorithm before defining a downstream task. It calculates the reward based on an estimatie of the particle based entropy of states. This reduces the training time if you want to define various tasks - i.e. robots for a warehouse.Phasic Policy GradientImproved Version of Proximal Policy Optimization, which uses auxiliary epochs to train shared representations between the policy and a value network.
Package: unsupported-python
unsupported-pythonThis package can be used as a conditional dependency to indicate lack of
support for particular versions of Python.[project]name="my-test-package"version="1.0.0"dependencies=[# package does not work on Python 3.11 and beyond"unsupported-python; python_version>='3.11'",]Installing a package which depends onunsupported-pythonwill fail with an
error message, without breaking the installation further by going back to
earlier versions of said package.To allow installation of theunsupported-pythondependency, and hence any
package that conditionally depends on it, set theALLOW_UNSUPPORTED_PYTHONenvironment variable to any value.
Package: unswamp
{un}swampDescription{un}swamp is a python library to create Data Quality Checks that can be run against a pandas dataframe.Quick ExamplesIn the following quick example we show the basic concept of:defining a Check Suite (that holds all the checks to run against a dataset)defining a Check and add it to a Check Suiterun the Check Suite against a datasetevaluate the Check ResultDatasetFor this Quick Example we use an open dataset from the City of New York that contains the NYS Math Test Results by Grade - Citywide by Race-Ethnicity for the years 2006 - 2011. Further details about the dataset can be found here:https://data.cityofnewyork.us/api/views/825b-niea/. In the following section we'll see a code example that does the following steps:collect the data as pandas dataframecreate a Check Suiteadd two different Checks to that suite (1 shall pass / 1 shall fail)run the Check Suite against the collected datasetevaluate the Check Result to hopefully see a pass rate of 50%Codeimportpandasaspdfromunswamp.objects.checksimportCheckColumnsExists,CheckColumnValuesInSetfromunswamp.objects.coreimportCheckRun,CheckSuite# We load the dataset into a pandas dataframedata_file="https://data.cityofnewyork.us/api/views/825b-niea/rows.csv?accessType=DOWNLOAD"dataset=pd.read_csv(data_file)# We generate a CheckSuite to add our checks tometa_data={"owner":"me"}suite=CheckSuite("NY-Math-Grades-CheckSuite","NY-Math-Grades",meta_data)# We generate a test that checks for columns in the dataset# The columns are available so the check will be successfulcolumns=["Grade","Year","Category"]check=CheckColumnsExists("CHK-001-ColsExists",columns,active=True,meta_data=meta_data)suite.add_check(check)# We generate a test that checks if all distinct values in column Year are in the provided values# The year 2011 is missing so the check will failcolumn="Year"values=[2006,2007,2008,2009,2010]check=CheckColumnValuesInSet("CHK-002-ColsValuesInSet",column,values,active=True,meta_data=meta_data)suite.add_check(check)# We run the suite against the dataset and print the pass rate# The pass rate is expected to be 50% since 1 test is successful and one failscheck_run=suite.run(dataset,"manual-test-run")print(f"passed -{check_run.pass_rate*100}%")Credits
Package: untable
UntableA lightweight Python module to scrape HTML tables.>>> import untable
>>> import requests
>>> from pprint import pprint
>>> html = requests.get("https://www.w3schools.com/html/html_tables.asp").text
>>> data = untable.multi(html)
>>> pprint(data)
[{'Company': 'Alfreds Futterkiste',
  'Contact': 'Maria Anders',
  'Country': 'Germany'},
 {'Company': 'Centro comercial Moctezuma',
  'Contact': 'Francisco Chang',
  'Country': 'Mexico'},
 {'Company': 'Ernst Handel', 'Contact': 'Roland Mendel', 'Country': 'Austria'},
 {'Company': 'Island Trading', 'Contact': 'Helen Bennett', 'Country': 'UK'},
 {'Company': 'Laughing Bacchus Winecellars',
  'Contact': 'Yoshi Tannamuri',
  'Country': 'Canada'},
 {'Company': 'Magazzini Alimentari Riuniti',
  'Contact': 'Giovanni Rovelli',
  'Country': 'Italy'}]Installationpip install untableUsageUntable can extract data from two types of tables: single and multi-entity.Multi-entity tables are like the one in the example above, where a table carries information for multiple items. They have headers on their first row and data on the others. Useuntable.multion them.Single entity tables carry data from only one entity, in a form-like structure. They can have headers and data on the same row. For example:Name:Ada LovelaceDOB:12-10-1815Profession:ProgrammerUseuntable.singleon this type of table. You may need to play with thethresholdparameter to get the extraction right.Readuntable.pyto find more information.ContributePRs welcome!
Package: untangle
untangleDocumentationConverts XML to a Python object.Siblings with similar names are grouped into a list.Children can be accessed withparent.child, attributes withelement['attribute'].You can call theparse()method with a filename, an URL or an XML string.Substitutes-,.and:with_<foobar><foo-bar/></foobar>can be accessed withfoobar.foo_bar,<foo.bar.baz/>can be accessed withfoo_bar_bazand<foo:bar><foo:baz/></foo:bar>can be accessed withfoo_bar.foo_bazWorks with Python 3.7 - 3.10InstallationWith pip:pip install untangleWith conda:conda install -c conda-forge untangleConda feedstock maintained by @htenkanen. Issues and questions about conda-forge packaging / installation can be donehere.Usage(See and runexamples.pyor this blog post:Read XML painlesslyfor more info)importuntangleobj=untangle.parse(resource)resourcecan be:a URLa filenamean XML stringRunning the above code and passing this XML:<?xml version="1.0"?><root><childname="child1"/></root>allows it to be navigated from theuntangled object like this:obj.root.child['name']# u'child1'Changelogsee CHANGELOG.md
Package: untanglepyut
This project is intended to be used byPyut Plugindevelopers to convertPyutXML files to theOgl Modelclasses.  These model classes can then be used by the Pyut UI to display as UML Diagrams.Use as follows:fromuntanglepyut.TypesimportDocumentfromuntanglepyut.TypesimportDocumentTitlefromuntanglepyut.TypesimportUntangledOglClassesfromuntanglepyut.TypesimportUntangledOglLinksfromuntanglepyut.TypesimportUntangledOglNotesfromuntanglepyut.TypesimportUntangledOglTextsfromuntanglepyut.UnTanglerimportUnTanglerfqFileName:str='MultiLinkDocument.xml'untangler:UnTangler=UnTangler()untangler.untangleFile(fqFileName=fqFileName)document:Document=untangler.documents[DocumentTitle('Diagram-1')]oglClasses:UntangledOglClasses=document.oglClassesoglLinks:UntangledOglLinks=document.oglLinksoglNotes:UntangledOglNotes=document.oglNotesoglTexts:UntangledOglTexts=document.oglTextsThe following is the UML diagram for the Pyut UntanglerDeveloper NotesThis project usesbuildlackeyfor day to day development buildsNoteFor all kind of problems, requests, enhancements, bug reports, etc.,
please drop me an e-mail.
Written byHumberto A. Sanchez II(C) 2023I am concerned about GitHub's Copilot projectI urge you to read about theGive up GitHubcampaign fromthe Software Freedom Conservancy.While I do not advocate for all the issues listed there I do not like that
a company like Microsoft may profit from open source projects.I continue to use GitHub because it offers the services I need for free.  But, I continue
to monitor their terms of service.Any use of this project's code by GitHub Copilot, past or present, is done
without my permission.  I do not consent to GitHub's use of this project's
code in Copilot.