semvec/pypi-packages huggingface dataset:
atd-agolutil 0.0.3
Python utilities interacting with the ArcGIS Online (AGOL) API.

# atd-utils-agol (agolutil)
Helper methods to work with [ArcGIS for Python API](http://arcgis.com), mixed in with a few manual [ArcGIS REST API](http://google.com) requests.

## Installation
```
pip install atd-agolutil
```

## License

As a work of the City of Austin, this project is in the public domain within the United States.

Additionally, we waive copyright and related rights of the work worldwide through the [CC0 1.0 Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
automatedai 0.5
Automated.ai Python Client Library

# Automated.ai Client Package

Python Client Package to access automated.ai hosted algorithms.  For more information, please see [automated.ai](http://automated.ai)

Building the python package:
```console
python setup.py bdist_wheel
source py27
python setup.py bdist_wheel
python -m twine upload --repository-url https://upload.pypi.org/legacy/ dist/*
```

Install localally:
```console
pip install dist/automatedai-0.1-py3-none-any.whl
```



Ways of using the client api:
```console

endpoint
token = "my_secret_token"
Algo=automatedai.Algorithm(endpoint,token)

# Send None to invoke algorithm 
output = Algo.send(None)
print(output)

# Send a string to your algorithm
output = Algo.send("send a string")
print(output)

# Send a file 
output = Algo.send(bytearray(open("my_image.jpg", "rb").read()))
print(output)

# Send a JSON structure, returns the JSON structure 
output = Algo.send({"MyString":"JSON String"})
print(output)

# Send an dictionary (np arrays are converted to list)
output = Algo.send({"int":1,"float":1.2,"array":[1,2],"string":"Ddd","np":np.random.randint(0,255,(100,100,3),np.uint8),"List":[1,2,3,4]})
print(output)

# Send a list
output = Algo.send(["send", "elements", "in", "array"])
print(output)

# send a file link to download, returns the file bytes downloaded
output = Algo.send_url_to_download("https://file_link
print(output)

```
sitemapbuilder 0.0.7
Simple sitemap builder
Keywords: sitemapbuilder sitemap builder http

A simple sitemap builder
========================

The sitemap builder traverses links from a website and constrains itself to
the given domain name. The final result will be a simple sitemap deduced
from the links visited. The crawler will accept & process only URLs with
http or https schemes.

Installation and usage
======================

To run the following command to install the tool:

.. code-block:: bash

    pip install -U sitemapbuilder

To run the sitemap builder:

.. code-block:: bash

    sitemapbuilder -u 'https://monzo.com' -o test_monzo.dot

Some websites have strong protection and the tool will not work for them:

.. code-block:: bash

    sitemapbuilder -u 'https://bloomberg.com' -o test_bloomberg.dot

Highlights
==========

#. Generate Graphviz `.dot` file showing directed links between pages. One can generate PNG/PDF and other image/document formats.
#. Have `configurable decay` (maximum depth) to avoid abuse.
#. Visit web link within the same hostname by default.
#. Use `5 threads` by default and times out after `10 seconds`.
#. Timeout after `5 seconds` when fetching a URL.
#. Handle timeout exceptions when querying a website.
#. Send a `HTTP HEAD` request and verify that `Content-Type` is `text/html` and `charset` is either `UTF-8` or `US-ASCII`.
#. Have a map of visited URLs to avoid revisiting them.
#. Follow HTTP redirects.

Upcoming features
=================
* Configure the number of threads and timeout via cmd args.
* Allow web links from all subdomains.
* Allow web links from a list of domains.
* Allow web links matching a pattern.
* Add an option for hierarchical sitemap instead of directed graph.
* Use PriorityQueue instead of Queue to process links with higher decay first.
* Fine-graned info, warn and error logging.
* Pass seed links from a file.
* Save to and resume from a DB/persistent data source.
* Faster concurrency and better performance with asyncio.
pvcastro-iberlef 0.4.1
System for the IberLEF 2019 NER Task

## Installation instructions

1. Download and install the latest Anaconda distribution from [here](https://www.anaconda.com/distribution/#download-section)

2. Start the *Anaconda Powershell Prompt* if running Windows, or a terminal otherwise. Make sure conda is available in the command line path.

3. Create a conda environment running `conda create -n pvcastro-iberlef python=3.6`

4. Update Anaconda running `conda update -n base -c defaults conda`

5. Install pytorch running `conda install pytorch-cpu -c pytorch -n pvcastro-iberlef`

6. Activate the created conda environment using `conda activate pvcastro-iberlef`

7. Install the AllenNLP framework running `pip install -U allennlp`

8. Download the spacy model running `python -m spacy download en_core_web_sm`

9. Install the pvcastro-iberlef module running `pip install -U pvcastro-iberlef`

---

## Execution instructions

Run the NER prediction with a command as `python -m pvcastro_iberlef.predict_ner --document-path path_to_the_input_document --out-path path_to_the_output_file`

Parameters: 

- **document-path**: path to the document containing the text to be predicted for NER, with one token per line, with sentences separated by blank lines.
- **out-path**: path to the document where the predictions results will be written. **Must specify a filename**. Example: C:\iberlef\predictions.txt

## Observations

Since the IberLEF NER model uses two language models based on [ELMo](https://github.com/allenai/allennlp/blob/master/tutorials/how_to/elmo.md), the trained model ended up quite big, with **1.4Gb aproximately**. It takes a while to download it the first time, but the model is cached, so following executions after the first will be quicker.