semvec/pypi-packages huggingface dataset:
schul-cloud-url-crawler 1.0.17
Crawler for Schul-Cloud Ressources
Keywords: Schul-Cloud Content API,Crawler

schul-cloud-url-crawler
=======================

.. image:: https://travis-ci.org/schul-cloud/url-crawler.svg?branch=master
   :target: https://travis-ci.org/schul-cloud/url-crawler
   :alt: Build Status

.. image:: https://badge.fury.io/py/schul-cloud-url-crawler.svg
   :target: https://pypi.python.org/pypi/schul-cloud-url-crawler
   :alt: Python Package Index

This crawler fetches ressources from urls and posts them to a server.

Purpose
-------

The purpose of this crawler is:

- We can provide test data to the API.
- It can crawl ressources which are not active and cannot post.
- Other crawl services can use this crawler to upload their conversions.
- It has the full crawler logic but does not transform into other formats.

  - Maybe we can create recommendations or a library for crawlers from this case.

Requirements
------------

The crawler should work as follows:

- Provide urls

  - as command line arguments
  - as a link to a file with one url per line

- Provide ressources_

  - as one ressource in a file
  - as a list of ressources

The crawler must be invoked to crawl.

Example
~~~~~~~

This example gets a ressource from the url and post it to the api.

.. code:: shell

    python3 -m ressource_url_crawler http://localhost:8080 \
            https://raw.githubusercontent.com/schul-cloud/ressources-api-v1/master/schemas/ressource/examples/valid/example-website.json

Authentication
~~~~~~~~~~~~~~

You can specify the authentication_ like this:

- ``--basic=username:password`` for basic authentication
- ``--apikey=apikey`` for api key authentication

Further Requirements
--------------------

- **The crawler does not post ressources twice.**
  This can be implemented by

  - caching the ressources locally, to see if they changed

    - compare ressource
    - compare timestamp

  - removing the ressources from the database if they are updated after posting new ressources.

This may require some form of state for the crawler.
The state could be added to the ressources in a ``X-Ressources-Url-Crawler-Source`` field.
This allows local caching and requires getting the objects from the database.

.. _ressources: https://github.com/schul-cloud/ressources-api-v1#ressources-api
.. _authentication: https://github.com/schul-cloud/ressources-api-v1#authorization
backports.print_function 1.1.1
Print function with Python 3.3 flush argument
Requires Python: >=2.6

.. image:: https://img.shields.io/pypi/v/backports.print_function.svg
   :target: https://pypi.org/project/backports.print_function

.. image:: https://img.shields.io/pypi/pyversions/backports.print_function.svg

.. image:: https://img.shields.io/pypi/dm/backports.print_function.svg

.. image:: https://img.shields.io/travis/jaraco/backports.print_function/master.svg
   :target: http://travis-ci.org/jaraco/backports.print_function

Add support for the Python 3.3 flush argument.

License
=======

License is indicated in the project metadata (typically one or more
of the Trove classifiers). For more details, see `this explanation
<https://github.com/jaraco/skeleton/issues/1>`_.

Usage
=====

from backports.print_function import print_

print_('Partial line', end='', flush=True)
backports.method_request 1.2
backports.method_request
Requires Python: >=2.6

.. image:: https://img.shields.io/pypi/v/backports.method_request.svg
   :target: https://pypi.org/project/backports.method_request

.. image:: https://img.shields.io/pypi/pyversions/backports.method_request.svg

.. image:: https://img.shields.io/pypi/dm/backports.method_request.svg

.. image:: https://img.shields.io/travis/jaraco/backports.method_request/master.svg
   :target: http://travis-ci.org/jaraco/backports.method_request

A backport of the urllib.request.MethodRequest class from Python 3.4 which
allows overriding of the method in a class attribute or as a keyword
parameter to the initializer.

See `Python 18978 <http://bugs.python.org/issue18978>`_ for details.

Works on Python 2.6 and later.


License
=======

License is indicated in the project metadata (typically one or more
of the Trove classifiers). For more details, see `this explanation
<https://github.com/jaraco/skeleton/issues/1>`_.

Usage
-----

Use ``method_request.Request`` in place of ``urllib.request.Request``::

    from backports.method_request import Request

    req = Request(..., method='PATCH')
    resp = urllib.request.urlopen(req)
    ...
iologger 1.1.9
Decorator which logs the wrapped function/method.

============
IOLogger
============
Decorator which logs the wrapped function/method.

    The following are logged:

        1. name of the function called
        2. arg(s) passed for the function called (if any)
        3. kwarg(s) passed for the function called (if any)
        4. execution time of the function called (in seconds)

        * also catches and logs any **exceptions** raised *gracefully*.
nlgen 0.1.2
natural language generator

=====
nlgen
=====

.. image::https://travis-ci.org/toumorokoshi/nlgen/settings#
    :alt: build status
    :target: https://travis-ci.org/toumorokoshi/nlgen

Natural Language Generator for Python

-------
Summary
-------

nlgen provides tools that help with natural language generation. nlgen currently features:

- a featured context free grammar representation and generator.

------------
Documentation
------------

http://nlgen.readthedocs.io/en/latest/