semvec/pypi-packages huggingface dataset:
dryscrape 1.0
a lightweight Javascript-aware, headless web scraping library for Python
License: MIT

# Overview

**Author:** Niklas Baumstark

dryscrape is a lightweight web scraping library for Python. It uses a 
headless Webkit instance to evaluate Javascript on the visited pages. This 
enables painless scraping of plain web pages as well as Javascript-heavy 
“Web 2.0” applications like
Facebook.

It is built on the shoulders of
[capybara-webkit](https://github.com/thoughtbot/capybara-webkit)'s 
[webkit-server](https://github.com/niklasb/webkit-server). A big thanks goes 
to thoughtbot, inc. for building this excellent piece of software!

# Changelog

* 1.0: Added Python 3 support, small performance fixes, header names are now
  properly normalized. Also added the function `dryscrape.start_xvfb()` to
  easily start Xvfb.
* 0.9.1: Changed semantics of the `headers` function in
  a backwards-incompatible way: It now returns a list of (key, value)
  pairs instead of a dictionary.

# Supported Platforms

The library has been confirmed to work on the following platforms:

* Mac OS X 10.9 Mavericks and 10.10 Yosemite
* Ubuntu Linux
* Arch Linux

Other unixoid systems should work just fine.

Windows is not officially supported, although dryscrape should work
with [cygwin](https://www.cygwin.com/).

# Installation, Usage, API Docs

Documentation can be found at 
[dryscrape's ReadTheDocs page](http://readthedocs.org/docs/dryscrape/).

Quick installation instruction:

    # pip install dryscrape

# Contact, Bugs, Contributions

If you have any problems with this software, don't hesitate to open an
issue on [Github](https://github.com/niklasb/dryscrape) or open a pull
request or write a mail to **niklas baumstark at Gmail**.
webkit-server 1.0
a Webkit-based, headless web client
License: MIT

# Overview

**Author:** Niklas Baumstark

A standalone version of the Webkit server included in [capybara-webkit][1].
It includes a slim Python wrapper and the following improvements over the
original version from thoughtbot:

* `Wait` command to wait for the current page to load
* `SetAttribute` command to [configure certain `QWebkit` settings][2]
* `SetHtml` command to [load custom HTML][3] into the browser (e.g. to
  execute scripts on web pages scraped by a static scraper)
* `SetViewportSize` command to set the viewport size of the in-memory browser

If you are interested in web scraping using this server, have a look at [dryscrape][4].

# Building and Installing

To install the Python binding (this also builds the server and places it into
Python's `site-package` directory):

    sudo python setup.py install

If you don't need the Python bindings, you can also use the supplied `build.sh`
shellscript to build the server only.

# Contact, Bugs, Contributions

If you have any problems with this software, don't hesitate to open an 
issue on [Github](https://github.com/niklasb/webkit-server) or open a pull 
request or write a mail to **niklas 
baumstark at Gmail**.

# License

This software is based on [capybara-webkit][1].
capybara-webkit is Copyright (c) 2011 thoughtbot, inc. It is free software, and
may be redistributed under the terms specified in the LICENSE file.

 [1]: https://github.com/thoughtbot/capybara-webkit
 [2]: https://github.com/thoughtbot/capybara-webkit/pull/171
 [3]: https://github.com/thoughtbot/capybara-webkit/pull/170
 [4]: https://github.com/niklasb/dryscrape
ipython-d3-sankey 0.1.1
an IPython widget for the d3 Sankey plugin
License: MIT
Keywords: ipython-d3-sankey ipython

d3 Sankey
=========

an IPython widget for the `d3 Sankey
plugin <http://bost.ocks.org/mike/sankey/>`__

Installation
------------

``pip install ipython-d3-sankey``

Usage
-----

See the example notebook.
muffin-jade 0.0.8
Jade templates for Muffin Framework
License: MIT
Keywords: asyncio,aiohttp,muffin,jade

Muffin-Jade
###########

.. _description:

Muffin-Jade -- Jade template engine for Muffin framework.

.. _badges:

.. image:: http://img.shields.io/travis/klen/muffin-jade.svg?style=flat-square
    :target: http://travis-ci.org/klen/muffin-jade
    :alt: Build Status

.. image:: http://img.shields.io/pypi/v/muffin-jade.svg?style=flat-square
    :target: https://pypi.python.org/pypi/muffin-jade

.. image:: http://img.shields.io/pypi/dm/muffin-jade.svg?style=flat-square
    :target: https://pypi.python.org/pypi/muffin-jade

.. image:: http://img.shields.io/gratipay/klen.svg?style=flat-square
    :target: https://www.gratipay.com/klen/
    :alt: Donate

.. _contents:

.. contents::

.. _requirements:

Requirements
=============

- python >= 3.3

.. _installation:

Installation
=============

**Muffin-Jade** should be installed using pip: ::

    pip install muffin-jade

.. _usage:

Usage
=====

Add **muffin_jade** to **PLUGINS** in your Muffin Application configuration.

Options
-------

**JADE_CACHE_SIZE** -- Cache size for compiled templates (100)

**JADE_ENCODING** -- Templates' encoding (UTF-8)

**JADE_PRETTY** -- Pretty output (True)

**JADE_TEMPLATE_FOLDERS** -- List of pathes to templates folder ([templates])

Views
-----

::

    # Register custom context provider
    # could be a function/coroutine
    @app.ps.jade.ctx_provider
    def custom_context():
        return { 'VAR': 'VALUE' }

    # Register a function into templates
    @app.ps.jade.register
    def sum(a, b):
        return a + b

    @app.register('/')
    def index(request):
        """ Check for user is admin. """
        local_context = {'key': 'value'}
        return app.ps.jade.render('index.jade', **local_context)


.. _bugtracker:

Bug tracker
===========

If you have any suggestions, bug reports or
annoyances please report them to the issue tracker
at https://github.com/klen/muffin-jade/issues

.. _contributing:

Contributing
============

Development of Muffin-Jade happens at: https://github.com/klen/muffin-jade


Contributors
=============

* klen_ (Kirill Klenov)

.. _license:

License
=======

Licensed under a `MIT license`_.

.. _links:


.. _klen: https://github.com/klen

.. _MIT license: http://opensource.org/licenses/MIT
shell.py 0.5.3
Shell power for Python.
License: MIT

shell.py
========

.. image:: https://badge.fury.io/py/shell.py.png
    :target: http://badge.fury.io/py/shell.py

.. image:: https://travis-ci.org/houqp/shell.py.svg?branch=master
    :target: https://travis-ci.org/houqp/shell.py

Bring the good part of Shell scripting to Python.


Install
-------

.. code-block:: bash

    $ pip install shell.py


Usage
-----

Execute a shell command
.......................

Block until return:

.. code-block:: python

    >>> from shell import ex
    >>> ex('echo hello shell.py').stdout()
    'hello shell.py\n'

Asynchronous execution:

.. code-block:: python

    >>> from shell import ex
    >>> c = asex('echo hello shell.py')
    >>> # do something else
    ...
    >>> c.stdout() # wait until process exit and read stdout
    'hello shell.py\n'



Pipe commands
.............

.. code-block:: python

    from shell import ex
    re = (ex("ifconfig")
          | "grep -A 1 eth0"
          | "grep inet"
          | "awk '{print $2}'"
          | "cut -d: -f 2").stdout()

Or

.. code-block:: python

    from shell import pipe_all
    pipe_all(["ls -la ~",
              "awk '{print $9}'",
              "grep -E '^\.'",
              "wc -l"]).stdout()


Use string as stdin
...................

.. code-block:: python

    >>> from shell import instream
    >>> instream("1 2 3").p("awk '{print $1}'").stdout()
    '1\n'

This is equivalent to:

.. code-block:: python

    >>> from shell import ex
    >>> ex("echo 1 2 3").p("awk '{print $1}'").stdout()


IO redirect
............

Overwrite a file:

.. code-block:: python

    >>> from shell import ex
    >>> ex('echo yolo').wr('/tmp/out')
    >>> ex('echo yolo') > '/tmp/out'

Append to a file:

.. code-block:: python

    >>> from shell import ex
    >>> ex('echo yolo').ap('/tmp/out')
    >>> ex('echo yolo') >> '/tmp/out'


Run commands in parallel
........................

Block until all commands return:

.. code-block:: python

    >>> from shell import parallel as par
    >>> par.ex_all(['sleep 2', 'sleep 2']) # return in 2s

Asynchronous parallel execution:

.. code-block:: python

    >>> from shell import parallel as par
    >>> pe = par.asex_all(['sleep 2', 'sleep 2']) # return immediately
    >>> # do something else
    ...
    >>> pe.wait()



Set working directory
.....................

Set the directory in which the commands are executed:

.. code-block:: python

    >>> with shell.cwd('~/server/data/upload/') as old_path:
    >>>     shell.ex('find ./images -name "*.png"') | 'minify ./public' >> 'upload.log'

This is equivalent to:

.. code-block:: python

    >>> shell.ex('find ~/server/data/upload/images -name "*.png"') | 'minify ~/server/data/upload/public' >> '~/server/data/upload.log'


See test cases for more examples.

Tests
-----

Run tests with nosetests(at least v1.3.0):

.. code-block:: bash

    $ make test