semvec/pypi-packages huggingface dataset:
python-yaml-logger 0.3.1
YAML formatter for the standard Python logging module
License: MIT

python-yaml-logger
==================

|travis_status|_

Overview
--------
``python-yaml-logger`` is a formatter for the standard Python logging module designed to allow formatting log data as YAML_. Using YAML allow for both human- and machine-readable log files, and allows developers to quickly parse and make better use of log data.


Installation
------------

Manual installation::

  $ wget http://github.com/OldhamMade/python-yaml-logger/archive/master.tar.gz -O- | tar zx
  $ cd python-yaml-logger-master
  $ python setup.py install


Usage
-----

::

   import logging
   import yamlformatter

   logger = logging.getLogger()
   logHandler = logging.FileHandler('my.yamllog')
   formatter = yamlformatter.YAMLFormatter()
   logHandler.setFormatter(formatter)
   logger.addHandler(logHandler)


Example output
--------------

::

   - asctime: 2013-04-10 15:39:26,014
     created: 1365604766.014612
     levelname: INFO
     message: test message
     name: logger_name

.. _YAML: http://en.wikipedia.org/wiki/YAML
.. |travis_status| image:: https://secure.travis-ci.org/OldhamMade/python-yaml-logger.png
.. _travis_status: https://secure.travis-ci.org/OldhamMade/python-yaml-logger
mdr 0.0.1
python library to detect and extract listing data from HTML page
License: MIT

===
MDR
===

.. image:: https://travis-ci.org/scrapinghub/mdr.svg?branch=master
    :target: https://travis-ci.org/scrapinghub/mdr

MDR is a library detect and extract listing data from HTML page. It implemented base on the `Finding and Extracting Data Records from Web Pages <http://dl.acm.org/citation.cfm?id=1743635>`_ but
change the similarity to tree alignment proposed by `Web Data Extraction Based on Partial Tree Alignment <http://doi.acm.org/10.1145/1060745.1060761>`_ and `Automatic Wrapper Adaptation by Tree Edit Distance Matching <http://arxiv.org/pdf/1103.1252.pdf>`_.


Requires
========

``numpy`` and ``scipy`` must be installed to build this package.

Usage
=====

Detect listing data
~~~~~~~~~~~~~~~~~~~

MDR assume the data record close to the elements has most text nodes::

    [1]: import requests
    [2]: from mdr.mdr import MDR
    [3]: mdr = MDR()
    [4]: r = requests.get('http://www.yelp.co.uk/biz/the-ledbury-london')
    [5]: candidates, doc = mdr.list_candidates(r.text.encode('utf8'))
    ...

    [8]: [doc.getpath(c) for c in candidates[:10]]
     ['/html/body/div[2]/div[3]/div[2]/div/div[1]/div[1]/div[2]/div[1]/div[2]/ul',
     '/html/body/div[2]/div[3]/div[2]/div/div[1]/div[2]',
     '/html/body/div[2]/div[3]/div[2]/div/div[1]/div[2]/div[2]',
     '/html/body/div[2]/div[3]/div[1]/div/div[4]/div[1]/div/div[1]/div/div[2]/div[1]/div[1]/div',
     '/html/body/div[2]/div[3]/div[1]/div/div[4]/div[2]/div/div[3]',
     '/html/body/div[2]/div[3]/div[1]/div/div[4]/div[1]/div/div[2]/ul/li[2]/div/div/ul',
     '/html/body/div[2]/div[3]/div[2]/div/div[1]/div[1]/div[2]/div[1]',
     '/html/body/div[2]/div[3]/div[2]/div/div[1]/div[2]/div[2]/div[1]/table/tbody',
     '/html/body/div[2]',
     '/html/body/div[2]/div[4]/div/div[1]']

Extract data record
~~~~~~~~~~~~~~~~~~~

MDR can find the repetiton pattern by using tree matching under certain candidate DOM tree.then it will build a mapping from so-called `seed element` to a list of matched elements from different DOM trees.

Used with annotation (optional)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can annotate the seed record with any tools (e.g. scrapely_) you like, then mdr will be able to find the other  data in the page.

e.g. you can find this demo page here_. the colored data in first row are annotated manually, the rest are extracted by MDR.

Author
======

Terry Peng <pengtaoo@gmail.com>

License
=======

MIT

.. _scrapely: https://github.com/scrapy/scrapely
.. _here: http://ibc.scrapinghub.com/tmp/h.html