semvec/pypi-packages huggingface dataset:
python-darc 1.0.3
Darkweb crawler & search engine.
License: BSD 3-Clause License
Requires Python: >=3.6
Keywords: darkweb, crawler

``darc`` - Darkweb Crawler Project
========================================

``darc`` is designed as a swiss army knife for darkweb crawling.
It integrates ``requests`` to collect HTTP request and response
information, such as cookies, header fields, etc. It also bundles
``selenium`` to provide a fully rendered web page and screenshot
of such view.

The general process of ``darc`` can be described as following:

There are two types of *workers*:

* ``crawler`` -- runs the ``darc.crawl.crawler`` to provide a
  fresh view of a link and test its connectability

* ``loader`` -- run the ``darc.crawl.loader`` to provide an
  in-depth view of a link and provide more visual information