semvec/pypi-packages huggingface dataset:
monetdb-pystethoscope 0.3.2
MonetDB profiler connection tool
Requires Python: >=3.5,<4.0

|PyPIBadge|_ |ActionsBadge|_ |DocsBadge|_ |CoverageBadge|_

Introduction
============

``pystethoscope`` is a command line tool to filter and format the events coming
from the MonetDB profiler. The profiler is part of the MonetDB server and works
by emitting two JSON objects: one at the start and one at the end of every MAL
instruction executed. ``pystethoscope`` connects to a MonetDB server process,
reads the objects emitted by the profiler and performs various transformations
specified by the user.

Installation
============

Installation is done via pip:

.. code:: shell

   pip install monetdb-pystethoscope

This project is compatible with Python 3.5 or later and with MonetDB server
version Jun2020 or later.

We recommend the use of virtual environments (see `this
primer <https://realpython.com/python-virtual-environments-a-primer/>`__
if you are unfamiliar) for installing and using
``monetdb-pystethoscope``.

Developer notes
===============

``pystethoscope`` is developed using
`Poetry <https://python-poetry.org/>`__, for dependency management and
packaging.

Installation for development
----------------------------

In order to install ``pystethoscope`` do the following:

.. code:: shell

   pip3 install --user poetry
   PYTHON_BIN_PATH="$(python3 -m site --user-base)/bin"
   export PATH="$PATH:$PYTHON_BIN_PATH"

   git clone git@github.com:MonetDBSolutions/monetdb-pystethoscope.git
   cd monetdb-pystethoscope
   poetry install
   poetry run pystethoscope --help

Documentation
=============

For more detailed documentation please see the documentation on `readthedocs
<https://monetdb-pystethoscope.readthedocs.io/en/latest/>`__.

.. |ActionsBadge| image:: https://github.com/MonetDBSolutions/monetdb-pystethoscope/workflows/Test%20pystethoscope/badge.svg?branch=master
.. _ActionsBadge: https://github.com/MonetDBSolutions/monetdb-pystethoscope/actions
.. |DocsBadge| image:: https://readthedocs.org/projects/monetdb-pystethoscope/badge/?version=latest
.. _DocsBadge: https://monetdb-pystethoscope.readthedocs.io/en/latest/?badge=latest
.. |CoverageBadge| image:: https://codecov.io/gh/MonetDBSolutions/monetdb-pystethoscope/branch/master/graph/badge.svg
.. _CoverageBadge: https://codecov.io/gh/MonetDBSolutions/monetdb-pystethoscope
.. |PyPIBadge| image:: https://img.shields.io/pypi/v/monetdb-pystethoscope.svg
.. _PyPIBadge: https://pypi.org/project/monetdb-pystethoscope/
marquez-pubsub 2021.17.860
Short Project Description

# marquez-pubsub

Short Project Description

[![Circle CI](https://circleci.com/gh/globality-corp/marquez-pubsub/tree/develop.svg?style=svg)](https://circleci.com/gh/globality-corp/marquez-pubsub/tree/develop)

## Quickstart

Create a virtualenv for the project, using ``virtualenvwrapper``:

    mkvirtualenv marquez_pubsub

Then run the following commands to bootstrap your environment.

    git clone https://github.com/globality-corp/marquez_pubsub
    cd marquez_pubsub
    pip install -e .

You will also need to create the local PostgreSQL users and databases initially:

    createuser marquez_pubsub
    createdb -O marquez_pubsub marquez_pubsub_db
    createdb -O marquez_pubsub marquez_pubsub_test_db


## Enabling in a service

In addition to the usual application setup, you will need to run this via uWSGI:
    
    pip install uwsgi

Set up the appropriate configuration enable the async publisher:

    {
        "message_producer": {
            "async_enabled": "True"
        }
    }

Include the `message_producer` as a component in your app:

    graph.use(
        "message_producer",
        "publish_message_worker_thread",
    )

And then run your app through uWSGI:

    <APP>__SETTINGS=<app>.conf uwsgi  -H ~/.virtualenvs/<app> --http-socket 127.0.0.1:<port> \
       --need-app \
       --module <app>.wsgi:app \
       --processes=1 \
       --master \
       --pidfile /tmp/<app>_master.pid \
       --enable-threads

Note the final configuration flag at the bottom. Without this, uwsgi will effectively prevent the background thread from progressing.

After this point, you may replace any instance of `sns_producer` with `message_producer` in your application.

## Database schema

To enable the `message_producer` to persist a given record, the application will also need a migration to create the new underlying tables for maintaining the queue.

    def upgrade():
        op.create_table(
            "message_queue",
            Column("id", Serial(), server_default=FetchedValue(), nullable=False),
            Column(
                "created_at",
                postgresql.TIMESTAMP(),
                server_default=func.now(),
                autoincrement=False,
                nullable=False,
            ),
            Column("message", postgresql.BYTEA(), nullable=False),
            Column(
                "retry_count",
                postgresql.INTEGER,
                server_default=text("0"),
                nullable=False,
            ),
        )
        op.create_primary_key("message_queue_pkey", "message_queue", ["id", "retry_count"])