Saravanan666/pypi-org huggingface dataset:
Package: logseq-migration
logseq-migrationDownload and use local copies of assets in logseq after importing a file from roam.NB:Users have reported problems running this on Windows. I have been unable to reproduce the problem, so I cannot
fixit. The software should run on any model of a Raspberry Pi so Windows users with a Pi could install and run the
software there.Roam stores the assets (images, pdfs, mp4 files etc.) in the cloud.Whilelogseqmakes it easy for you to import a roamjsonexport, yourassetswill remain in the cloud.That worries me, becauseI want to store all the data that I own in my local file system.I want to be able to view my pages even if I am not connected to the Internet.The assets could disappear if roam research ceased trading or changed its policy about closed accounts.So I adapted the code fromhereto suitlogseqinstead ofObsidian, wrote some minimal tests and refactored the code.It's currentlybetasoftwareIt's had limited testing, but it worked for me on a graph with about 1800 nodes.Once installed, you can run the code each time you want to process a recently imported
roam graph.Since the program messes with the markdown in your pages (in order to make asset links local),back up your graph before you run this code!The application is easy to install and run.
You'll find instructions below.InstallationInstall required softwarePython 3The code is written in Python, and you  will need an installed copy of Python3.8 or later.If you don't already have a suitable copy of Python installed, you'll find installation instructionshere.Under Windows, it appears that you must install Python system-wide (with
script wrappers) and ask for Python to be added to your PATH.pip3pipis Python's package manager. You'll needpip3, the version that supports Python3.This is normally installed with Python3. If it isn't for some reason you can followthese instructions.Install the applicationrunpip3 install logseq-migrationRun the applicationrunlocalise_assets <vault_dir>wherevault_diris the directory in which yourlogseqgraph lives.You'll see version information printed, and the worddonewhen the
migration is completeThe migrator will create a logfile calledmigration.log.
By default, this will only log the start and end of the migration, with any
errors.If you wish to see detailed debug information,
runlocalise_assets <vault_dir> 1The code isidempotent.
In other words, if you run it two or more times, nothingshouldchange after the first run.Let me know how you get on, and please raise an issue if there are problems,
or you want to suggest new features.There's aroadmapand aproject journal. If you're interested in contributing, get in
touch.
Help is always welcome!I'm@rareblogon Twitter.
Package: logshipper
Log shipper===================Introduction---`logshipper`  is a tool to gather logs from various sources, process them and send them elsewhere. It is similar to [logstash](https://logstash.net), except it doesn't use the java virtual machine and ruby class library, which should help improve performance and decrease memory hunger.Key concepts---**Pipelines** are lists of steps to be performed on a log message. Pipelines are stored in `yaml` format.  Common tasks are to drop irrelevant messages, extract valuable data and sending the message somewhere.**Steps** are part of pipelines. They typically contain a `match` action to apply the task only to specific messages. Tasks can contain multiple actions.**Actions** are part of steps. Actions include `match`, `set` and `statsd`. An action receives the message, and can alter it, send it somewhere, ignore it, or decide this action should be skipped.Actions---###matchThe match action matches a regex to a specific field of a message. Only when all of the regexes matches, the rest of the actions of this step will get executed.Named groups get registered as variables, which can be accessed in subsequent steps. When there's a single match, unnamed groups get registered as backreferences, and are avail during the rest of the step.Example:match:message: (start_time):\s+(?P<time>\d+)set:start_time: "{1} {time}"A shorter syntax is available when there's a single match against the `message` field, the above example is equivalent to:match: (start_time):\s+(?P<time>\d+)set:start_time: "{1} {time}"###setSets fields of the message. For an example, see `match`###rabbitmqSends messages to RabbitMQ. Accepts the following parameters:* **username** RabbitMQ username, defaults to `guest`* **password** RabbitMQ password, defaults to `guest`* **host** RabbitMQ hostname, defaults to `127.0.0.1`* **port** RabbitMQ port, defaults to `5672`* **exchange** Defaults to `logshipper`* **queue** Defaults to `logshipper`* **key** The routing key. Defaults to `logshipper`###statsdSends a value to statsd.* **host** defaults to `127.0.0.1`* **port** defaults to `8125`* **sample_rate** defaults to `1.0`* **type** Accepted values are `counter`, `gauge` and `timer`, defaults to `counter`* **value** The value to send. Defaults to `1.0`* **multiplier** The amount to multiply the value by. Defaults to `1.0`* **delta** boolean, only used for gauge, whether to send differential values or absolute values. Defaults to `False`* **prefix** the prefix for the stat name backreferences not allowed* **name** the name for the stat, backreferences allowed (required)### dropDrops the message.
Package: logshot
Log-ShotA simplified log tool with some custom features that allows to manage loggers across multiple python files, with a root logger that is easy to enable or disable whenever it is necessary from any file.Installationpip install logshotUsage Example# -- Global logger config
title = "log-file"
logs_dir = 'logs'
if not os.path.exists(logs_dir):
    os.mkdir(logs_dir)
logging.config_log_capture(logs_dir, logfile_title=title)
logging.root_level = logging.DEBUG;
logging.start_log_capture()
# -- Logger
logger = logging.Logger(module_name=__name__)
logger.level = logging.DEBUG
...
logger.info("Opening example path...")
try:
    with open('C:\\Users\\example\\of\\logshot\\package', 'r') as file:
        ...
except Exception as err:
    logger.error(err); exit(1)File Output2022-01-24 00:59:32.138305
[INFO] __main__: Opening example path...

2022-01-24 00:59:40.446509
[ERROR] __main__: [WinError 3] The system cannot find the path specified: 'C:\\Users\\example\\of\\logshot\\package'
Package: logshuttle
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.For more information, please refer to <http://unlicense.org>Description: UNKNOWN
Platform: MacOS X
Platform: Posix
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Package: logsight-sdk-py
Logsight SDK Python is a set of open source libraries, tools, documentation,
and code samples that provides an easy, lightweight solution to instrument
the source code of custom applications so that you can monitor their behavior
and performance with logsight.ai.Logsight SDK uses a client/server architecture and communicates via HTTP with
the AI-driven log analytics platformlogsight.ai.Main advantages:Ease of useNo 3rd-party dependenciesEasily portability to other programming languagesMain functionalities:Create and delete applicationsSend log data records to logsight.aiRetrieve incident reportsAnalyze the quality of logsCompare data logsThe Logsight SDK Python package is deployed to the following external platforms:Test Python Package Index (TestPyPI):Test PyPIPython Package Index (PyPI):PyPIDocumentation:docs
Package: logsnag
LogSnagGet notifications and track your project events.Installationpip3installlogsnagUsageImport LibraryfromlogsnagimportLogSnagInitialize Clientlogsnag=LogSnag(token='7f568d735724351757637b1dbf108e5',project="my-saas")Publish Eventlogsnag.track(channel="waitlist",event="User Joined",user_id="user_123",description="Email: john@doe.com",icon="🎉",tags={"source":"google",},notify=True)User Propertieslogsnag.identify(user_id="user_123",properties={"name":"John Doe","email":"john@doe.com","plan":"free",})Group Propertieslogsnag.group(user_id="user_123",group_id="group_123",properties={"name":"ACME Inc.","plan":"enterprise","fund":"Series A",})Publish Insightlogsnag.insight.track(title='User Count',value=100,icon='👨',)Increment Insightlogsnag.insight.increment(title='User Count',value=1,icon='👨',)
Package: logsparsersubsystem
LogsParserCommon log parser and analyzerProject structureHow to devpoetry shell
poetry install├── debug.py
├── docs
│   ├── context diagram.drawio
│   └── uml.drawio
├── logs_parser
│   ├── analyzer - analyzer for logs
│   │   ├── analyzer.py
│   │   ├── data - data for IP2Location Lib
│   │   │   ├── IP2LOCATION-LITE-DB11.BIN
│   │   │   ├── LICENSE_LITE.TXT
│   │   │   └── README_LITE.TXT
│   │   └── plots.py - plots (deprecated)
│   ├── parser (multithread parser)
│   │   ├── filter.py - filter decorators
│   │   ├── logs_reader.py - logs reader
│   │   ├── log_structure.py - data classes
│   │   ├── logs_writer.py - logs writers
│   │   └── parser.py - parser
│   └── python_callables.py - callables for Airflow
├── notebooks - Jupyter notebooks,some experiment
│   ├── experiment.ipynb
│   └── gorod_09102021_231021.csv
├── poetry.lock
├── pyproject.toml
├── README.md
└── scripts - utils
    ├── split_logs.py
    └── unite_logs.pyArhitecture contex diagramUML diagramHow to public to PypiFirst you need create api on pypipoetryconfigpypi-token.pypi$(cat.token)poetrypublish--build
Package: logs-py
LOGS-PyLOGS is a scientific data management system (SDMS) software allowing for automated data collection, visualization, and organization. Within its internal organizational concepts it allows you to enrich you experimental data with metadata. LOGS allows you to adapt many of its organization structures which enables your data management to follow your internal workflows.LOGS-Py is a Python packageinteracting with the LOGS web API to enable you to extract and push data to LOGS and generally operateinteract with the LOGS backend in a programatic way. The main motivation behind in the desing of library is to keep this interaction as pythonic as possible. The communication with the API remains mainly in the background while the user of the library handles native Python objects whereby theuser is still able to interact with nearly are LOGS functionalities and entities.Thusthis libraryfirstlytargets lab and data scientistallowing them to freely interact with experimental data and its meta-data without any pre-knowledge of Web technologies and communication. Secondly, itfacilitates power-usersto implement highly specific workflows automations and 3rd-party software intergrations with LOGS and other lab software.InstallationTheLOGS-Pypackage can be easily installed by usingpip.For this open theterminaland do:pipinstalllogs-py
Package: logstack
LogStackLogStackis a convenience package to communicate with aLogStackserver,
which is available as aDockerimagehere.Getting StartedpipinstalllogstackContributingIf you face any issues with project, please create and submit an issue,
detailing the problem and providing examples to reproduce the problem.If you wish to contribute to the project via enhancements, please submit an
issue outlining how your suggested changes improves the project and the scope of
the change.
Package: logstamp
logstampAnother logger, made for data science and heavy-compute workloads. Logs to timestamped file too.Usagefromlogstampimportlogimporttimelog("doing big thing")time.sleep(3)log("did big thing, doing other thing")time.sleep(5)log("all done")logs to file:doing big thing ... is now 2022-12-19 18:05:22.769930 ... last interval 0:00:00.000089
did big thing, doing other thing ... is now 2022-12-19 18:05:25.773236 ... last interval 0:00:03.003306
all done ... is now 2022-12-19 18:05:30.778863 ... last interval 0:00:05.005627
Package: logstash-easy
logstash-easyVery easy to use Logstash logging handler.InstallationInstall via pip:pip install logstash-easyUsageJust write one line of code and enjoy:importlogstash_easyBy default logging messages will be sent tolocalhost:5000but you
can set environment variablesLOGSTASH_HOST,LOGSTASH_PORTandLOGSTASH_TAGfor tagging your application.  If you defineLOGSTASH_APP_NAMEvariable thenapp_namefield will be added
to all logging messages (it can be used for creating Elasticsearch
indices per application).
Package: logstash-filter-test
logstash-filter-testLogstashis a program that collects
json documents from various inputs, transform them according to a configuration
file (afilter), and sends them to various outputs. This script helps you make
sure that your filter does what you expect by writing a test suite.InstallationYou can install using pip:pip install logstash-filter-testor just copylogstash_filter_run.pyandlogstash_filter_test.py. There are
no dependencies. Python 3.5+ and 2.7 are supported.ExampleHere is a simple test suite file:testcases.js:[// Test a message with a value[{"message":"<<2018-06-11 13:45:39,127+0300>> TAU = 6.2831853"},{"date":"2018-06-11 13:45:39,127+0300","@timestamp":"2018-06-11T10:45:39.127Z","msg_text":"TAU = 6.2831853","metric":"TAU","value":6.2831853,}],]filter.conf:filter {
  grok {
    match => {
      "message" => [
        "^<<%{TIMESTAMP_ISO8601:date}>> %{GREEDYDATA:msg_text}$"
      ]
    }
  }
  date {
    match => ["date", "ISO8601"]
  }
  grok {
    match => {
      "msg_text" => [
        "^%{WORD:metric} = %{NUMBER:value:float}"
      ]
    }
    tag_on_failure => []
  }
}Put them in a directory, run./logstash_filter_test.pyand it will inform you
if the testcases passed, and if not, why.testcases.jscan be a simple json file. It has the extension.jsbecause
javascript comments and trailing commas are allowed, thanks tojstyleson. It is a list of
test cases. Each test case is a list of size 2. The first item is the input
json document. The second item is compared to the output json document. All
fields that are defined must be equal to the fields in the output document.
The output document may include other fields. To test that an output field
doesn't exist, use"field": null.Testing from PythonIf you don't like the testcase file format, it's easy to test by yourself:>>> from logstash_filter_run import logstash_filter_run
>>> logstash_filter_run([{"a": 3}], 'filter { mutate { copy => { "a" => "b" } } }')
[{'@timestamp': '2018-07-10T18:26:58.411Z',
  '@version': '1',
  'a': 3,
  'b': 3,
  'host': 'myhost'}]
Package: logstashHandler
logstashHandler is a basic logging handler for sending to a logstash instance via UDP or TCP encoded as json.Basic usageSetup an input on the relevant logstash server:input {
  udp {
    port => 12345
    codec => json
  }
}Setup handler and ship logs:python
from logstashHandler import logstashHandler

lhandler = logstashHandler(host='mylogserver.example.com',port=12345,proto='UDP')
logger.addHandler(lhandler)
logger.warn("Something went wrong")To send additional fields to logstash, use the keyword extra and send a dict starting with extraFields:logger.warn('DANGER DANGER',extra={'extraFields':{'name':'W. Robinsson', 'planet':'Unkown'}})The default level field name and individual level names can be overridden as so:levels = {'ERROR': 3, 'WARNING': 2, 'INFO': 1, 'DEBUG': 0}
lhandler = logstashHandler(
    host='mylogserver.example.com',
    port=12345,
    proto='UDP',
    levels=levels,
    levelLabel='severity'
)More info can be found on the github page.
Package: logstash_handler
This library is provided to allow standard python logging to stream JSON logs
to logstash.InstallingPip:pip install logstash_handlerPypi:https://pypi.python.org/pypi/logstash_handlerManual:python setup.py installUsageJson outputs are provided by the LogstashFormatter logging formatter.import logging
from logstash_formatter import LogstashFormatter
from logstash_handler import LogstashHandler

logger = logging.getLogger()
handler = LogstashHandler('localhost', 5000, ssl=False)
formatter = LogstashFormatter()

handler.setFormatter(formatter)
logger.addHandler(handler)The LogstashHandler takes the following named parameters:host: host to connect toport: host to connect tossl: boolean indicating if the stream should be encrypted. Default is Truekeyfile: The path to the encrypting keycertfile: The path to the encrypting certificate
Package: logstash-index-cleaner
THIS PROJECT IS DEPRECATED! USE elasticsearch-curator:https://github.com/elasticsearch/curatorDelete logstash indexes older than n days from ElasticSearch.$ logstash-index-cleaner -h
usage: logstash-index-cleaner [-h] [-v] [--host HOST] [--port PORT]
                              [-t TIMEOUT] [-p PREFIX] [-s SEPARATOR]
                              [-H HOURS_TO_KEEP] [-d DAYS_TO_KEEP] [-n]

Delete old logstash indices from Elasticsearch.

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  --host HOST           Elasticsearch host.
  --port PORT           Elasticsearch port
  -t TIMEOUT, --timeout TIMEOUT
                        Elasticsearch timeout
  -p PREFIX, --prefix PREFIX
                        Prefix for the indices. Indices that do not have this
                        prefix are skipped.
  -s SEPARATOR, --separator SEPARATOR
                        Time unit separator
  -H HOURS_TO_KEEP, --hours-to-keep HOURS_TO_KEEP
                        Number of hours to keep.
  -d DAYS_TO_KEEP, --days-to-keep DAYS_TO_KEEP
                        Number of days to keep.
  -n, --dry-run         If true, does not perform any changes to the
                        Elasticsearch indices.Project historyThis is a fork ofhttps://github.com/crashdump/logstash-elasticsearch-scripts.Since ElasticSearch 0.90 all fields are compressed, and some benchmarks show that
nothing improves optimizing those already compressed indices. That’s why this fork
does not include the originallogstash_index_optimize.pyOptimizations references:https://github.com/logstash/logstash/wiki/Elasticsearch-Storage-Optimizationhttps://github.com/jordansissel/experiments/tree/master/elasticsearch/disk
Package: logstash_logger
Logstash LoggerLogstash Logger handler connect to ELK stackFree software: MIT licenseDocumentation:https://logstash-logger.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2017-11-17)First release on PyPI.
Package: logstash-pipeline-parser
About The ProjectHow Logstash Pipeline WorksThe Logstash event processing pipeline has three stages: inputs → filters → outputs.Inputs generate events, filters modify them, and outputs ship them elsewhere.Inputs and outputs support codecs that enable you to encode or decode the data as it enters or exits the pipeline without having to use a separate filter.The pipeline configuration file is a custom format developed by the Logstash folks usingTreetop.
The grammar itself is described in the source filegrammar.treetopand compiled using Treetop into the customgrammar.rbparser.
That parser is then used to set up the pipeline from the Logstash configuration.See also:How Logstash WorksCreating a Logstash pipelineDocumentationThe latest documentation is athttps://tomaskoutek.github.io/logstash-pipeline-parser/and contains description of all methods, classes with individual examples and their results.
Or you can display description directly with the help function.For example:fromlogstash_pipeline_parserimportPipelinehelp(Pipeline.parse)Getting StartedInstallingInstall frompip:pip install logstash-pipeline-parserDependenciespyparsingfor creatingPEGparserQuick ExampleFor full documentation with examples please seedocumentation.Let's try pipeline with two input plugins:fromlogstash_pipeline_parserimportPipelinedata=r"""input {beats {host => "0.0.0.0"port => 5044client_inactivity_timeout => 3600include_codec_tag => trueenrich => [source_metadata, ssl_peer_metadata]ssl => truessl_key => "/some/path/my.key"id => "input_beats"}udp {port => 5045host => "0.0.0.0"}}"""pipeline=Pipeline(data)ast=pipeline.parse()will produceAbstract syntax tree:fromipaddressimportIPv4AddressfrompathlibimportPath[["input",[["beats",[["host",[IPv4Address("0.0.0.0")]],["port",[5044]],["client_inactivity_timeout",[3600]],["include_codec_tag",[True]],["enrich",[["source_metadata","ssl_peer_metadata"]]],["ssl",[True]],["ssl_key",[Path("/some/path/my.key")]],["id",["input_beats"]]]],["udp",[["port",[5045]],["host",[IPv4Address("0.0.0.0")]]]]]]]Of course, it is possible to parse all kinds of plugins, conditions and data types.
You can also search in the pipeline. The searched key can also contain the wildcard*, for example"output.*.hosts"will return
(if the pipeline definition contains them):("output.elasticsearch.hosts", [["127.0.0.1:9200","127.0.0.2:9200"]])("output.logstash.hosts", ["127.0.0.1:9801"])results=pipeline.search("input.*.port")print(list(results))# [#   ("input.beats.port", [5044]),#   ("input.udp.port", [5045])# ]Thesearchmethod returns a generator, so we can easily iterate:forkey,valueinpipeline.search("*.port"):print(f"key:{key}, value:{value[0]}")# key: input.beats.port, value: 5044# key: input.udp.sub.port, value: 5045The return value can be any element from the tree (integer, string, field, plugin,...):results=pipeline.search("input.beats.enrich")print(list(results))# [#   ("input.beats.enrich", [["source_metadata", "ssl_peer_metadata"]])# ]LicenseDistributed under the MIT License. See LICENSE for more information.