Saravanan666/pypi-org huggingface dataset:
Package: logstashpy
logstashpy : python logging handlers for logstash with SSL/TLS supportThis repository is inspired from original projectpython-logstashand lot of code and pending PR's are also
integrated here as well, some of them like SSL/TLS support. Moreover it is planned to support more serialization formats
as supported in logstash codecs. Currently msgpack is integrated.Some old formatters are shipped but ELK stack now promotes use of ECS. Check outecs-loggingfor latest update. For all handlers default formatter is now StdlibFormatter from ecs_logging but you can always chose
some other formatter or the old formatters.Currently no class is exposed with__all__under__init__.pyfor now as API might change in future.InstallationTo install logstashpy, simply use all time favorite pip and type :$ pip install logstashpy✨Usage>>> from logstash.handlers.tcp_handler import TCPLogstashHandler
>>> from ecs.logger 
>>> host = 'localhost'
>>> logger = logging.getLogger(__name__)
>>> logger.setLevel(logging.INFO)
>>> handler = TCPLogstashHandler(host, 5959, ssl=False, serializer='msgpack')
>>> logger.addHandler(handler)

>>> # add extra field to logstash message
>>> extra = {
>>>     'test_string': 'python version: ' + repr(sys.version_info),
>>>     'test_boolean': True,
>>>     'test_dict': {'a': 1, 'b': 'c'},
>>>     'test_float': 1.23,
>>>     'test_integer': 123,
>>>     'test_list': [1, 2, '3'],
>>> }
>>> logger.info('python-logstash: test extra fields', extra=extra)SSL/TLS SupportSSL is enabled by default. To disable passssl=Falsein Handler's constructor.To quickly setup SSL/TLS related certificates follow some commands below:Generate certificates for Certificate Authority (for self signed certificates)Generate CA key (kind of like private key for CA, will prompt for password, keep it safe)openssl genrsa -des3 -out localCA.key 2048Generate CA pem file (kind of like public key for CA)openssl req -x509 -new -nodes -key localCA.key -sha256 -days 1024 -out localCA.pemNow we have a local certificate authority ready to sign some certificates.Lets generate some private key along with CSR (Certificate Signing Request) for our local logstash server
which will be signed by our local certificate authority. You can pass some other configs here but left for simplicity.openssl req -new -sha256 -nodes -out logserver.csr -newkey rsa:2048 -keyout logserver.keyLet's submit our CSR to our local CA and get out certificate (as protected, you will be prompted for the password as set before)openssl x509 -req -in server.csr -CA localCA.pem -CAkey localCA.key -CAcreateserial -out logserver.crt -days 1000 -sha256You will get a logserver.crt which logstash server can present to its clients valid for 1000 days.Python Version Supportlogstashpy supports python3Sample Logstash ConfigurationExample Logstash Configuration (logstash.conf) for Receiving Events from logstashpy via TCP (omit ssl
related fields if not required):input {
  tcp {
    port => 5959
    codec => json
    ssl_enable => true
    ssl_cert => "/path/to/server.crt"
    ssl_key => "/path/to/server.key"
    ssl_verify => false
  }
}
output {
   stdout {
     codec => rubydebug
   }
}DocumentationComing Soon.
Package: logstash-python-formatter
Python logging formatter for creating log entries in a JSONlogstash-friendlyformat.Supports renaming of python default logging fields to logstash friendly names (e.g: renamingasctimeto@timestamp)LogstashFormatter can receive the following arguments:fmt, list or tuple containing the fields to include in each entry. Defaults to [“asctime”, “levelname”, “filename”, “funcName”, “msg”, “exc_info”].datefmt, date format string to be passed to formatTime(). Defaults to ISO8601 time format.rename, dictionary containing mapping of { key: new_key } to be renamed. Defaults to { “asctime”: “@timestamp” }.version, version as for the @version attribute used in Logstash. Defaults to “1”.Sample output{"@timestamp":"2016-09-28T16:24:24,799","@version":"1","exc_info":null,"filename":"<ipython-input-21-de248ad5b09c>","funcName":"<module>","levelname":"INFO","message":"This is a normal message to be logged"}UsageAddLogstashFormatteras the formatter of your handler, as usual:self.logger=logging.getLogger()self.handler=logging.StreamHandler()self.handler.setFormatter(LogstashFormatter())self.logger.addHandler(self.handler)Using in DjangoInclude logstash formatter in your settings file:LOGGING={'formatters':{'logstash':{'()':'logstash_formatter.LogstashFormatter','format':("asctime","levelname","name","lineno","message","pathname","module","funcName","process",),'datefmt':'%Y-%m-%dT%H:%M:%S','rename':{'asctime':'@timestamp',},'version':'1'},}}
Package: logs-to-ocsf
logs_to_ocsfInstallpipinstalllogs_to_ocsfHow to useAdd sample data and run.fromlogs_to_ocsf.convert_messagesimport*c=Convert()test_dir="data/sample_logs"test_files=["messages.txt","cron.txt","secure.txt"]fora_fileintest_files:a_json=c.log_to_ocsf(test_dir+"/"+a_file)print(f"\nJSON in OCSF Schema from file{a_file}has this many records:")print("-----------------------------------------------------------")print(len(a_json))print("-----------------------------------------------------------")JSON in OCSF Schema from file messages.txt has this many records:
-----------------------------------------------------------
37
-----------------------------------------------------------

JSON in OCSF Schema from file cron.txt has this many records:
-----------------------------------------------------------
40
-----------------------------------------------------------

JSON in OCSF Schema from file secure.txt has this many records:
-----------------------------------------------------------
40
-----------------------------------------------------------
Package: logstop
Logstop.py:fire: Keep personal data out of your logslogger.info('Hi test@example.org!')# Hi **********!By default, scrubs:email addressesphone numberscredit card numbersSocial Security numbers (SSNs)passwords in URLsWorks even when data is URL-encoded with plus encodingInstallationRun:pipinstalllogstopAnd add it to your logger:fromlogstopimportLogstopFilterlogger.addFilter(LogstopFilter())OptionsTo scrub IP addresses (IPv4), use:LogstopFilter(ip=True)To scrub MAC addresses, use:LogstopFilter(mac=True)Disable default rules with:LogstopFilter(email=False,phone=False,credit_card=False,ssn=False,url_password=False)NotesTo scrub existing log files, check outscrubadubTo scan for unencrypted personal data in your database, check outpdscanHistoryView thechangelogContributingEveryone is encouraged to help improve this project. Here are a few ways you can help:Report bugsFix bugs andsubmit pull requestsWrite, clarify, or fix documentationSuggest or add new featuresTo get started with development:gitclonehttps://github.com/ankane/logstop.py.gitcdlogstop.py
pipinstall-rrequirements.txt
pytest
Package: log-storage
Log StorageA django installable app that can be used as log handler to store messages
with a database record and optional file storage.How to installAdd this package as requirement to your django app requirements.txt# requirements.txt
log_storageAdd this app to yourINSTALLED_APPSin yoursettings.py# settings.py
INSTALLED_APPS = (
    'log_storage',
)DevelopmentHow to installClone the projectgit clone git@github.com:Virtualstock/log_storage.gitInstall itmake setupHow to run testsmake testHow to releaseMake sure you bump the package version onsetup.py.make releaseLicenseLicensed underMIT license. Viewlicense.
Package: logstyle
logstyleGetting startedLogstyle was developed for simplified usage like in below examples:Create your own CustomLog instancecustom_logger = CustomLog("MyLogger")Now you can try it using different  styles and colors like below:Box style logs:custom_logger.box_log("This is a box log example.", CustomLog.COLOR_GREEN)Title style logs:custom_logger.title_log("Title Log", "This is a title log example.", CustomLog.COLOR_BLUE)JSON style logs:custom_logger.json_log("JSON Log", "This is a JSON log example.", {"key": "value"}, CustomLog.COLOR_AMBER)YAML style logs:custom_logger.yaml_log("YAML Log", "This is a YAML log example.", {"key": "value"}, CustomLog.COLOR_RED)
Package: logtable
logtableMonitor and Compare Logs on Terminal.InstallationpipinstalllogtableUsage%cdexamples
%logtable*Configfile:.logtable*Logdirectory:logs
╒════╤════════════════════════╤═════════╤═════════════╤════════════╤══════════╤═══════════╤════════════╤═══════╤══════════════╤═══════════════╕
││log_│epoch│iteration│elapsed_│class_│githash│hostname│lr│main/│validation/│
││dir│││time│ids││││loss(min)│main/│
│││││││││││loss(min)│
╞════╪════════════════════════╪═════════╪═════════════╪════════════╪══════════╪═══════════╪════════════╪═══════╪══════════════╪═══════════════╡
│0│20190310_093252.724597│1│1740│1:47:02│[1]│b48ce48│computer1│0.001│0.00879│0.18│
││││││││││(1,1580)│(0,880)│
├────┼────────────────────────┼─────────┼─────────────┼────────────┼──────────┼───────────┼────────────┼───────┼──────────────┼───────────────┤
│1│20190310_093829.691289│1│1720│1:45:37│[1]│f766b97│computer2│0.001│0.0123│0.187│
││││││││││(1,1620)│(0,440)│
╘════╧════════════════════════╧═════════╧═════════════╧════════════╧══════════╧═══════════╧════════════╧═══════╧══════════════╧═══════════════╛# examples/.logtableexclude:-out-updated_at-timestamp-loglevel-gpu-seed-'lr.*'-'.*main/loss.*\(max\)'-'.*loss_.*'params_basename:args.jsonlog_basename:log.json# below configs are the same as the defaultlog_dir:logsinclude:[]significant_figures:3index:nullmulti_column:Falsekey_epoch:epochkey_iteration:iterationkey_elapsed_time:elapsed_time
Package: logtailtool
LogTailToolLogTailTool tail a log file in python.Usage (python code)fromlogtailtoolimportLogTailToollines=LogTailTool("name.log")forlineinlines:print(line)Usage (command line)usage: logtailtool.py [-h] -f FILENAME [-p PATTERN] [-b] [-i INTERVAL]

LogTailTool

optional arguments:
  -h, --help            show this help message and exit
  -f FILENAME, --file FILENAME
                        Log file path (required)
  -p PATTERN, --pattern PATTERN
                        Pattern to extract (default: None)
  -b, --read-from-begin
                        Read from the begin of the file (default: false)
  -i INTERVAL, --interval INTERVAL
                        Interval time (default: 1)
Package: logtale
LogTale - A python logging frameworkA simple, easy to use logging framework that builds on top of the built-in logger module.InstallationpipinstalllogtaleUsageSimple Usage:importlogtale.logtaleastaledefmain():logtale=tale.LogTale("example","./example.toml")logger=logtale.logger(__name__)logger.debug("test - debug")logger.info("test - info")logger.warning("test - warning")logger.error("test - error")logger.critical("test - critical")if__name__=="__main__":main()Prepend/Postpend text to the log message:logger.addFilter(filter.LogAppendFilter(prepend_text="ExamplePrepend"))logger.addFilter(filter.LogAppendFilter(postpend_text="ExamplePostpend"))Configuration FileCreate a configuration file for logging settings using thetemplates/config_template.tomlfile as a template.[output.colors]# Set the colors associated with each debug level.# Colors are only applied for logs printed to the console.# The logging color feature can be enabled/disabled in '[output.console]' section.DEBUG='\033[1;36m'INFO='\033[1;38m'WARNING='\033[1;33m'ERROR='\033[1;35m'CRITICAL='\033[1;31m'[output.file]enable=true# enable/disable logging to a .log file (default=true)level="DEBUG"# the base log level for logging to a file (default=DEBUG)path="logs/"# the directory to create log files in (default=logs/)format="(%(asctime)s)[%(name)s][%(levelname)s]::%(message)s"# the log message format to use for file loggingname="example.log"# the naming scheme of the log file, by default it's '<project_name>_<version>_<timestamp>.log'[output.console]enable=true# enable/disable logging to the console (default=true)level="DEBUG"# the base log level for logging to the console (default=DEBUG)format="[%(levelname)s]::%(message)s"# the log message format to use for console logginguse_colors=true# enable/disable the use of colors for log levels. colors can be customized in '[output.colors]' section.ExamplesCheck theexamplesdirectory for example scripts and configuration files.# ./examples/example.pyimportlogtale.logtaleastaleimportlogtale.filterasfilterdefmain():logtale=tale.LogTale("example","./example.toml")logger=logtale.logger(__name__)logger.addFilter(filter.LogAppendFilter(prepend_text="ExamplePrepend"))logger.debug("test - debug")logger.info("test - info")logger.warning("test - warning")logger.error("test - error")logger.critical("test - critical")if__name__=="__main__":main()Example log console outputExample log file output
Package: logthing
Some useful tools for logging.  First of is the SRotHandler, a rotating file
handler with a size limit.  Second is a series of utilities for configuring
your loggingInstallation$pipinstalllogthingSupportslogthing has been tested with Python 3.6, 3.7, and 3.8For Python 2 compatible source use version 0.10.1Docs & SourceDocs:http://logthing.readthedocs.io/en/latest/Source:https://github.com/cltrudeau/logthing
Package: logthis
logthislogthis is a singleton, two-level, colorful, thread-safe, knob-free, logging library for in-house software.singleton: There is no object to create. There are only two logging functions,say()anderr().two-level: There is only the information level and the error level. Nothing else. We found it way too mentally
involving to have more than two logging levels. We want to avoid unnecessary cognitive load at every message (“Is this
a warning? Or an information? Or debugging information?”). We don’t think that is important. Either there is a problem
and needs to be resolved (so useerr()), or everything is fine and no action is required by the operator (so usesay()).colorful: The prefix of a message is colored indicating the log level. This makes reading the logs easier on the
eyes and helps direct the attention. Colors are included even when the logging is redirected to a file. We inspect
our logs with Unix utilities (catand the ilk) and find it cool to preserve colors even when we inspect files such
as supervisord logs.thread-safe: We use a global lock so that multi-threaded logging is not garbled. STDOUT and STDERR are flushed on
every logging.knob-free: There are no options or targets/sinks/streams to set. The information is written to STDOUT and the
errors are written to STDERR. We found it daunting to learn and deal with all the special knobs in libraries such as
Pythonlogging.in-house software: logthis is meant to be used for the software developed and operated in-house. Its output will
be examined by people who are familiar with the code and would like to inspect it on problems. We include the name of
the script and the line number in the messages as well as time in UTC so that it is easier to trace bugs and see
where in the code the logging comes from.If you are developing a library or a program for wider audience, then logthis is probably not for you.Usageimportlogthis# inform the userlogthis.say("Hello!")# alert the user that there is an errorlogthis.err("Something bad happened".)The output is:InstallationCreate a virtual environment:python3-mvenvvenv3Activate it:sourcevenv3/bin/activateInstall logthis with pip:pip3installlogthisDevelopmentCheck out the repository.In the repository root, create the virtual environment:python3-mvenvvenv3Activate the virtual environment:sourcevenv3/bin/activateInstall the development dependencies:pip3install-e.[dev]We use tox for testing and packaging the distribution. Assuming that the virtual environment has been activated and
the development dependencies have been installed, run:toxWe also provide a set of pre-commit checks that lint and check code for formatting. Run them locally from an activated
virtual environment with development dependencies:./precommit.pyThe pre-commit script can also automatically format the code:./precommit.py--overwriteVersioningWe followSemantic Versioning. The version X.Y.Z indicates:X is the major version (backward-incompatible),Y is the minor version (backward-compatible), andZ is the patch version (backward-compatible bug fix).
Package: logthon
LogthonA simple logger for PythonGetting StartedThese instructions will install the logger to your machine.PrerequisitesPython3PIP3Installationpip3 install logthonUsageImportClassic importfrom logthon.logthon import Logthon as Logger

Logthon = Logger()Save output on filefrom logthon import Logthon as Logger

Logthon = Logger(save_log=True)Save output on custom filefrom logthon import Logthon as Logger

Logthon = Logger(save_log=True, filename='/path/to/file')Define module name on log formatfrom logthon import Logthon as Logger

Logthon = Logger(module_name=__name__)  # or Logger(module_name='my.module.path')

# Example
Logthon.info('This is an info log')
# [2012-01-14 00:00:00] my.module.path - INFO: This is an info logInfo levelLogthon.info('This is an info log')
# [2012-01-14 00:00:00] INFO: This is an info logWarn levelLogthon.warn('This is a warn log')
# [2012-01-14 00:00:00] WARN: This is a warn logError levelLogthon.error('This is an error log')
# [2012-01-14 00:00:00] ERROR: This is an error logSuccess levelLogthon.success('This is a success log')
# [2012-01-14 00:00:00] SUCCESS: This is a success logCritical levelLogthon.critical('This is a critical log')
# [2012-01-14 00:00:00] CRITICAL: This is a critical logDebug levelLogthon.debug('This is a debug log')
# [2012-01-14 00:00:00] DEBUG: This is a debug logLog and exit program with error codeYou can log a message and exit the program defining optionally and error code.# Default log level is CRITICAL
# Default error code is 1
Logthon.log_and_exit_with_code('This is a message')Disable debug level log printinglogthon = Logthon(debug=False)
logthon.debug('This is a debug log')

# Nothing printedIf you want to define log level, you need to import level you wantfrom logthon.logthon import INFO_LEVEL
from logthon.logthon import WARN_LEVEL
from logthon.logthon import ERRO_LEVEL
from logthon.logthon import SUCC_LEVEL
from logthon.logthon import CRITICAL_LEVEL
from logthon.logthon import DEBUG_LEVEL

# e.g. we want to log with error level and exit
Logthon.log_and_exit_with_code('Log message', level=ERRO_LEVEL)

# e.g. we want to log with debug level and exit with code 5
Logthon.log_and_exit_with_code('Log message', level=DEBUG_LEVEL, error_code=5)Log a prettified dictionaryYou can log a message with a dictionary pretty printed# Default log level is DEBUG 
Logthon.pretty_print('This is a message', {'key': 'word'})If you want to define log level, you need to import level you wantfrom logthon.logthon import INFO_LEVEL
from logthon.logthon import WARN_LEVEL
from logthon.logthon import ERRO_LEVEL
from logthon.logthon import SUCC_LEVEL
from logthon.logthon import CRITICAL_LEVEL
from logthon.logthon import DEBUG_LEVEL

# e.g. we want to log with info level
Logthon.pretty_print('This is a message', {'key': 'word'}, level=INFO_LEVEL)ContributingPlease readCONTRIBUTING.mdfor details on code of conduct, and the process for submitting pull requests.LicenseThis project is licensed under the GNU General Public License v3 (GPLv3), readLICENSEfor detailsAuthorGiuseppe "mastrobirraio" Matranga-Initial work-Github
Package: log-throttling
log-throttlingA light-weight python log throttling utilityimporttimeimportloggingimportlog_throttlinglogger=logging.getLogger(__name__)start=time.monotonic()whiletime.monotonic()-start<10:log_throttling.by_time(logger,interval=1).info("This line will be logged once every second.")time.sleep(0.01)foriinrange(100):log_throttling.by_count(logger,once_every=10).info("This line will only log values that are multiples of 10:%s",i)
Package: logtime
logtimeParse the date and time from log files and print an ASCII graph of the
occurrences.  Time can be grouped by minute, hour or day.matth-at-mtingers.comUsageUsage:
     logtime <grouping> <timepattern> <timespec> < FILENAME

Example Usage:
   Apache logs grouped by hour
     grep FOO /var/log/apache2/access.log | \
         logtime H '\[(.+?) ' '%d/%b/%Y:%H:%M:%S'

     logtime H '\[(.+?) ' '%d/%b/%Y:%H:%M:%S' < /path/file

Grouping spec:
     H - hour
     M - Minute
     S - Second
     y - year
     m - month
     d - day

Timespecs:
     Directive   Meaning
     %a  Locale's abbreviated weekday name.
     %A  Locale's full weekday name.
     %b  Locale's abbreviated month name.
     %B  Locale's full month name.
     %c  Locale's appropriate date and time representation
     %d  Day of the month as a decimal number [01,31]
     %H  Hour (24-hour clock) as a decimal number [00,23]
     %I  Hour (12-hour clock) as a decimal number [01,12]
     %j  Day of the year as a decimal number [001,366]
     %m  Month as a decimal number [01,12]
     %M  Minute as a decimal number [00,59].
     %p  Locale's equivalent of either AM or PM.
     %S  Second as a decimal number [00,61].
     %U  Week number of the year (Sunday as the first day of the
         week) as a decimal number [00,53].
     %w  Weekday as a decimal number [0(Sunday),6]
     %W  Week number of the year (Monday as the first day of the
         week) as a decimal number [00,53].
     %x  Locale's appropriate date representation
     %X  Locale's appropriate time representation
     %y  Year without century as a decimal number [00,99]
     %Y  Year with century as a decimal number
     %Z  Time zone name (no characters if no time zone exists)
     %%  A literal '%' character.

Important notes:
     Regex is Python regex re.findall()
     Timepsec is Python's time module
Package: logto
Logto Python SDKPrerequisitesPython 3.8 or higherALogto Cloudaccount or a self-hosted LogtoA Logto traditional web application createdIf you don't have the Logto application created, please follow the⚡ Get startedguide to create one.Installationpipinstalllogto# or `poetry add logto` or whatever you useTutorialSeetutorialfor a quick start.API referenceSeeAPI referencefor more details.Sample codeSeesamplesdirectory for example usages.ResourcesLogto websiteLogto documentationJoin Discord
Package: logtogss
logtogssDescriptionThis is a small tool to log data to a Google Spreadsheet. Google Spreadsheets are very handy for manipulating data, but it’s less easy to insert data programmatically. The tool depends on the thepython gdata clientpackage, which should be automatically installed as a reqirement when installing thelogtogssfrom setup package.InstallationTo install, download the setuppackageand install using pip:$ pip install logtogss-0.1.tar.gzAlternatively, let pip download from pypi and install:$ pip install logtogssUsageThe basic Usage is like this:$ logtogss --name "Spreadsheet Name" --sheet="Sheet Name" col1:val1 col2:val2You can insert multiple rows at once by piping on stdin, whilst specifying just the column names, like this:$ cat data
val1 val2
val3 val4
$ logtogss --name 'Spreadsheet Name' --sheet="Sheet Name" col1 col2 < dataIf you need to know what the valid column names are, leave off the data from the command line, like this:$ logtogss --name "Spreadsheet Name" --sheet="Sheet Name"
col1
col2You can also use the--listoption wth or with out the--nameand--sheetoptions to see a list of the columns in the spreadsheet(s).Note thatlogtogssuses OAuth, so the first time that you use it, you will be prompted with an URL to visit in order to allowlogtogssaccess to your google docs.Runlogtogsswith--helpoption to see the advanced options and their usage.CreditsHappygiraffecreator of the initial version.modern-package-templateused to bootstrap the project setup.BuildoutNewsWhat’s new 0.1Release date: *UNRELEASEDCreating a setup package