Package: haufe.requestmonitoring
Table of contentsIntroductionRequirementsFeaturesFine resolution request loggingFieldsSuccess request loggingMonitoring long running requestsDump trace on pdbInstallationAuthorLicenseChangelog0.6.0 (2021-05-12)0.5.1 (2018-04-03)0.5.0 (2016-09-29)0.4.0 (2013-09-20)0.3.0 (2012-10-16)0.2.3 - (2009/08/11)0.2.2 - (2009/07/20)0.2.1 - (2009/05/28)0.2.0 - (2009/05/12)Introductionhaufe.requestmonitoringimplements a detailed request logging functionality
on top of the publication events as introduced with Zope 2.12.RequirementsZope 2.12.0b2 or higherCurrently tested on Zope 2.13.21You can use this with older Zope releases (2.10.x) but you must also includeZPublisherEventsBackport.FeaturesFine resolution request loggingUsed as base forztopandzanalyse, i.e. helps to determine the Zope load,
detect long running requests and to analyse the causes of restarts.The implementation in this module registers subscribers forIPubStartandIPubSuccess/IPubFailure.  For each of these events, a log entry of the form:timestamp status request_time type request_id request_infois written.Fieldstimestampis the current time in the format%y%m%dT%H%M%S.statusis0forIPubStartevents,390for requests that will
be retried and the result ofIStatusapplied to the response otherwise.request_timeis0forIPubStartevents. Otherwise, it will be
the request time in seconds.typeis+forIPubStartand-otherwise.request_idis the (process) unique request id.request_infoisIInfoapplied to the request.In addition, a log entry withrequest_info == restartedis written when this
logging is activated. Apart fromrequest_infoandtimestampall other
fields are0. It indicates (obviously) that the server has been restarted.
Following requests get request ids starting with1.To activate this logging, bothtimelogging.zcmlmust be activated (on by
default) and aproduct-configsection with nametimeloggingmust be
defined containing the keyfilebase.  It specifies the basename of the
logfile;.<date>will be appended to this base.  Then,ITicket,IInfoadapters must be defined (e.g.  the one frominfo).  AnIStatusadapter may be defined for response.Example:<product-config timelogging>
filebase /path/to/request-logs/instance-foo
</product-config>Success request loggingThis logging writes two files<base>_good.<date>and<base>_bad.<date>.
For each request, a character is written to either the good or the bad logfile,
depending on whether the request was successful or unsuccessful. This means,
that only the file size matters for these logfiles.Usually, response codes >= 500 are considered as unsuccessful requests.  You
can register anISuccessFulladapter, when you need a different
classification.To activate this logging, bothsuccesslogging.zcmlmust be activated (on by
default) and aproduct-configsection with namesuccessloggingmust be
defined containing the keyfilebase.  It specifies the basename of the
logfiles (represented as<base>above).Example:<product-config successlogging>
filebase /path/to/request-logs/successful-foo
</product-config>Monitoring long running requestshaufe.requestmonitoringallows you to monitor long-running request. The
following configuration within yourzope.confconfiguration file will
install the DumpTracer and check after theperiodtime passed for requests
running longer thantime.To activate this logging, bothmonitor.zcmlmust be activated (off by
default) and the requestmonitor configuration section must be present:zope-conf-additional =
    %import haufe.requestmonitoring
    <requestmonitor requestmonitor>
        # default is 1m
        period 10s
        # default is 1
        verbosity 2
        <monitorhandler dumper>
            factory haufe.requestmonitoring.DumpTraceback.factory
            # 0 --> no repetition
            repeat -1
            time 10s
        </monitorhandler>
    </requestmonitor>A typical dump trace looks like this (it shows you the URL and the current
stacktrace):2009-08-11 14:29:09 INFO Zope Ready to handle requests
2009-08-11 14:29:09 INFO RequestMonitor started
2009-08-11 14:29:14 INFO RequestMonitor monitoring 1 requests
2009-08-11 14:29:19 INFO RequestMonitor monitoring 1 requests
2009-08-11 14:29:24 INFO RequestMonitor monitoring 1 requests
2009-08-11 14:29:24 WARNING RequestMonitor.DumpTrace Long running request
Request 1 "/foo" running in thread -497947728 since 14.9961140156s
Python call stack (innermost first)
Module /home/junga/sandboxes/review/parts/instance/Extensions/foo.py, line 4, in foo
Module Products.ExternalMethod.ExternalMethod, line 231, in __call__
- __traceback_info__: ((), {}, None)
Module ZPublisher.Publish, line 46, in call_object
Module ZPublisher.mapply, line 88, in mapply
Module ZPublisher.Publish, line 126, in publish
Module ZPublisher.Publish, line 225, in publish_module_standard
Module ZPublisher.Publish, line 424, in publish_module
Module Products.ZopeProfiler.ZopeProfiler, line 353, in _profilePublishModule
Module Products.ZopeProfiler.MonkeyPatcher, line 35, in __call__
Module ZServer.PubCore.ZServerPublisher, line 28, in __init__The log line “RequestMonitor monitoring X requests” simply says that a request
is under monitor and sometimes you get useless noise in the log file.You can play with theverbosityoption: put the value to0for disable
the log line.
Default value (1) will display the log line every time one or more requests
are under monitor.
A value of2is more verbose, displaying also info about requests URLs.Dump trace on pdbTraceback dump can became quickly a nightmare if you put a Python debug line on your source code
and then you want to test it running Zope.In that case you can disable traceback dump when you are executing the debugger. Simply add theDISABLE_HAUFE_MONITORING_ON_PDBenvironment variable:environment-vars =
    ...
    DISABLE_HAUFE_MONITORING_ON_PDB TrueInstallationAddhaufe.requestmonitoringto botheggsandzcmloption of
your buildout.cfg file.Authororiginal author: Dieter Maurercurrent maintainer: Andreas Jung,info@zopyx.comLicensehaufe.requestmonitoringis published under the Zope Public License V 2.1
(ZPL). See LICENSE.txt.Changelog0.6.0 (2021-05-12)Support Zope 4 / Python 3.
[tschorr]0.5.1 (2018-04-03)Fix logging format.
Refs: #9
[mamico]0.5.0 (2016-09-29)For all logging output, use unicode and stringformatinstead of string substitution to avoid possible encoding errors.
Refs: #5.
[thet]Allow themonitorhandlerdefine it’s zLOG loglevel.
This makes possible to define theERRORlevel instead ofWARNINGfor long running requests and get notified by a tool like Sentry, when it’s configured to noticeERRORlevel logs.
[thet]Remove Logger.py, which is a simpler version of the “Long Request Logger” in DumpTraceback.py and isn’t used nor documented.
[thet]Movedocs/HISTORY.txttoCHANGES.rst.
[thet]PEP8 compatibility0.4.0 (2013-09-20)Added retro-compatibility with old versions of Zope without bein forced to manually patch Zope.
[giacomos]You can now add aDISABLE_HAUFE_MONITORING_ON_PDBenvvar to stop dumping traceback when on Pythonpdb.
[keul]0.3.0 (2012-10-16)Do not use deprecatedthreadframedependency anymore on recent Python versions.
[keul]Fixed egg dependencies for Zope 2.13.
[keul]Added theverbosityconfiguration option for the logger.
[keul]0.2.3 - (2009/08/11)Updated documentation.0.2.2 - (2009/07/20)Minor cleanup.Minor documentation cleanup.0.2.1 - (2009/05/28)Configure ‘successlogging’ by default.Slightly updated documentation.0.2.0 - (2009/05/12)Initial release.