Package: pyramid-session-redis
Overviewpyramid_session_redisis a mature, stable and actively maintained Server-Side
Sessions plugin for the Pyramid web framework.Originally, this library was a fork ofpyramid_redis_sessions, focused on
improvements and API changes designed for high performance (particularly with
servers under load), and a slightly different API designed for developer
convenience.This package was planning to follow a multi-version release process, however
that plan was abandoned in favor of ensuring backards compatibility.Themainand1.6branches are designed to support both:Python 2Python 3andPyramid 1.xPyramid 2.0Prior BranchesThe 1.5.x branch is in maintenance mode. No updates are expected, as 1.6 is
backwards compatible.The 1.4.x branch is EOL. It led to the stable 1.5.0 API release.The 1.2.x branch is EOL as of 1.2.2. Previous plans were to support a final
1.3.0 release.Prior Branch DetailsThe 1.2.x branch and earlier are largely a "drop-in-replacement" compatible with
Eric Rasmussen'spyramid_redis_sessionsas-is.  If you are migrating from that
project and do not want to upgrade code, you should pin your install of this
library topyramid_session_redis<=1.3.0orpyramid_session_redis<1.3Please be aware that you will be limited to using outdated versions of Pyramid
1.x if using the 1.2.x branch.Starting with the 1.4.x branch, several design changes were made and this library
is not a drop-in replacement,HOWEVERupgrading will only require minimal
editing of your code: some editing for migration; some kwargs have changed;
the structure of the package changed (imports); and advanced users who leverage
the internal systems may need to upgrade. It should not take more than 5 minutes
to convert.  The package is still a plug-and-play Pyramid ISessions interface, so
there are very small changes.Prior Branch IncompatibilitiesIMPORTANT: The internal payload structure changed in the 1.4 branch, and is no
longer compatible with sessions created under 1.2 - they will be invalid.PRs that can handle a graceful fallback are welcome.The 1.2 format guaranteed internal sessions to be in this format:{'managed_dict': {},
 'created': INT,
 'timeout': INT,
 }The 1.4 version streamlines the keys for a lighter footprint and generally
looks like this:{'m': {},  # managed_dict
 'c': INT,  # created
 'v': SESSION_API_VERSION,  # api version INT
 # the following are optional and not guaranteed to be in a session
 't': INT,  # timeout
 'x': INT,  # expiry
 }Key Takeaways:Keys were shortened to 1 letterAn API Version is now used, to handle graceful changes in the futureThe Timeout is no longer guaranteed to be present.
The library now supports the entire Timeout to be handled in RedisAn "Expiry Timeout" may also existKey ConceptsTimeout vs Expires/MaxAgeThis package allows you to set bothTimeoutandExpiryvalues.  The two
concepts are closely related, but very different.ATimeoutis an internal value and a moving target. It specifies how long a
Session should last before it times out due to inactivity. The timeout is
constantly refreshed when the session is accessed.For example, consider these settings:timeout = 1800
timeout_trigger = 900Thetimeoutsetting is specifying the Session should remain active for 1800
seconds since it's last activity.  Thetimeout_triggersetting is an
optimization that defers modifying the Session's timeout on READ operations until
at least 900 seconds have passed since the last modification. Modifications will
always occur on WRITE operations.  Depending on your configuration, the timeout
may be stored in Redis, as part of the Python Payload, or both.To illustrate how the above settings work, consider a timeline in which a User
visits a website and then visits it again 2000 seconds later:Visit 1 | 0s    | Session A (new)
Visit N | 2000s | Session B (new)When the User makes the last visit, the Session would have timed out, because it
exceeded the 1800 second mark in the timeout.Now assume the User makes 3 visits in this timeline, and a second visit happens
at the 899 mark:Visit 1 | 0s    | Session A (new)
Visit 2 | 899s  | Session A
Visit N | 2000s | Session B (new)When the User makes the last visit, the Session would still
have timed out, because the second visit did not meet the timeout_trigger
threshold.But, if the second visit happens just 2 seconds later, at the 901 mark, the
timeout_trigger is reached and the session's timeout will be extended:Visit 1 | 0s    | Session A (new)
Visit 2 | 901s  | Session A (update timeout trigger)
Visit N | 2000s | Session A (update timeout trigger)When the User makes the last visit, it would still happen within the context of
the original Session and the timeout will be extended even further.ExpiresandMax-Ageare generally values that related to Cookie settings.Expiresis created on a Cookie as either a specified Date, or a null value; if
the value is null (explicit or unspecified), the Cookie will be set as a
"Session" cookie and only last for the duration of the web browser application
being open.Max-Ageis created on a Cookie as the number of seconds until the Cookie
expires.  A zero or negative value will expire (delete) the cookie immediately.
If both Expires and Max-Age are set, Max-Age has precedence.The Confusion?If Redis stores our Timeout info, it stores it in an "expiry" value viaSETEXor similar calls.Key Differences From pyramid_redis_sessionsDepending on your needs, this package is probably more desirable than the
original project. This package was designed to significantly cut down on the
communication between Redis and Pyramid. Some options are offered to minimize
the size of Redis payloads as well.This package contains additional hooks and features to aid developers who are
using Redis-based sessions in high-traffic situations.This package does not recommend any "best deployment" strategies, but supports a
wide range of strategies to implement the "best deployment" under any given
circumstance.Through 1.2.xThe original package communicates with Redis on most attribute accesses and
writes within a given request. The traffic can be burdensome in some
implementations.pyramid_session_rediswill queue a singlepersistorrefreshtask per-request using Pyramid'sadd_finished_callbackhook.The original version usedEXISTSto check if a Session existed or not, then
proceeded toGETorSETa new Session.pyramid_session_rediswill
immediately attempt aGET, and willSETa new Session on failure.
This approach eliminates a call.Separate calls toSETandEXPIREwere replaced with a singleSETEX.A flag can be set to enable a LRU Cache (least recently used) mode. No expiry
data will be sent to Redis, allowing the Redis server to handle the LRU logic
itself.The active Session is decoupled from therequest.sessionattributs; this
allows for the Session to be set up on alternate attributes and supports
integration withpyramid_session_multi.The original library does not detect changes in nested dictionaries. This
package useshashlib.md5to fingerprint the serialized value on READ; if no
changes were detected, a failsafe willSerialize+md5the data to decide if a
WRITE should occur. This behavior can be disabled by settingdetect_changestoFalse.The original library raises a fatal error if a Session can not be
deserialized. By passing indeserialized_fails_newto the constructor,
you can create a new Session on deserialization errors.Support for disabling Sessions on CDN generated content viafunc_check_response_allow_cookiesThanks to @ github/hongyuan1306, Token Generation has been consolidated to use
Python3's stdlib (or reimplemented if not available). Tokens are also 32,
not 20, characters.Redis is supported in a LRU mode (seehttp://redis.io/topics/lru-cache) by
setting the optionset_redis_ttltoFalse(by default, it isTrue).
This will eliminate calls toEXPIREand will useSETinstead ofSETEX.In the 1.2.x branch, the created time can be set to an integer viause_int_time=True. This will cast thecreatedtime viaint(math.ceil(time.time())). This reduces a payload by several bits.Other Updates 1.4.x+Onlyint()time is supported; Fractional time wastes bytes.Sessions now have version control to support future upgrades via a "version"vkey.The format of the internal payload was rewritten, the encoded payload now
uses 1-letter keys instead of words. This should offset the addition of an
expires timestamp and version id.There was no logic for Python timeout control (whoops!) this has been fixed.
An "expires"xkey now tracks the expiration.Added atimeout_triggeroption.  This will defer expiry data updates to
lower usage on Redis.  This is explained below in more detail.In high load situations, Redis can have performance and storage issues because
in the original package Session IDs are created on every request (such as a
getting spidered by a botnet that does not respect Sessions). In this package,
a 'lazycreate' method is used: a session_id/cookie will not be generated
unless a session is needed in the callback routine. In order to generate
session_id/cookie beforehand, one can use theRedisSession.ensure_idfunction.  To safely check if a session id exists, one can use theRedisSession.session_id_safecheckmethod as well.Addedfunc_invalid_loggerto the constructor. This can be used to log
invalid Sessions. It is incredibly useful when integrated with a statsd
system. (see below)Addedset_redis_ttl_readheavyto the factory and session constructors.
This option will optimize Sessions which have Redis-maintained TTLs by
executing a GET+EXPIRE together within a pipeline.InstallationInstall via pypi:pip install pyramid_session_redisConfigurationConfigurepyramid_session_redisvia your Paste config file or however you
prefer to configure Pyramid. Onlyredis.sessions.secretis required.
All other settings are optional.For complete documentation on theRedisSessionFactorythat uses these
settings, see :doc:api. Otherwise, keep reading for the quick list:# session settings
redis.sessions.secret = your_cookie_signing_secret
redis.sessions.timeout = 1200

# session cookie settings
redis.sessions.cookie_name = session
redis.sessions.cookie_max_age = max_age_in_seconds
redis.sessions.cookie_path = /
redis.sessions.cookie_domain =
redis.sessions.cookie_secure = False
redis.sessions.cookie_httponly = False
redis.sessions.cookie_on_exception = True

# you can supply a redis connection string as a URL
redis.sessions.url = redis://username:password@localhost:6379/0

# or as individual settings (note: the URL gets preference if you do both)
redis.sessions.host = localhost
redis.sessions.port = 6379
redis.sessions.db = 0
redis.sessions.password = None

# additional options can be supplied to redis-py's StrictRedis
redis.sessions.socket_timeout =
redis.sessions.connection_pool =
redis.sessions.charset = utf-8
redis.sessions.errors = strict
redis.sessions.unix_socket_path =

# in the advanced section we'll cover how to instantiate your own client
redis.sessions.client_callable = my.dotted.python.callable

# along with defining your own serialize and deserialize methods
redis.sessions.serialize = cPickle.dumps
redis.sessions.deserialize = cPickle.loads

# you can specify a prefix to be used with session keys in redis
redis.sessions.prefix = mycoolprefix

# or you can supply your own UID generator callable for session keys
redis.sessions.id_generator = niftyuidInitializationLastly, you need to tell Pyramid to usepyramid_session_redisas your
Session factory. The preferred way is adding it withconfig.include,
like this:def main(global_config, **settings):
    config = Configurator(settings=settings)
    config.include('pyramid_session_redis')Alternately, instead of using the Configurator's include method, you can
activate Pyramid by changing your application's ".ini" file, use the following
line:pyramid.includes = pyramid_session_redisThe above method is recommended because it's simpler, idiomatic, and still fully
configurable. It even has the added benefit of automatically resolving dotted
python paths used in the advanced options.However, you can also explicitly pass a settings dict to thesession_factory_from_settingsfunction. This can be helpful if you configure
or modify your settings in code:from pyramid_session_redis import session_factory_from_settings

def main(global_config, **settings):
    config = Configurator(settings=settings)
    session_factory = pyramid_session_redis(settings)
    config.set_session_factory(session_factory)Timeout NotesIfset_redis_ttlis False, it does not imply there is no timeout at all --
only that Redis will not be sent timeout data viaSETEXorEXPIRE.
Timeout data will still be stored in Python.If Redis is functioning as an LRU Cache, abandoned sessions will never be seen
by Python, but will eventually be cleared out to make room for new sessions by
the inherent Redis LRU logic.Timeout data stored in Python is relatively small when compared to the timeout
data stored in Redis.If you want to NEVER have sessions timeout, set the initialtimeoutto "0" or "None".Setting atimeout_triggerwill require Python to track the expiry.Enablingset_redis_ttl_readheavyrequires atimeoutandset_redis_ttl;
it also requires not enablingtimeout_triggerorpython_expires.Timeout ExamplesTimeout in Python, with Redis TTL viaSETEX/EXPIRE:timeout = 60Timeout in Python, no Redis TTL (onlySETused)timeout = 60
assume_redis_ttl = TrueNo Timeout in Python, no Redis TTL (onlySETused)timeout = 0  # or None
assume_redis_ttl = TrueWarning: Session Locking and Race ConditionsThis package does not incorporate any sort of locking for session updating or
handling of race conditions.This should not be a problem for the vast majority of users, and is a feature
not present on any known Pyramid Session providers.For more information, please see:https://github.com/jvanasco/pyramid_session_redis/issues/9https://github.com/Pylons/pyramid/issues/3041Feature - Timeout TriggersA timeout trigger can be used to limit the amount of updates/writes.
It may be more beneficial to your usage pattern.For the sake of clarity, I'll use an oversimplification that Redis essentially
has two different internal data stores that are used independently: one for a
Key's payload and another for the Key's expiry.In the 'classic' behavior (project this was forked from): every time you access
an existing session, the GET is followed by sending Redis a new EXPIRE time;
essentially every "read" has a corresponding "write" for the Redis-tracked
expiry record.In order to minimize the writes via SETEX, I introduced the timeout trigger.
The trigger works by storing some timeout information in the Redis data payload,
and using that information to determine when to send a write. Instead of having
a GET+EXPIRE for every read, we only have a single GET and eliminate the write
caused by the EXPIRE. This has a maintenance cost though - once we hit the
timeout trigger, instead of just the GET we need to update the internal timeout
payload and issue a SET.Going back to your situation: when the user stops activity at 40 minutes in, if
the timeout trigger is enabled then there has never been an update to the
internal payload or Redis about the user activity since the session was first
created. The purpose of the trigger is to defer that "write" operation.In order to make a session valid for "reading" for around an hour, you should
do something like:a two-hour session with a 10 minute trigger, ora one-hour session with a 50 minute triggerYou can also disable the functionality by setting the trigger to 0. Up to a few
thousand daily users, you shouldn't have any issues with the overhead of the
"classic" mode. When you hit 10k users and/or start to have clustered web
servers communicating with dedicated Redis instances, setting a new EXPIRE after
every read operation becomes something you want to avoid.Scenario 1 - Classic RedisIn the typical "classic" Redis usage pattern, the session usage is refreshed via
anEXPIREcall on every session view.This is useful, but means many session operations will trigger two Redis calls
(GET+EXPIRE).  On a high performance system, this can be a lot.This is a typical scenario with refreshing:timeout = 200The following timeline would occur:timeRedis callstimeout0GET+SETEX200100GET+EXPIRE300200GET+EXPIRE400300GET+EXPIRE500400GET+EXPIRE600500GET+EXPIRE700Scenario 2 - Timeout TriggerThe 1.4.x branch introduces atimeout_triggerto augment the Session'stimeout.Whereas atimeoutstates how long a session is good for, atimeout_triggerdefers how long a Session's refresh should be deferred for:Given the following example, the package will use a 1200s timeout for requests,
but only trigger an update of the expiry time when the current time is within
600s of the expiry:timeout = 1200
timeout_trigger = 600The following timeline would occur:timeRedis callstimeoutnext threshold0GET+SET*12006001GET1200600...599GET1200600600GET+SET*18001200601GET18001200...1199GET180012001200GET+SET*24001800This method is compatible with setting a TTL in redis viaSETEXor doing
everything within Python if Redis is in a LRU modeThe removes all calls toEXPIREbefore the threshold is reached, which can be
a considerable savings in read-heavy situations.The caveat to this method: an expiry timestamp must be stored within the payload
AND updating the timeout requires aSEToperation.Feature - set_redis_ttl_readheavyThis is a new option in1.4.2which should improve performance on readheavy
installations BUT may degrade performance on writeheavy installations.  This
option will aggregate a GET+EXPIRE on every read.set_redis_ttl_readheavyrequires the following:atimeoutvalue is setset_redis_ttlisTruetimeout_triggeris NOT setpython_expiresis NOT TrueThe default behavior of this library during a read-only request is this:On session access, query Redis viaredis.GET {session_id}In a Pyramid callback, update Redis viaredis.EXPIRE {session_id} {expiry}During a read-write:On session access, query Redis viaredis.GET {session_id}In a Pyramid callback, update Redis viaredis.SETEX {session_id} {payload} {expiry}The newset_redis_ttl_readheavychanges this. If enabled during a read-only
request, the behavior will be lighter on the Redis instance:On session access, open a pipeline with two Redis commands:pipeline.GET {session_id},pipeline.EXPIRE {session_id} {expiry}.In a Pyramid callback, the duplicate expire is suppressed.However during a read-write, the activity will be:On session access, open a pipeline with two Redis commands:pipeline.GET {session_id},pipeline.EXPIRE {session_id} {expiry}.In a Pyramid callback, update Redis viaredis.SETEX {session_id} {payload} {expiry}Read-heavy applications should see a slight performance bump via the pipeined
GET+EXPIRE, however write-heavy applications are likely to see a performance
degradation as it adds an extra EXPIRE to every request.Invalid LoggingThe default behavior of this library is to silently create new session when bad
session data is encountered, such as a cookie with an invalid id or corrupted
datastore.  A graceful "new session" is the ideal situation for end-users.The problem with that strategy is that problems in code or your application
stack can be hidden, and you might not know about a bad datastore.The 1.4 release introducesfunc_invalid_loggerto the factory constructor.
This can be used to track the invalid sessions that are safely caught and
silently upgraded.How?  The package tracks why a session is invalid with variant classes ofpyramid_session_redis.exceptions.InvalidSession.Specifically there are the following classes:InvalidSession(Exception)Catchall base classInvalidSession_NoSessionCookie(InvalidSession)The Session is invalid because there is no cookie.
This is the same as "new session".InvalidSession_NotInBackend(InvalidSession)The Session id was not in the backend.InvalidSession_DeserializationError(InvalidSession)Error deserializing.
This is raised ifdeserialized_fails_newis True. Otherwise the
exception is wrapped in aRawDeserializationErrorand raised without
being caught.InvalidSession_PayloadTimeout(InvalidSession)The inner Python payload timed out.InvalidSession_PayloadLegacy(InvalidSession)The Session is running on an earlier version.The factory accepts afunc_invalid_loggercallable argument. The input is the
raised exception BEFORE a new cookie is generated, and will be the request and
an instance ofInvalidSession.from pyramid_session_redis.exceptions import *
from my_statsd import new_statsd_client()

statsd_client = new_statsd_client()

def my_logger(request, raised_exception):
    """
    raised_exception will be an instance of InvalidSession
    log the exception to statsd for metrics
    """
    if isinstance(raised_exception, InvalidSession_NoSessionCookie):
        statsd_client.incr('invalid_session.NoSessionCookie')
    elif isinstance(raised_exception, InvalidSession_NotInBackend):
        statsd_client.incr('invalid_session.NotInBackend')
    elif isinstance(raised_exception, InvalidSession_DeserializationError):
        statsd_client.incr('invalid_session.DeserializationError')

factory = RedisSessionFactory(...
                              func_invalid_logger=my_logger,
                              ...
                              )Thefunc_invalid_loggerargument may be provided as a dotted-notation string
in a configuration file.Uncaught ErrorsThe exceptionpyramid_session_redis.exceptions.RawDeserializationErrorwill be
raised if deserialization of a payload fails anddeserialized_fails_newis notTrue. The first arg will be the caught exception. This allows for a standard
error across multiple deserialization options.FAQ:Q. What serialization is used?A. Serialization is used at two pointsSerializing the Session data.
The server-side data serialization is handled bypickle. This Session data is
created on the server by your application and remains on the server.pickleis safe to use in this context, as user-generated payloads can not be introduced
to these (de)serialization routines. If you wish to avoidpickle, or your
prefer to use another encoder, you can easily specify a different (de)serialization
routine such as a custom JSON, msgpack or pretty much anything else.Serializing thesession_idto encode into a signed cookie.
This library usesWebOb.cookies.SignedSerializerto securely manage a HMAC
signature of thesession_idcombined with a site-secret. The library allows
for a custom replacement to be provided as well.In the original library, the session id and signature were turned into a
cookie-safe value viapickle(de)serialization - a detail that remained in
this library through<=v1.5.0. This approach was identified as a security
risk, and was removed from this library starting inv1.5.1.ExamplesThere is an example of using this package inpyramid_session_multiexamples/single_file_app.py.Further Reading:For more information about Redis performance under Python please see an
associated project:https://github.com/jvanasco/dogpile_backend_redis_advancedTo suport multiple Sessions under Pyramidhttps://github.com/jvanasco/pyramid_session_multiUntil Nov 2016 this was maintained asjvanasco/pyramid_redis_sessionsThe main branch forjvanasco/pyramid_redis_sessionswas "custom_deployment"The branched named "main" is the upstream source from ericrasmussenAs of Nov 2016, this was forked into it's own project to allow for distribution
under PyPi.All support is handled via GitHub :https://github.com/jvanasco/pyramid_session_redisToDoseeTODO.mdChangelogseeCHANGES.mdSupportYou can report bugs or open feature/support requests via GitHubhttps://github.com/jvanasco/pyramid_session_redisLicensepyramid_session_redisis available under a FreeBSD-derived license. SeeLICENSE.txt <https://github.com/jvanasco/pyramid_session_redis/blob/main/LICENSE.txt>_
for details.