Package: justredis
What ?A Redis client for Python supporting many Redis features and Python synchronous (Python 3.5+) and asynchronous (Python 3.6+) communication.Why?Transparent API (Just call the Redis commands, and the library will figure out cluster routing, script caching, etc...)Per context and command properties (database #, decoding, RESP3 attributes)Asynchronous I/O support with the same exact API (but with the await keyword), targeting asyncio, trio and curio (usingAnyIOwhich needs to be installed as well if you want async I/O)Modular API allowing for easy support for multiple synchronous and asynchronous event loops and disabling of unneeded featuresCI Testing for CPython 3.5, 3.6, 3.7, 3.8, 3.9 and PyPy3 with Redis 5 and Redis 6No legacy support for old language featuresRedis features support tableFeatureSupportedNotesTransactionsVSeeexamplesandTransaction sectionPub/SubVSeeexamplesandPub/Sub and monitor sectionPipeliningVSeeexamplesandPipelining sectionClusterVSeeCluster commandsRESP3 supportVSeeRESP2 and RESP3 difference sectionSSLVSee theSSL connection parametersScript cachingXClient side cachingXSentinelXRoadmapGetting it out of alpha:API Finalization (your feedback is appreciated)Should calling redis be using __call__() or a special method such as "command" for refactoring ?Is the modify() API flexiable enough ?Is it easy to extend the module ?Should we add helper functions for SCAN iterators and other API ?Resolving some of the TODO in codeGeneral:More features in the support tableBetter test coverageResolve all TODO in codeMove documentation to topics + docstringsAutomate code convertion between sync and asyncMore CI checks such as flake8, pylint, etc..Not on roadmap (for now?)High level features which are not part of the Redis specification (such as locks, retry transactions, etc...)Manual command interface (maybe for special stuff like bit operations ?)Python 2 support (can be added, but after removing some misc syntax features)InstallingThe project can be found in PyPI as justredis. Install it via pip or requierments.txt file:pipinstalljustredisIf you want to use asynchronous I/O frameworks asyncio or trio or curio with this library, you need to install the AnyIO library as well:pipinstallanyioExamplesfromjustredisimportRedis# Let's connect to localhost:6379 and decode the string results as utf-8 stringsr=Redis(decoder="utf8")assertr("set","a","b")=="OK"assertr("get","a")=="b"assertr("get","a",decoder=None)==b"b"# But this can be changed per command# We can even run commands on a different database numberwithr.modify(database=1)asr1:assertr1("get","a")==None# In this database, a was not set to b# Here we can use a transactional set of commands# Notice that when we take a connection, if we plan on cluster support, we need# to tell it a key we plan on using inside, or a specific server addresswithr.connection(key="a")asc:c("multi")c("set","a","b")c("get","a")assertc("exec")==["OK","b"]# Or we can just pipeline the commands from beforewithr.connection(key="a")asc:result=c(("multi",),("set","a","b"),("get","a"),("exec",))[-1]assertresult==["OK","b"]# Here is the famous increment example# Notice we take the connection inside the loop,# this is to make sure if the cluster moved the keys, it will still be okwhileTrue:withr.connection(key="counter")asc:c("watch","counter")value=int(c("get","counter")or0)c("multi")c("set","counter",value+1)ifc("exec")isNone:# Redis returns None in EXEC command when the transaction failedcontinuevalue+=1# The value is updated if we got herebreak# Let's show some publish and subscribe commands,# here we use a push connection (where commands have no direct response)withr.connection(push=True)asp:p("subscribe","hello")assertp.next_message()==["subscribe","hello",1]assertp.next_message(timeout=0.1)==None# Let's wait 0.1 seconds for another resultr("publish","hello",", World !")assertp.next_message()==["message","hello",", World !"]APIRedis(**kwargs)@classmethodfrom_url(url,**kwargs)__enter__()/__exit__()close()# kwargs options = endpoint, decoder, attributes, database__call__(*cmd,**kwargs)endpoints()# kwargs options = decoder, attributes, databasemodify(**kwargs)# Returns a modified settings instance (while sharing the pool)# kwargs options = key, endpoint, decoder, attributes, databaseconnection(push=False,**kwargs)__enter__()/__exit__()close()# kwargs options = decoder, attributes, database__call__(*cmd,**kwargs)# On push connection no result for calls# kwargs options = decoder, attributes, databasemodify(**kwargs)# Returns a modified settings instance (while sharing the connection)# Push connection only commands# kwargs options = decoder, attributesnext_message(timeout=None,**kwargs)__iter__()__next__()URI connection optionsRedis.from_url()options are:Regular TCP connectionredis://[[username:]password@]host[:port][/database][[?option1=value1][&option2=value2]]SSL TCP connection (you can use ssl instead of rediss)rediss://[[username:]password@]host[:port][/database][[?option1=value1][&option2=value2]]Unix domain connection (you can use unix instead of redis-socket)redis-socket://[[username:]password@]path][[?option1=value1][&option2=value2]]For cluster, you can replace host:port with a list of host1:port1,host2:port2,... if you want fallback options for backup.You can add options in the end from the Redis constructor options below.Settings optionsThis are theRedis()constructor options:pool_factory ("auto")
    "auto" / "cluster" - Try to figure out automatically what the Redis server type is (currently cluster / no-cluster)
    "pool" - Force non cluster aware connection pool (simpler code)
address (None)
    An (address, port) tuple for tcp sockets, the default is (localhost, 6379)
    An string if it's a path for unix domain sockets, the default is "/tmp/redis.sock"
username (None)
    If you have an ACL username, specify it here
password (None)
    If you have an AUTH / ACL password, specify it here
client_name (None)
    If you want your client to be named on connection specify it here
resp_version (2)
    Specifies which RESP protocol version to use for connections
    -1 = auto detect
    2 = RESP2
    3 = RESP3
socket_factory ("tcp")
    Specifies which socket type to use to connect to the redis server
    "tcp" tcp socket
    "unix" unix domain socket
    "ssl" tcp ssl socket
connect_retry (2)
    How many attempts to retry connecting when establishing a new connection
max_connections (None)
    How many maximum concurrent connections to keep to a server in the connection pool, the default is unlimited
wait_timeout (None)
    How long (float seconds) to wait for a connection when the connection pool is full before returning an timeout error, the default is unlimited
cutoff_size (6000)
    The maximum ammount of bytes that will be appended together instead of sent seperatly before sending data to the socket, 0 to disable this feature
custom_command_class (None)
    Register a custom class to extend redis server commands handling
encoder (None)
    Specify how to encode strings to bytes, it can be a string, list or dictionary that are passed directly as the parameters to str.encode, the default is "utf8"
connect_timeout (None)
    How many (float seconds) to wait for a connection with a server to be established, the default is unlimited
socket_timeout (None)
    How many (float seconds) to wait for a socket operation (read/write) with a server, the default is unlimitedThis parameters can be passed to theRedis()constructor, or to themodify()method or per__call__():decoder (None)
    Specify how to decode the string results from the server, it can be a string, list or dictionary that are passed directly as parameters to bytes.decode, the default is normal bytes conversion
attributes (False)
    Specify if you want to handle the attributes fields from the RESP3 protocol (read the special section about this in the readme)
database (None)
    Set which database to operate on the server, the default is 0This can be provided to theRedis()constructor if you are using the cluster pool_factory:addresses (None)
    Multiple (address, port) tuples for cluster ips for fallback. The default is ((localhost, 6379), )This can be provided to theRedis()constructor for tcp and ssl socket_factory:tcp_keepalive (None)
    How many seconds to check the TCP connection liveness, the default is disabled
tcp_nodelay (True)
    Enable or disable the TCP nodelay algorithmThis can be provided to theRedis()constructor for ssl socket_factory:ssl_context (None)
    An Python SSL context object, the default is Python's ssl.create_default_context()
ssl_cafile (None)
    A path to the CA certificate file on disk, works only if ssl_context is None
ssl_certfile (None)
    A path to the server certificate file on disk, works only if ssl_context is None
ssl_keyfile (None)
    A path to the server key file on disk, works only if ssl_context is NoneRead the cluster and connection documentation below for the options for theconnection()and__call__()APIExceptionsValueError - Will be thrown when an invalid input was given. Nothing will be sent to the server.
Error - Will be thrown when the server returned an error to a request.
PipelinedExceptions - Will be thrown when some of the pipeline failed.
RedisError - Will be thrown when an internal logic error has happened.
    CommunicationError - An I/O error has occured.
    ConnectionPoolError - The connection pool could not get a new connection.
    ProtocolError - Invalid input from the server.Redis command replacementsThe following Redis commands should not be called directly, but via the library API:Username and Password (AUTH / ACL)If you have a username or/and password you want to use, pass them to the connection constructor, such as:r=Redis(username="your_username",password="your_password")Database selection (SELECT)You can specify the default database you want to use at the constructor:r=Redis(database=1)If you want to modify it afterwards for a specific set of commands, you can use a modify context for it:withr.modify(database=2)asr1:r1("set","a","b")Transaction (WATCH / MULTI / EXEC / DISCARD)To use the transaction commands, you must take a connection, and use all the commands inside. Please read theConnection sectionbelow for more details.Pub/Sub and monitor (SUBSCRIBE / PSUBSCRIBE / UNSUBSCRIBE / PUNSUBSCRIBE / MONITOR)To use push commands, you must take a push connection, and use all the commands inside. Please read theConnection sectionbelow for more details.UsageConnection commandsTheconnection()method is required to be used for sending multiple commands to the same server (such as transactions) or to talk to the server in push mode (pub/sub and monitor).You can pass to the methodpush=Truefor push mode where commands have no direct response (else it defaults to a normal connection).While you do not have to pass akey=, it's better to provide one you are about to use inside, in case you want to talk to a cluster later on.There are some instances you might want to talk to a specific server in a cluster (like getting keyspace notifications from it), so you can passendpoint=instead ofkey=with that server's address.Check thetransaction or pubsub examplesabove for syntax usage.PipeliningYou can pipeline multiple commands together by passing an list of commands to be sent together. This is usually to have better latency.Notice that if you are talking to a cluster, the pipeline must contain commands which handle keys in the same keyslots of a given server.If some of the commands failed, an PipelinedExceptions exception will be thrown, with it's args pointing to the result of each command.Check thepipeline exampleabove for syntax usage.Cluster commandsCurrently the library supports talking to Redis master servers only. It knows automatically when you are connected to a cluster (unless you disabled that feature in the constructor settings explicitly).If you want to specify multiple addresses for redundency, you can do so:r=Redis(addresses=(('host1',port1),('host2',port2)))You can get the list of servers with theendpoints()method.You can also send a command to all the masters by addingendpoint='masters'to the__call__():r("cluster","info",endpoint="masters")You can also open a connection to a specific instance, for example to get key space notifications or monitor it by addingendpoint=<the server address>to theconnection()method).RESP2 and RESP3 differenceThe library supports talking both in RESP2 and RESP3. By default it will use RESP2, because this way you'll get same response whether you are talking to a RESP3 supporting server (Redis server version 6 and above) or not.You can still tell it to use RESP3 or to auto negotiate the highest version with the specific server:r=Redis(resp_version=2)# Talk RESP2 onlyr=Redis(resp_version=3)# Talk RESP3 only (will throw an Exception if server does not support it)r=Redis(resp_version=-1)# Talk in the highest version possibleYou can read about RESP3 protocol and responses in theRedis documentation.RESP3 allows the clients to know the response type (such as strings, lists, dictionaries, sets...), and Justredis supports all of the response types.RESP3 provides an option to get with the results extra attributes. Since Python's type system cannot add the attributes easily, another configuration value was added,attributeswhich specifies if you care about getting this information or not, the default is False:r=Redis(attributes=True)If attributes is disabled, you will get the direct Python mapping of the results (set, list, dict, string, numbers, etc...) and if enabled, you will get a special object which will hold the raw data in thedataattribute, and the attributes in theattrsattribute. Notice that this feature is orthogonal to choosing RESP2 / RESP3 (but in RESP2 the attrs will always be empty), for ease of development.Here is an example of the difference in Redis version 6, with and without attributes:>>>importjustredis>>>r=justredis.Redis()# By default it connects via RESP2>>>r("hgetall","aaa")[b'bbb',b'ccc',b'ccc',b'ddd']>>>r("hgetall","aaa",attributes=True)# This is RESP2 with attributes, it has .data and .attrsArray:[String:b'bbb',String:b'ccc',String:b'ccc',String:b'ddd']>>>r=justredis.Redis(resp_version=-1)# This will connect to Redis 6 with RESP3>>>r("hgetall","aaa")OrderedDict([(b'bbb',b'ccc'),(b'ccc',b'ddd')])# This is Python's OrderedDict>>>r("hgetall","aaa",attributes=True)Map:OrderedDict([(String:b'bbb',String:b'ccc'),(String:b'ccc',String:b'ddd')])Thread and async safetyThe library is thread safe and async safe. Do not pass Connection objects between different threads or coroutines.ModifyYou can change some of the settings on a per__call()__call, or if you want multiple calls to have different settings, you can use themodify()method on a Connection or Redis objects.Currently you can change the string decoder used, the database number and the attributes flag. Check theexamplesabove to see how it's done.Serialization and deserializationThe library supports as inputs only this types: bytes, bytearray, memoryview, str, int and float. If you pass a string, it will be encoded to bytes by the given encoder option (default is utf-8). Passing anything else will result in a ValueError.The library will return the data types that the RESP protocol returns as described in the RESP section. Exceptions will always be utf-8 string encoded and for other string results, you can decide to keep them as bytes, or to decode them to a string.Async supportThe API for the asynchronous commands is exactly the same, just adding "await" where it's needed:AsyncRedis(**kwargs)@classmethodfrom_url(url,**kwargs)async__aenter__()/__aexit__()asyncaclose()# kwargs options = endpoint, decoder, attributes, databaseasync__call__(*cmd,**kwargs)asyncendpoints()# kwargs options = decoder, attributes, databasemodify(**kwargs)# Returns a modified settings instance (while sharing the pool)# kwargs options = key, endpoint, decoder, attributes, databaseasyncconnection(push=False,**kwargs)async__aenter__()/async__aexit__()asyncaclose()# kwargs options = decoder, attributes, databaseasync__call__(*cmd,**kwargs)# On push connection no result for calls# kwargs options = decoder, attributes, databasemodify(**kwargs)# Returns a modified settings instance (while sharing the connection)# Push connection only commands# kwargs options = decoder, attributesasyncnext_message(timeout=None,**kwargs)__iter__()async__next__()Don't forget there is no__del__()method in async code, so callaclose()or use async context managers when needed.Extending the library with more command supportYou can extend the Redis object to support real redis commands, and not just calling them raw, here is an example:fromjustredisimportRedisclassCustomCommands:def__init__(self,base):self._base=basedefget(self,key,**kwargs):returnself._base("get",key,**kwargs)defset(self,key,value,**kwargs):returnself._base("set",key,value,**kwargs)r=Redis(custom_command_class=CustomCommands)r.set("hi","there")assertr.get("hi",decoder="utf8")=="hi"withr.modify(database=1)asr1:assertr1.get("hi")==None