Package: ldapter
ldapterA lightweight adapter API for python-ldap that provides simplified
access to LDAP objects and attributes in native Python types, as well as
transparent read-through caching support using either a simple in-memory
cache or Redis.InstallationLdapteris available from PyPi and can be installed usingpipor
whatever package management solution you prefer.pipinstallldapterIf you want to usedatetimetypes to represent ISO-8601 formatted
date strings in your LDAP entries, you'll also need to install
python-dateutilpipinstallpython-dateutilRunning the DemoThe source repository includes a demo. Of course, you'll need an LDAP
server and (optionally) a Redis cache in order to seeldapterat work.
To make this easy, the repository includes everything you need to run
the demo using Docker and Docker Compose on your local workstation.Start up the demo LDAP server and Redis cache in a terminal.cdsrc/demo/docker
dockercomposeup--buildDepending on the version of Docker you have installed, you might find
that you need to usedocker-composeinstead ofdocker composein
the second line above.Run the demo in another terminal.cdsrc/demo/pythonPYTHONPATH=../main/pythonpythondemo.pyUsing the ldapter APITheDirectoryobject class is the top-level API.fromldapterimportDirectoryfromldapterimportinet_org_person_schema# Create a directory adapter with no cache.directory=Directory(schema=inet_org_person_schema())TheDirectory.fetch_allmethod returns a list of tuples, each of which
contains aDistinguishedNameand anEntry. The list will be empty if
the LDAP search returned no entries.fromldapterimportDirectory,Scopefromldapterimportinet_org_person_schemadirectory=Directory(schema=inet_org_person_schema())base="dc=example,dc=org"search_filter="uid=fletcher"fordn,entryindirectory.fetch_all(base,Scope.SUBTREE,search_filter):print(dn)fork,vinentry.items():print(f"{k}:{v}")ADistinguishedNameis an object that implements a subset of the
dict protocol, allowing relative names (RDN) to be easily extracted.AnEntryis an object that uses the directory schema to transform
attribute values into useful Python data types. It implements a subset
of the dict protocol, so that attribute values can be easily accessed
by name. A single-valued attribute has a common Python
data type (e.g. str, int, bool, datetime) orDistinguishedName.
A multi-valued attribute is a Python list whose values are one of
these types.Caching StrategyWhen an LDAP search is performed (via theDirectory.fetch_allorDirectory.fetch_onemethods) a resulting set of matching
LDAP entries is returned (which may be empty). Each entry has an LDAP distinguished name that serves as the
"primary key" for retrieving that specific entry. Each retrieved object
is placed into the cache using its distinguished name as the key.
Additionally, the search base, scope, and filter expression specified for
the search are used as a key for a cache entry that contains the
set of distinguished names found by the search.Conceptually you can think of the cache as a simple hash table
where each entry has a key string and corresponding JSON value (an
array or object). If we perform a search usingdc=example,dc=orgas the base, SUBTREE as the scope, and a filter expression such as(&(sn=Smith)(givenName=Mary*), the cache might have entries such as the
following:KeyValues:dc=example,dc=org:SUBTREE:(&(sn=Smith)(givenName=Mary*)["uid=liz,dc=example,dc=org", "uid=mas87,dc=example,dc=org", "uid=msmith,dc=example,dc=org", ...]e:uid=liz,dc=example,dc=org{ "uid": "liz", sn="Smith", "givenName": "Mary Elizabeth", ... }e:uid=mas87,dc=example,dc=org{ "uid": "mas87", sn="Smith", "givenName": "Mary Anne", ... }e:uid=msmith,dc=example,dc=org{ "uid": "msmith", sn="Smith", "givenName": "Mary", ... }The keys are prefixed with a namespace identifier based on the type
of the value. In this example, we uses:for a search result ande:for an LDAP entry. This makes the keys easy to distinguish when viewing
the cache and prevents any potential for collisions of keys representing
different types.On a subsequent search request with the same base, scope, and filter
expression, the cached list of distinguished names is retrieved,
and those names are used to retrieve the corresponding LDAP entries from
the cache. In this way, different searches can share the same LDAP
entries in the cache -- any given LDAP entry is stored at most once.No attempt is made to analyze the filter expression to predict what might
be returned by the LDAP server. Cached searches are simply the list of
distinguished names that matched the given search criteria when the
search was performed. This strategy works well for accelerating searches
that are performed repeatedly with thesamesearch criteria, but it
does nothing for a search whose criteria are even slightly different
from a previously executed search. As discussed below inOptimizing
Single Entry Fetches, additional
optimizations are included to accelerate the common case of fetching
LDAP entries using the distinguished name as a primary key.Using a CacheThe module includes two cache providers.A provider that acts as a Redis client.A provider that uses a simple dict-based in-memory cache.The simple dict-based cache is suitable mostly for testing. For
optimal performance and persistence beyond the runtime of a Python
script, Redis is recommended.To use a cache, specify the cache provider when creating aDirectory.fromldapterimportDirectoryfromldapterimportRedisDirectoryCachefromldapterimportinet_org_person_schema# Create a directory adapter that uses a Redis cachecache=RedisDirectoryCache(host="localhost",port=6379,db=0)directory=Directory(schema=inet_org_person_schema(),cache=cache)By default, the Redis cache provider uses the same time-to-live (TTL)
for all LDAP search result lists and corresponding LDAP entries. In most
cases, you will want to differentiate the TTL for an entry, based on the
object type, since the probability that an entry will be updated in the
source LDAP server tends to vary based on object type. For example,
attributes about a person change relatively infrequently, while the
composition of groups of people changes more often and it may be
important to observe such changes when groups are used for
authorization.When constructing the cache object, you can specify a dict with
TTLs for different object types defined in the schema.fromdatetimeimporttimedeltafromldapterimportDirectoryfromldapterimportRedisDirectoryCachefromldapterimportinet_org_person_schema# Create a directory adapter that uses a Redis cachecache=RedisDirectoryCache(host="localhost",port=6379,db=0,search_ttl=timedelta(minutes=15),entry_ttl={"inetOrgPerson":timedelta(hours=4)})directory=Directory(schema=inet_org_person_schema(),cache=cache)Thesearch_ttlis used when caching the search criteria (base, scope,
filter) and corresponding list of distinguished names that were
retrieved by a search.Theentry_ttldict is used when caching the LDAP entries returned by
a search.  By default, the Redis cache provider uses theobjectClassattribute to distinguish LDAP entries by type. It checks theentry_ttldict for a TTL specific to the given type. If not found it uses thesearch_ttlas the fallback.Configuring RedisWhen using the Redis cache provider, the cache eviction policy is an
important consideration. When the cache encounters memory pressure,
Redis can evict some cache entries to make room for new additions to
the cache. For LDAP, the Redis Least Frequently Used (LFU) eviction
policy tends to be good choice. The LFU policy uses an estimator of
cache entry usage frequency, and tends to prefer evicting cache entries
that are unlikely to be needed based on frequency, rather than last
access time.All cache entries managed byldapterhave a TTL that controls the
maximum lifetime of the cache entry -- whether a search result list or
and LDAP object entry. To configure Redis to evict cache entries based
on the LFU policy, configure a maximum memory size and specify the
volatile LFU policy in the Redis configuration.maxmemory 2gb
maxmemory-policy volatile-lfuOptimizing Single Entry FetchesWhen fetching a specific LDAP entry using the entry's distinguished
name, the caching implementation can provide an additional optimization,
allowing a single object search to be satisfied from cache entries
created by prior searches.A common situation where this optimization is useful is when a UI
provides a means to search for a person using the person's name as
search criteria.people=directory.fetch_all("dc=example,dc=org",Scope.SUBTREE,"(&(sn=Smith)(givenName=Mary*))")Since the surname and given name are both quite common, this is likely
to return several matching entries. In the UI, the matches are presented
with other distinguishing attributes (middle name, department, etc)
allowing the user to select specific person. Often, after the specific
person has been selected, there will be a need to fetch the specific
person entry again soon after the original search is performed. The subsequent
search could be performed using a filter expression based on some unique
attribute of LDAP person entry.person=directory.fetch_one("dc=example,dc=org",Scope.SUBTREE,"uid=liz")This approach will indeed work. However, it will not necessarily
take advantage of the fact the person entry to be fetched was just placed
into the cache by the prior search. In fact, it will perform an LDAP
search unless the same search criteria (base, scope, filter expression) was
performed recently to retrieve this person's LDAP entry.However, if the subsequent fetch for this person object instead used
the distinguished name of the person entry, the previously cached
entry would be returned without the need to perform an LDAP search.importrandomfromldapterimportDirectory,ScopefromldapterimportRedisDirectoryCachefromldapterimportinet_org_person_schema# Create a directory adapter that uses a Redis cache with default configcache=RedisDirectoryCache()directory=Directory(schema=inet_org_person_schema(),cache=cache)# Find everyone named Mary Smithpeople=directory.fetch_all("dc=example,dc=org",Scope.SUBTREE,"(&(sn=Smith)(givenName=Mary*))")# pick one at randomselected=random.randint(0,len(people))selected_person=people[selected]# get the DN from the selected tupledn=selected_person[0]# this fetch will be always be satisfied from cacheperson=directory.fetch_one(dn,Scope.BASE)When the scope is BASE, the search is essentially a primary key search
and has no filter expression. If the cache contains the entry with the
requested DN, there's no reason to perform the search.