Package: scim2-filter-parser
DescriptionSCIM 2.0 defines queries that look like this:'emails[type eq "work" and value co "@example.com"] or ims[type eq "xmpp" and value co "@foo.com"]'These can be hard to work with and covert into SQL to run against a database.That’s where SCIM 2.0 Filter Parser (SFP) can help.SFP is broken up into four modules, each handling a different part of
translating a SCIM call into a SQL query.The first step is tokenization or lexical analysis where the filter query
is broken down into many tokens that make it up.sfp-lexer 'emails[type eq "work" and value co "@example.com"] or ims[type eq "xmpp" and value co "@foo.com"]'

Token(type='ATTRNAME', value='emails', lineno=1, index=0)
Token(type='LBRACKET', value='[', lineno=1, index=6)
Token(type='ATTRNAME', value='type', lineno=1, index=7)
Token(type='EQ', value='eq', lineno=1, index=12)
Token(type='COMP_VALUE', value='work', lineno=1, index=15)
Token(type='AND', value='and', lineno=1, index=22)
Token(type='ATTRNAME', value='value', lineno=1, index=26)
Token(type='CO', value='co', lineno=1, index=32)
Token(type='COMP_VALUE', value='@example.com', lineno=1, index=35)
Token(type='RBRACKET', value=']', lineno=1, index=49)
Token(type='OR', value='or', lineno=1, index=51)
Token(type='ATTRNAME', value='ims', lineno=1, index=54)
Token(type='LBRACKET', value='[', lineno=1, index=57)
Token(type='ATTRNAME', value='type', lineno=1, index=58)
Token(type='EQ', value='eq', lineno=1, index=63)
Token(type='COMP_VALUE', value='xmpp', lineno=1, index=66)
Token(type='AND', value='and', lineno=1, index=73)
Token(type='ATTRNAME', value='value', lineno=1, index=77)
Token(type='CO', value='co', lineno=1, index=83)
Token(type='COMP_VALUE', value='@foo.com', lineno=1, index=86)
Token(type='RBRACKET', value=']', lineno=1, index=96)The second step is to convert that series of tokens into a abstract syntax tree.sfp-parser 'emails[type eq "work" and value co "@example.com"] or ims[type eq "xmpp" and value co "@foo.com"]'

Filter(expr=LogExpr, negated=False, namespace=None)
    LogExpr(op='or', expr1=Filter, expr2=Filter)
        Filter(expr=Filter, negated=False, namespace=None)
            Filter(expr=Filter, negated=False, namespace=AttrPath)
                Filter(expr=LogExpr, negated=False, namespace=None)
                    LogExpr(op='and', expr1=Filter, expr2=Filter)
                        Filter(expr=AttrExpr, negated=False, namespace=None)
                            AttrExpr(value='eq', attr_path=AttrPath, comp_value=CompValue)
                                AttrPath(attr_name='type', sub_attr=None, uri=None)
                                CompValue(value='work')
                        Filter(expr=AttrExpr, negated=False, namespace=None)
                            AttrExpr(value='co', attr_path=AttrPath, comp_value=CompValue)
                                AttrPath(attr_name='value', sub_attr=None, uri=None)
                                CompValue(value='@example.com')
                AttrPath(attr_name='emails', sub_attr=None, uri=None)
        Filter(expr=Filter, negated=False, namespace=None)
            Filter(expr=Filter, negated=False, namespace=AttrPath)
                Filter(expr=LogExpr, negated=False, namespace=None)
                    LogExpr(op='and', expr1=Filter, expr2=Filter)
                        Filter(expr=AttrExpr, negated=False, namespace=None)
                            AttrExpr(value='eq', attr_path=AttrPath, comp_value=CompValue)
                                AttrPath(attr_name='type', sub_attr=None, uri=None)
                                CompValue(value='xmpp')
                        Filter(expr=AttrExpr, negated=False, namespace=None)
                            AttrExpr(value='co', attr_path=AttrPath, comp_value=CompValue)
                                AttrPath(attr_name='value', sub_attr=None, uri=None)
                                CompValue(value='@foo.com')
                AttrPath(attr_name='ims', sub_attr=None, uri=None)The third step is to transpile this AST into a language of our choice.
The above query is transpiled to SQL below.sfp-transpiler 'emails[type eq "work" and value co "@example.com"] or ims[type eq "xmpp" and value co "@foo.com"]'

((emails.type = {0}) AND (emails.value LIKE {1})) OR ((ims.type = {2}) AND (ims.value LIKE {3}))
{0: 'work', 1: '%@example.com%', 2: 'xmpp', 3: '%@foo.com%'}The fourth step is to take what is a segment of a SQL WHERE clause and complete
the rest of the SQL query.sfp-query 'emails[type eq "work" and value co "@example.com"] or ims[type eq "xmpp" and value co "@foo.com"]'

>>> DO NOT USE THIS OUTPUT DIRECTLY
>>> SQL INJECTION ATTACK RISK
>>> SQL PREVIEW:
    SELECT DISTINCT users.*
    FROM users
    LEFT JOIN emails ON emails.user_id = users.id
    LEFT JOIN schemas ON schemas.user_id = users.id
    WHERE ((emails.type = work) AND (emails.value LIKE %@example.com%)) OR ((ims.type = xmpp) AND (ims.value LIKE %@foo.com%));Please note that SFP does not build SQL queries with parameters pre-injected.
That would create a SQL injection attack vulnerability. Instead aSQLQueryobject is created and can be forced to display itself as seen above
byprinting the query object.Installationpip install scim2-filter-parser

# Or ...

pip install scim2-filter-parser[django-query]UseAlthough command line shims are provided, the library is intended to be used
programmatically. Users of the library should instantiate thescim2_filter_parser.queries.SQLQueryclass with an attribute map and optionally
any joins necessary to make all required fields accessible in the query.For example, if user information is stored in theuserstable and email
information is stored in a different tableemails, then the attribute map
and the joins might be defined as so:from scim2_filter_parser.queries import SQLQuery

attr_map = {
    ('userName', None, None): 'users.username',
    ('name', 'familyName', None): 'users.family_name',
    ('meta', 'lastModified', None): 'users.update_ts',
    ('emails', None, None): 'emails.address',
    ('emails', 'value', None): 'emails.address',
}

joins = (
    'LEFT JOIN emails ON emails.user_id = users.id',
)

filter_ = 'name.familyName co "Simpson" or emails.value eq "lisa@example.com"'

q = SQLQuery(filter_, 'users', attr_map, joins)

q.sql # Will be...

SELECT DISTINCT users.*
FROM users
LEFT JOIN emails ON emails.user_id = users.id
WHERE (users.family_name LIKE %s) OR (emails.address = %s);

q.params # Will be...

['%Simpson%', 'lisa@example.com']The attribute_map (attr_map) is a mapping of SCIM attribute, subattribute,
and schema uri to a table field. You will need to customize this to your
particular database schema.TheSQLQuery.sqlmethod returns SQL that can be used as the first
argument in a call tocursor.execute()with your favorite DB engine.
If you are using a database that requires a replacement character other than ‘%s’,
then you can subclass theSQLQueryclass and override theplaceholderclass
level variable. See the query module and unit tests for an example of this subclassing
with SQLite.TheSQLQuery.paramsmethod returns a list of items that can be used as the
second argument in a call tocursor.execute().DjangoIf you would like to produce aDjango Qobject instead of a raw SQL query, you can pass
a SCIM filter query and attribute map to theget_queryfunction from the modulescim2_filter_parser.transpilers.django_q_object. For example:get_query(scim_query: str, attr_map: Mapping)This Q object can then be passed to a Django filter query like so:query = get_query(scim_query, attr_map)
User.objects.filter(query)Please note that you will need to install the Django Query extra like for this feature to be available:pip install scim2-filter-parser[django-query]SpeedSFP is pretty fast. Check out the speed_test.py script for details on the long and short
filter queries tested. SFP transpiled a short filter query into SQL in under 54 microseconds.
For a longer query, SFP only took 273 microseconds.➜  scim2-filter-parser git:(master) ✗ python -m timeit -s "import speed_test" "speed_test.short()"
10000 loops, best of 3: 53.8 usec per loop
➜  scim2-filter-parser git:(master) ✗ python -m timeit -s "import speed_test" "speed_test.long()"
1000 loops, best of 3: 273 usec per loopDevelopment SpeedSince this project is relatively stable, time is only dedicated to it on
Fridays. Thus if you issue a PR, bug, etc, please note that it may take a week
before we get back to you. Thanks you for your patience.DevelopmentThis project uses Poetry to manage dependencies, etc. Thus to install the
necessary tools when developing, run:poetry install -v --extras "django-query"Testshttps://github.com/15five/scim2-filter-parser/actionsTests are typically run locally withtox(https://tox.wiki/). Tox will test
all supported versions of Python.toxTo run the test suite with a single version of Python (the version you created
the virtualenv with), run:poetry run pytest tests/poetry run pytest tests/Coveragehttps://codecov.io/gh/15five/scim2-filter-parsertox -e coverageDeploymenthttps://pypi.org/project/scim2-filter-parser/To deploy this package to PyPI, run:poetry build
poetry publish