Package: jtools
JToolsJTools is a robust library for interacting with JSON-like objects:
providing the ability to quickly query, format, and filter JSON-like data.JTools is available in Python and JavaScript and consists of three main components:Query: Extract and transform the values of nested fields.Query("data.timestamp.$parse_timestamp.$time_part('year')").many(items)Filter: Combine the querying capabilities ofQuerywith the ability
to define filtering conditions to find just the elements you want.Filter(Key("data.timestamp.$parse_timestamp.$time_part('year')").gt(2015)).many(data)Formatter: Take multiple queries and format them into a stringFormatter("Item @data.id was released in @data.timestamp.$parse_timestamp.$time_part('year')").single(data[0])JTools exists in two different langauges with almost identical names and capabilities to allow you to move between the packages seamlessly.Python -jtoolsJavaScript -@blending_jake/jtoolsWritten in TypeScript and distributed as an ES6 style module with type
declaration files.DifferencesIn JTools-Py, the==and===filter operators will behave the same, as well as!=and!==.JTools-Py uses thedatetimepackage for time related queries, while JavaScript usesmomentJTools-Py replicates JavaScript's lack of differentiation betweenitem[0]anditem["0"]by default.
However, this can be changed in the Python version by settingQuery(..., convert_ints=False).
If that argument is set toFalsein Python, thenitem.0would work on{"item": {"0": ...}},
but not on{"item": {0: ...}}. The JavaScript version essentially always hasconvert_ints=True.Recent Changes2.0.0Numerous new language featuresSupport for arithmetic in arguments. The supported operators are-,+,*,/,//(floor division),**(exponentiation), and%. The operators can be used any
place a value is valid within an argument, such as$special(14000 * 2),$special(@value // 2),$special([1, 1+1, 1+2]). The order of operations follows typical precedence, and arithmetic
can be nested in().Support for keyword arguments:$index("value", fallback=12)Whitespace is allowed in a lot more places in a query now, which is very helpful since both
JavaScript and Python support multi-line strings.Performance increase for cases were aQueryfield doesn't have any specials. A faster parsing
method can be used for about a 40x performance increase in creating theQueryobject. Note, this
just speeds upQuerycreation and not the speed of.single(),.many(), etc.Added functionality so thatFiltervalues can be queries.Using JSON:{ field: <field>, operator: <op>, value: { query: <value query> } }UsingKey:Key(<field>).<op>(Key(<value_query>))For example:Key("tag.tag1").gt(Key("tag.tag2"))Four new filters:subset,!subset,superset,!superset. These should be readfield value <op> filter value, sosubsetmeans all of the values returned from the field query
are in the filter value, which allows better filtering when both the field query and value are iterable.Many new specials$key_of_min_value- Gets the key of the min value in an dict/map/object$key_of_max_value- Gets the key of the max value in an dict/map/object$time_part(part) -> int- Get a specific part of a datetime/moment object. Valid values forpartare as follows:'millisecond''second''minute' - Starting from 0'hour' - Starting from 0'day' - Day of month, starting from 1'month' - Month of year, starting from 1'year''dayOfWeek' - Value in 0-6, corresponding to Monday-Sunday'dayOfYear' - Day of year, starting from 1$minand$max$arith(op, arg_value)- Perform an arithmetic operation on the current value.opcan be
any of the supportmath operators.$lowercase,$uppercase, and$titlecase$filter(field, op, value, single=true)OR$filter({<filter>}, single=true)OR$filter([<filters>], single=true)- Allow a single value or list of values to be filtered.
The filters should be formatted that same way they are forFilter.$pipeline(pipeline)- Apply multiple specials to value in a given order. Can be very useful
when used in conjunction with$mapto apply multiple specials at once.pipelinemust be of format:[
    <special>

    OR

    [<special>, arg1, arg2, ..., (optional) map/object/dict of keyword arguments],
    ...
]

for example

$pipeline([
    ['index', 'balance'],
    ['range', 1],
    ['replace', {'old': ',', 'new': ''}],
    ['float']
])

is the same as
$index("balance").$range(1).$replace(old=",", new="").$floatUpdates to old specials$index(i, fallback=null, extended=false)- addedextendedargument to allow$indexto support any valid JQL query foriifextended=trueBug fixesFixed issue with filters not shortcutting properlyFixed Python issue with$printFixed issues with$fallbackwithFilterandFormatterFixed issue causing an error when trying to use an integer value in$sortand$group_byGlossaryInstallationQuickstartJQLArgumentsMath OperatorsQueryContextSpecialsFilterKeyConditionFormatterPerformanceChangelogInstallationPythonpip install jtools# importfromjtoolsimportQuery,Filter,Key,Condition,FormatterJavaScriptnpm i @blending_jake/jtools// importimport{Query,Filter,Key,Condition,Formatter}from"@blending_jake/jtools";QuickstartfromjtoolsimportQuery# sort by nested timestamp field by monthq=Query("meta.timestamp.$parse_timestamp.$time_part('month')")data=[...]sorted_data=sorted(data,key=q.single)JQLJQLor theJSON Query Languageis a custom built query language forJToolswhich supports powerful features
like accessing nested fields, transforming values, and even using nested queries as arguments.
The basic format of the language is:(<field> | $<special>) (. (<field> | $<special>))*
EX: 'data', 'data.timestamp', 'data.$split', '$split.0'fieldA field is just a value that can be used as an index, like a string or integer key for a map/dict or an integer for an array. JavaScript has very loose type-checking between strings
and integers, so either can essentially work in place of the other when indexing.Fields can only contain the following characters:[-a-zA-Z0-9_]. However, fields with prohibited characters can still
be indexed by using the$indexspecial, so to indexrange[0]use$index("range[0]").$specialA special is a function that is applied to the value that has been queried so far. There is a complete list of specialshere. These specials can be passed arguments, which is one of the most powerful features
ofJQL.ArgumentsArguments can be passed into specials as a positional or keyword arguments. Positional arguments
must come first and are denoted using the standard notation in most programming languages:$special(arg1, arg2, ...). Keyword arguments must come after any positional arguments and
can be used to specify a specific argument to set the value for:$special(arg1, name1=arg2, name2=arg3, ...). Keyword arguments can be used to make it clear
what argument you are setting - improving readability, or can be used to skip arguments which
have default values.For example, the index special has the following definition:$index(key, fallback=null, extended=false). If you want to use a JQL query forkey, then you need
to setextended=true. You can either write$index(key, null, true), or you can skip the
fallback argument and write$index(key, extended=true). Using keyword arguments might not save many
characters if the the special has few arguments, but can still greatly improve readability.It is important to note that arguments, and the parenthesis surrounding them can be left
completely off of a special if the default argument values are acceptable or there are no
arguments - so a query likedata.$values.$max.$divide(@data.$values.$sum)can be written
where most of the parenthesis and arguments are just left off.valueArguments are consideredvaluesand must be one of the following types.Math Expression
[] or [<value>(, <value>)*] - List
{} or {<value>(, <value>)*} - Set
{:} or {<key>: <value>(, <key>: <value>)*} - Map/Dict/Object (see below for <key> spec)
Integer
Float
String w/ '' or ""
true
false
null
@<query> - Yep! Nested queries!

Math Expression:
    Operators listed below
    Operands must be one of: Integer, Float, or @<query>

<key>:
    Math Expression
    @<query>    
    Integer
    Float
    String
    true
    false
    nullAs shown above, values and queries can be nested, so[[1, 2], ["bob"], {"Ann", 'Ralph'}, {'key': 4, 23: 5}]is valid. Additionally, the support for nesting queries is extremely powerful and allows for queries like:item.tag.$lookup(@table.colors), which, for{"item": {"tag": "product"}, "table": {"colors": {"product": "red"}}}results in"red"Basic math expressions are supported in arguments and, subsequently, nested values.
The supported operators are listed below and follow normal arithmetic precedence with
the order of evaluation from first -> last being:(),**,*  and / and // and %,+ and -.The operands for math expressions are limited to numbers (Float and Integer) and nested
queries.Math expressions can be nested in other values, for example:$special([ 1, @value, @value**2, @value**3 ])Math Operators+-*///- Floor division, akaMath.floor(a / b)%- Mod operator**- Exponentiation, akaMath.pow(a, b)ora^b()- Parenthesis can be use to change precedence and/or improve readabilityQueryQuerytakes the power ofJQLand puts it into practice querying and transforming values
in JSON-like data.Query(query, fallback=null, [convert_ints=true (if Python)])query:str | List[str]The field or fields to queryfallback: The value that will result if a non-existent field is queriedconvert_ints: Whether or not to convert any digit-only fields to integers.single(item, context={})item: The item to querycontext: SeeContextfor more detailsTake a single item and query it using the query(ies) providedQuery(field).single(...) -> resultQuery([field, field, ...]).single(...) -> [result, result, ...].many(items, context={})items: The items to querycontext: SeeContextfor more detailsTake a list of items, and query each item using the query(ies) providedQuery(field).many(...) -> [result, result, ...]Query([field, field, ...]).many(...) -> [[result, result, ...], [result, result, ...], ...]NotesMore specials can be added by using the class method.register_special()like so:Query.register_special(<name>, <func>, <argDef> [if JavaScript]). The function should be formatted as such:# Pythonlambdavalue,*args,context:...// JavaScript(value,context,args)=>{...}Wherevalueis the current query value, andcontextis the current context.Registering a special in JavaScriptRegistering a special works differently in JavaScript because the language does not support keyword arguments. To get
around this, an argument definition should be provided whenever the special is registered. The argument definition
should be formatted:['<argument name>'|{'name':'<argument name>',['default':<defaultvalue>]}]The argument definition allows you to state what the name of each parameter is, and allows a default value to be provided.
Additionally, '$args' and '$kwargs' can be placed in the argument definition to capture a variable number of positional
arguments or keyword arguments, respectively.Each registered special function is then passed an object,args, which is{ <argument name>: <argument value>, ... }.
The keys '$args' and '$kwargs' capture the variable number of positional and keyword arguments, respectively.Some example argument definitions and usages:Query.register_special('index',(value,context,args)=>{// use args.key, args.fallback, and args.extended to access arguments},['key',{name:'fallback',default:null},{name:'extended',default:false}])Query.register_special('filter',(value,context,args)=>{// args.$args is an array of any positional arguments// args.single must be specified as a keyword argument (as it's after $args)},['$args',{name:'single',default:false}])Query.register_special('update',(value,context,args)=>{// args.$kwargs is an object of any captured key->value pairsreturn{...value,...args.$kwargs};},['$kwargs'])ContextContext is a way of putting temporary variables into the query search space.How do I add something tocontext?Manually introduce values through.single(..., context)or.many(..., context).Use the$store_as()special to place a value in the current context for later useHow do I access something incontext?Any top-level field name is first looked for on the current item, then incontext. Note, top-level means
it is the main query in aQuerystring, or it follows an@, either as an argument or in aFormatterstring.It is important to note that fields on the current item will shadow fields in the context, so make sure to use unique field names.Ok, but give me an example.# Pythoncontext={"NOW":time.time()}Query("NOW.$subtract(@meta.timestamp).$divide(86400).$round.$suffix(' Days Ago')").single({...},context)// JavaScriptconstcontext={NOW:Date.now()/1000}newQuery("NOW.$subtract(@meta.timestamp).$divide(86400).$round.$suffix(' Days Ago')").single({...},context)SpecialsGeneral$inject(value: any) -> any: Inject a value into the query$length -> int$lookup(lookup: dict, fallback=null) -> any: Lookup the current value in the provided map/dict$store_as(name: string) -> any: Store the current query value in the current context for later use in the query. This does not
change the underlying data being queried.$pipeline(pipeline) -> any: Take a value and run it through multiple specials. Can be used in conjunction
with$mapor$value_mapto apply a series of transformations across a list of values.pipelinemust
be a list where each value is:string- Name of special[string, *args]- Name + positional arguments[string, dict of keyword args]- Name + dict/map/object of keyword arguments[string, *args, dict of keyword args]- Name + positional arguments +
dict/map/object of keyword arguments# PythonQuery("""meta.timestamp.$pipeline([['divide', 1000],'parse_timestamp',['time_part', 'year']])""")$print -> any: Print the current query value before continuing to pass that value along$filter(field, op, value, single=true)OR$filter({<filter>}, single=true)OR$filter([<filters>], single=true): Allow a single value or list of values to be filtered.
The filters should be formatted that same way they are forFilter.singlecan only be supplied
as a keyword argument.Dict/Maps/Objects$items -> List[[any, any]]$keys -> List[any]$key_of_max_value(just_key=true) -> any | [any, any]: Find the key of the max value in a dict/object/map. Ifjust_key=true, then just the value of the key will be returned. Otherwise, the[key, value] will be
returned.$key_of_min_value(just_key=true) -> any | [any, any]: Find the key of the min value in a dict/object/map. Ifjust_key=true, then just the value of the key will be returned. Otherwise, the[key, value] will be
returned.$wildcard(nxt, just_field=true) -> List[any]: On a given map or list, go through all values and see ifnxtis
a defined field. If it is, then return just the value ofnxton that item, ifjust_field=true, or the entire
item otherwise. This special allows a nested field to be extracted across multiple items where it it present.
For example:# Pythondata={"a":{"tag":"run"},"b":{"tag":"to-do","other":"task"},"meta":None}Query('$wildcard("tag")').single(data)# => ["run", "to-do"]Query('$wildcard("tag", false)').single(data)# => [{"tag": "run"}, {"tag": "to-do", "other": "task"}]// JavaScriptletdata={"a":{"tag":"run"},"b":{"tag":"to-do","other":"task"},"meta":null}newQuery('$wildcard("tag")').single(data)// => ["run", "to-do"]newQuery('$wildcard("tag", false)').single(data)// => [{"tag": "run"}, {"tag": "to-do", "other": "task"}]$values -> List[any]$value_map(special, duplicate=true, *args, **kwargs): Dict[any: any]: Go through the values on the current item in the query, applying a special
to each one in-place. Ifduplicate=true, then the original value will not be modified. Similar to:# Pythonforkeyinvalue:value[key]=SPECIALS[special](value[key],*args,**kwargs)// JavaScriptObject.keys(value).forEach(key=>{value[key]=SPECIALS[special](value[key],...args,...kwargs);});Type Conversions$dict -> Dict[any: any]: Take an incoming list of(key, value)pairs and make a map/dict/object out of them.$fallback(fallback) -> value or fallback: If the value isnull, then it will be replaced withfallback.$float -> float$int -> int$not -> bool: Returnsnot valueor!value$set -> set$string -> str$ternary(if_true, if_false, strict=false) -> any: Returnif_trueif the value istruish, otherwise,
returnif_false. Passtrueforstrictif the value must betrueand not justtruish.Datetime$parse_timestamp -> datetime or moment: Take a Unix timestamp in seconds and return a corresponding datetime/moment object$strftime(fmt="%Y-%m-%dT%H:%M:%SZ" or "YYYY-MM-DD[T]HH:mm:ss[Z]") -> string: Format a datetime/moment object as a string usingfmt. Refer todatetimeormomentfor formatting instructions$strptime(fmt=null) -> datetime or moment: Parse a datetime string and return a corresponding datetime/moment object.
Iffmt=None, then standard formats will be tried. Refer todatetimeormomentfor formatting instructions$timestamp -> int: Dump a datetime/moment object to a UTC timestamp as the number of seconds since the Unix Epoch$time_part(part) -> int: Take adatetimeormomentvalue and get a specific part of
the date.partcan be:millisecondsecondminutehourdaymonthyeardayOfWeekdayOfYearMath / Numbers$abs(num) -> number$add(num) -> number$arith(op, arg_value) -> number: Take the value and use it as the first operand of an
expression with the specified math operator.opcan be any of themath operators.
The expression will bevalue op arg_value.$distance(other) -> float: Euler distance in N-dimensions$divide(num) -> number$math(attr, ...args) -> any: Returnsmath[attr](value, ...args), which can be used for
operations likefloor,cos,sin,min, etc. See thePython Docsfor validattrvalues in Python and theJavaScript Docsfor validattrvalues in JavaScript.$multiply(num) -> number$pow(num) -> number$round(n=2) -> number$subtract(num) -> numberStrings$lowercase -> string$prefix(prefix) -> string: Prefix the value with the specified string$replace(old, new) -> string: Replace all occurrences of a string$split(on=" ") -> List[string]: Split a string$strip -> string: Strip leading and trailing whitespace$suffix(suffix) -> string: Concatenate a string to the end of the value$titlecase -> string$trim(length=50, suffix="...") -> string: Trim the length of a string$wrap(prefix, suffix) -> string: Wrap a string with a prefix and suffix. Combines features of above two specials.$uppercase -> stringLists$group_by(key="", count=false) -> Dict[any: any[] | int]: Take an incoming list and group the values
by the specified key. Any valid JQL query or integer can be used for the key.""means the value itself.
The result by default will be keys to a list of values. However, ifcount=true, then the result will be keys
to the number of elements in that group.$index(key, fallback=null, extended=false) -> any: Index the current piece of data either using
a string or index key, or a full JQL query ifextended=true. If the index is invalid, thenfallbackwill
be returned. Ifextended=falseandkeyis an integer, then a negative key can be used to index from the
end of the array.$join(sep=", ") -> string: Join a list using the specified separator$join_arg(arg: List[any], sep=", "): Similar to$joinexcept this operates on an argument instead of the query value.
Essentially a shortened form of$inject(arg).$join(sep).$map(special, *args, **kwargs) -> List[any]: Applyspecialto every element in the
value. Arguments can be passed through to the special being used.$max -> number: Find the max value in the list$min -> number: Find the min value in the list$sum -> number: Return the sum of the items in the value$range(start, end=null) -> List[any]: Get a sublist. Defaults tovalue.slice(start)orvalue[start:],
but an end value can be specified. Negative indices are allowed.$remove_nulls -> List[any]: Remove any values that areNoneif Python, ornullandundefinedif JavaScript$sort(key="", reverse=false) -> List[any]: Sort an incoming list of values by a given key which can be any valid JQL query
or an integer. By default,key=""means the top-level value will be sorted on.Attributes$attr(attr) -> any: Access an attribute of the given object, implemented asgetattr(value, attr)andvalue[attr]$call(func, *args, **kwargs) -> any: Call a function that is on the current value, implemented asgetattr(value, func)(*args)andvalue[func](...args)FilterFiltertakes the power ofJQLand combines it with
filtering conditions to allow lists of items to be filtered down to just those of
interest. The filters can be manually built, or theKeyandConditionclasses can
be used to simplify your code.new Filter(filters, [convert_ints=true (if Python)], empty_filters_response=true, missing_field_response=false)filters:Condition | List[dict]The filters to apply to any data. IfList[dict], then the
filters should be formatted as shown below.[
    {"field": <field>, "operator": <op>, "value": <value>},

    OR

    {"or": <nested outer structure>},

    OR

    {"not": <nested outer structure>},

    ...
]
<field>: any valid `JQL` query
<op>: See list below
<value>: Any valid value for the operator, or 
    { "query": <JQL query> } to have a value which is a queryconvert_ints:boolCorresponds with the argument with the same name inQuery. Determines
whether digit only fields are treated as integers or strings. Defaults totrue.empty_filters_response:boolDetermines what gets returned when no filters are supplied.missing_field_response:boolDetermines the result of a filter where the field could not be found.Filters are evaluated in order and short-cutting can be used to avoid executing filters.
For example, apresentoperator can be used before trying to access or manipulate a field.
Additionally, the$fallbackspecial can be used to the same effect in certain cases..single(item, context={})context: SeeContextfor more detailsTake a single item and determine whether it satisfies the filters or notFilter(filters).single(...) -> boolean.many(items, context={})context: SeeContextfor more detailsTake a list of items, and returns only those which satisfy the filters. Note,Filter.many()introducesINDEXinto
the query namespace. Allowing items to be filtered by their 0-based index.Filter(filters).many(...) -> [result, result, ...]Notes{"or": [ 
    [ {filter1}, {filter2} ], 
    {filter3} 
]} === (filter1 AND filter2) OR filter3Nesting in anorwill cause those filters
to beAND'dand then everything in the toplevel of thatorwill beOR'd.Operators:><>=<===!====!==in:<field> in <value>!incontains:<value> in <field>!containssubset:x in <value> for x in <field>!subset:exists x in <field> such that x not in <value>superset:x in <field> for x in <value>!superser:exists x in <value> such that x not in <field>interval:<field> in interval [value[0], value[1]](closed/inclusive interval)!interval:<field> not in interval [value[0], value[1]]startswithendswithpresent!presentKeyIntended to simplify having to write{"field": <field>, "operator": <operator>, "value": value}a lot. The basic usage is:Key(<field>).<op>(<value>), or for the first six
operators, the actual Python operators can be used, soKey(<field>) <op> <value>.
For example:Key("meta.id").eq(12)is the same asKey("meta.id") == 12,
which is the same as{"field": "meta.id", "operator": "==", "value": 12}.To use a filter where the value is a query itself, useKey(<field>)as the value.
For example:Key("tags.tag1").gt(Key("tag.tag2")).The table below describes all of the functions which map to the underlying conditions.
Additionally, several other methods are provided to simplify certain aspects of
building filters and conditions:Key(field).is_true(): Same asKey(field).seq(true)Key(field).is_false(): Same asKey(field).seq(false)Key(field).is_null(): Same asKey(field).seq(null)Key(field).op(operator).value(value): Same asKey(field).<op>(value). Can be used for times when
you want to construct a filter and the operator is dynamic.Operators:underlying operatorKeyfunctionPythonoperator>gt><lt<<=lte<=>=gte>===eq==!=ne!====seqN/A!==sneN/Ainin_N/A!inninN/AcontainscontainsN/A!containsnot_containsN/AsubsetsubsetN/A!subsetnot_subsetN/AsupersetsupersetN/A!supersetnot_supersetN/AintervalintervalN/A!intervalnot_intervalN/AstartswithstartswithN/AendswithendswithN/ApresentpresentN/A!presentnot_presentN/AConditionIntended to be used in combination withKeyto make creating filters
easier than manually creating theJSON. There are three conditions supported:and,or, andnot. They can be manually accessed viaand_(...conditions),or_(...conditions), andnot_().The conditions can also be accessed through the overloaded operators&,|, and~, respectively, if in Python.Caution:&and|bind tighter than the comparisons operators and~binds the tightestKey("first_name") == "John" | Key("first_name") == "Bill"is actually(Key("first_name") == ("John" | Key("first_name"))) == "Bill", not(Key("first_name") == "John") | (Key("first_name") == "Bill")Condition.ander(cond1, cond2, *conditions)AND two or more conditions togetherCondition.orer(cond1, cond2, *conditions)OR two or more conditions together.and_(*conditions)AND the current condition with multiple other conditions.or_(*conditions)OR the current condition with multiple other conditions.not_()NOT the current condition.clone(deep=false)Clone the current condition. If deep, then all aspects will be duplicated..traverse(callback, on_duplicate=false) -> ConditionTraverse through all of the filters in the current condition, callingcallback(filter)with each one.on_duplicateindicates whether the current condition is iterated over, or a duplicate.
Whatever condition was iterated over will be returned..to_list() -> listReturn a list of filters/conditions for this conditionCondition.from_list(conditions) -> ConditionTake a list of filters/conditions and make a Condition object out of itExamples# PythonKey("state").eq("Texas")|Key("city").eq("New York")(Key("gender")=="male")&(Key("age")>=18)&(Key("selective_service")==False)Key('creation_time.$parse_timestamp.$attr("year")').lt(2005).or_(Key('creation_time.$parse_timestamp.$attr("year")').gt(2015)).and_(Key("product_id")==15)# (year < 2005 OR year > 2015) AND product_id == 15// JavaScriptKey('creation_time.$parse_timestamp.$call("year")').lt(2005).or_(Key('creation_time.$parse_timestamp.$call("year")').gt(2015)).and_(Key("product_id").seq(15));FormatterFormatterallows fields to be queried from an object and then formatted
into a string. Any queries in a format string should be prefixed with@and any validJQLquery can be used. For example,new Formatter('Name: @name}').single({"name": "John Smith"})results inName: John Smith.Formatter(spec, fallback="<missing>", [convert_ints=true (if Python)])spec:strThe format stringfallback:strThe value that will be used in the formatted string if a query could not be performed. For example, if the fieldmissingdoes exist, then the query"Age: @missing"will result in"Age: <missing>"convert_ints:boolWhether digit-only fields get treated as integers or strings.single(item, context={})context: SeeContextfor more detailsReturn a formatted string or the fallback value if the query fails.many(items, context={})context: SeeContextfor more detailsReturn a list of formatted strings or the fallback value.NotesThe differences betweenQueryandFormatterare:Querycan return a value of any type,Formatterjust returns stringsFormattersupports multiple queries, end-to-end,Querydoes notAll queries must be prefixed with@withFormatter, not just when used as an argument like withQueryBoth support all the features ofJQLQueryactually can theoretically do everythingFormatterdoes by using$prefix,$suffix, and$string.
For example,'@name @age'->'name.$suffix(" ").$suffix(@age)'. However, the latter is much longer than the formerExample (flattening operations):# Pythonerrors={"errors":{"Process Error":"Could not communicate with the subprocess","Connection Error":"Could not connect with the database instance"}}Formatter('Errors:\n@errors.$items.$map("join", ":\\n\\t").$join("\\n")').single(errors)# Errors:# Process Error:#   Could not communicate with the subprocess# Connection Error:#   Could not connect with the database instance// JavaScriptconsterrors={errors:{"Process Error":"Could not communicate with the subprocess","Connection Error":"Could not connect with the database instance"}};newFormatter('Erros: @errors.$items.$map("join", ": \\n\\t").$join("\\n")}').single(errors);// Errors:// Process Error://   Could not communicate with the subprocess// Connection Error://   Could not connect with the database instanceThe above example shows a powerful usage of flatteningerrorsinto its items,
then joining each item; splitting the error name and message between lines, then
joining all the errors together.Example (nested replacement):# Pythonitem={"x1":1,"y1":1,"x2":12,"y2":54}Formatter("Midpoint: [@x2.$subtract(@x1).$divide(2), @y2.$subtract(@y1).$divide(2)]")# Midpoint: [5.5, 26.5]// JavaScriptconstitem={x1:1,y1:1,x2:12,y2:54};newFormatter("Midpoint: [@x2.$subtract(@x1).$divide(2), @y2.$subtract(@y1).$divide(2)]").single(item);// Midpoint: [5.5, 26.5]PerformanceThere are several ways to increase the performance of querying, filtering, and formatting. The performance gains can be had
by limiting the amount of times a query string has to be parsed. This means that using aQuery,Filter, orFormatterobject multiple times will be faster then creating a new object every time.Examples:# Python# slowerforiteminitems:f=Query("timestamp.$parse_timestamp").single(item)# do other stuff# fasterquery=Query("timestamp.$parse_timestamp")foriteminitems:f=query.single(item)# do other stuff// JavaScript// sloweritems.forEach(item=>{letf=newQuery("timestamp.$parse_timestamp").single(item);// do other stuff});// fasterconstquery=newQuery("timestamp.$parse_timestamp");items.forEach(item=>{letf=query.single(item);// do other stuff});Across 10,000 runs:PythonreusingQuerycan improve performance by 302xreusingFiltercan improve performance by 132xreusingFormattercan improve performance by 377x.JavaScriptreusingQuerycan improve performance by 192reusingFiltercan improve performance by 120xreusingFormattercan improve performance by 210x.Changelog1.1.5UnifyREADMEbetween Python and JavaScript versionsExpand documentation1.1.4Added$value_map, which allows the values on an map/dict/object to be modified with a special, either in-place
or on a duplicateExposedcontextso additional fields can manually be put into the current query space. This was already
being used by$store_as.contextcan be passed to any.single()or.many()call.Additionally,Filter.many()is now placingINDEXinto the query space to allow
items to be filtered by their 0-based indexExposed many of the internal TypeScript types1.1.3Changed the behavior ofnew Query(""), from returning the fallback value, to returning the source data element itself.
For example,new Query("").single(data) === data.AddedSpecialNotFoundError, which is raised when an invalid special is queried. Can be imported asimport { SpecialNotFoundError } from "@blending_jake/jtools";Added new specials$store_as(name)Store the current query value in the current context for later use in the query. This does not
change the underlying data being queried.$group_by(key="", count=false)Take an incoming list and group the values by the specified key.
Any valid JQL query can be used for the key, so""means the value itself. The result by default will be
keys to a list of values. However, ifcount=true, then the result will be keys to the number of elements with each
key.$sort(key="", reverse=false)Sort an incoming list of values by a given key which can be any valid JQL query.
By default,key=""means the top-level value will be sorted on.$dictTake an incoming list of(key, value)pairs and make a dict out of them.$join_arg(arg, sep=', ')Similar to$joinexcept this operates on an argument instead of the query value.
Essentially a shortened form of$inject(arg).$join(sep).Changed the underlying special function definition to now include the keyword argumentcontext. This argument is
implemented to only be accessed by name to avoid collision if the user provides too many arguments in their query.
The purpose of the context is to support specials adding values temporarily to the data
namespace of the query, like$store_asdoes.1.1.2Version1.1.1was skipped to keep on track withJTools-PyCatch and handleExtraneous Input ErrorChangeJQLso that field and special names must only contain[-a-zA-Z0-9_].$indexcan be used to get fields
with prohibited characters. The change was to support more formatting use-cases, likeAge: @age, DOB: @dob, which
previously would have failed because the,would have been considered part of the field name.ChangeFormatterso thatfallbackis just a string that is substituted for invalid queries, instead of being
the entire return value. Previously,"Age: @missing"would result inNone, now it results in"Age: <missing>".
This change allows for better debugging as it becomes clear exactly which queries are failing.Add function docstrings1.1.0RenameGettertoQueryto more accurately describe what the class doesMigrate queries to useJQLThe migration opens the door to nested queries inQuery, allowing queries, prefixed with@to be used
as arguments to specials, or even as values in the supported argument data structuresSpecial arguments are no longer parsed asJSON, allowing features like sets, query nesting, and support
for single and double quoted strings.Formatter no longer uses{{}}to surround queries. Instead, all queries must be prefixed with@, so"{{name}} {{age}}"->"@name @age".@@must be used to get a literal@in a formatted string:"bob@@gmail.com"->"bob@gmail.com"Formatter got about a 2x performance boostAdded$wrap(prefix, suffix)to combine$prefixand$suffixAdded$remove_nullsAdded$lookup(map, fallback=None)Added$wildcard(next, just_value=True), which allows level of nesting to be "skipped", such that a list
of sub-values wherenextis presentAdded afallbackargument to$indexAdded$printto display the current value in the queryAdded$injectto allow any valid argument value to be injected into the query to be
accessed and transformed by subsequent fields and specials1.0.6Migrate to TypeScript, so a declaration file is now included in the distributionAdd thisREADMEAdd===and!==filters for strict equality checking. The methodsseqandsnehave
been added toKeyto correspond with the new filters.Renamenull->!presentand!null->present. Corresponding methods have been renamed
tonot_presentandpresent. This filter will catch values that arenullorundefined.Make membership filters (in,contains,!inand!contains) work properly with
strings, arrays, associative arrays, and sets.Remove$datetime. See below for replacement.Add$calland$attrfor calling a function and accessing an attribute. Can be used to replace$datetimefunctionality.RemoveFormatter.formatand addFormatter.singleandFormatter.manyto be consistent across
other classes and support formatting arrays of items.Add more tests to increase coverage and do basic performance testing