Change:
Fixes URL
Merging to unblock other editorial work.
Even after the clarification of URL, comparison of two structured values always produces a "false" comparison result. This includes the following cases: Comparing two equal structured values with gives "false". Comparing two equal structured values with gives "false". Comparing two unequal structured values with gives "false". Comparing two unequal structured values with gives "false". With these semantics, the following law of comparisons does not hold: if and only if
I think this is worth declaring explicitly while we don't allow structured values. If/when we move to support them, we would need to remember to remove this.
NAME I'm not sure what you mean by "this". I would favour changing the spec so that the above law does hold in all cases._
Yes. I agree. This topic. All of the above. !
I've attempted to fix this in URL
I don't understand why the draft doesn't support sensible equality comparisons between arbitrary JSON values, the Committee doesn't have to invent anything, there is prior experience that covers this. Jayway, possibly the most widely used of the current implementations (to many Stack Overflow users, Jayway is JSONPath), has no difficulty with these comparisons, or, for that matter, comparisons with array and object literals.
IIRC we discussed this on more than one call and after considering the options, we settled on the current behaviour. I'm afraid I can't remember the rationale, but maybe NAME or NAME can. See: URL URL
I would describe the rationale as focus on shipping the MVP. I agree with Daniel that it is a quite limited amount of work to support equality for all JSON values. Inequalities are more work, less general (as long as excluding normalized comparison), and don't have a good solution for maps, but also could be done. If we find that we are creating a major interop issue by the MVP approach, I think we should still be open to addressing general equality (and inequalities). We'll have to solve the absent value problem in any case.
What I didn't mention is the not quite insignificant accretion of new grammar to make literals for all of JSON.
(This is a follow-on from URL) URL covers comparing absent with absent using == and !=, but we still haven't decided what to do about the likes of "absent >= absent". The spec currently says: This doesn't make it clear what to do when comparing two singular paths which each select no value. I can see two options: We might want "absent >= absent" to be true because "absent == absent" is true. We might say that "absent" isn't a numeric value and so "absent >= absent" is false. I prefer the first option as it preserves the usual law that "x >= y" if and only if "x >y" OR "x ==y". (I guess we might want to add such laws to the spec too!)
I, too, prefer the first option.
If your model is Javascript and "absent" has the same properties as "undefined", then inequality operators wouldn't be supported. If your model is a language that supports the notion of "optional", I can't think of any that support inequality operators for values of that type. But on this, the draft really needs to define its type system more precisely, as introducing the notion of "absent" extends the type system beyond JSON.
According to the spec, the results of singular paths are nodelists consisting of at most one node. So one way to address this issue would be to say that: when a nodelist consisting of a single node appears in a comparison, it is replaced by the value of the node, and when an empty nodelist appears in a comparison, the result of the comparison is true if and only if the comparison is being made with another empty nodelist and the comparison operation is , or .
You also have to think about what it means for an expression that evaluates to "absent" to be passed to a function (assuming the draft supports functions.) By comparison, in Javascript, a value of type undefined is passed to the function.
Noted in URL
Currently the syntax section of the "Filter Selector" section describes some semantics. These descriptions should be folded into the semantics section.
If yes, is false. Need to discuss impact on >, >=, etc. See also: URL
I've extended the query which compares two paths via equals to include absent keys: URL No consensus, not even a majority at this point.
... which is not necessarily a problem: I gives us more leeway to nail down something sane (if we want to).
... with JavaScript ... due to .
Isolating the empty key behavior, I think the results are consistent with what we understand about the implementations. By my count, 12 implementations select the item. It depends on whether the implementation returns a value (such as ) for an absent key, or whether it handles it as a recoverable error. The Javascript ones that return for an absent key select , as would be expected. Jayway, which handles an absent key as a recoverable error that takes out the term, does not select , again as expected. . I think it would be interesting to add to the document.
Done :)
I would like to add here, just to ensure that it's documented, that C# recently added as an expression equivalent to . Therefore returning could be quite confusing for some users.
Okay, from the data (checked cells correspond to selected values), it appears that twelve of 42 implementations consider absent is equal to absent (column 2), while one, PHP (galbar-jsonpath), considers is equal to absent (column 5), and absent to (column 6), but not absent to absent (column 2). The ones that consider absent is equal to absent do so for different reasons. Cpp (jsoncons) and dotNET (JsonCons.JsonPath) evaluate absent to be , and is equal to . The JavaScript implementations evaluate absent to be , and is equal to . Three of the JavaScript implementations also show equal to and equal to , no doubt because of conversion when comparing and . Implementation ---------------------------- Clojure (json-path) Cpp (jsoncons) dotNET (JsonCons.JsonPath) dotNET (JsonPathLib) dotNET (Manatee.Json) Elixir (warpath) Erlang (ejsonpath) JavaScript (brunerd) JavaScript (Goessner) JavaScript (jsonpath) JavaScript (jsonpath-plus) Ruby (jsonpath) PHP (galbar-jsonpath) We can deduce that the remaining 30 of 42 implementations do not consider absent equal to absent. Again, the reasons vary. 15 implementations either fail or return an empty collection. While there is more going on in this test than "absent equal to absent", the number 15 lines up with the number in URL that fail when attempting to evaluate an absent member. So for these, it looks like absent is always an unrecoverable error. When Jayway evaluates with absent and absent , it considers the first absence a recoverable error, and it will branch to some code that ignores this comparison, so this comparison selects nothing. That applies to all comparators, , , , .
112 minutes: So we need to discuss this.
By , do you mean literally an IEEE 754 , to have evaluate to that? First, I don't think any existing JsonPath implementation does that. Second, not all mainstream programming languages require IEEE 754 or . Generally speaking, notions such as or tend to be language specific. Some languages support as a primitive value for when a variable is declared but not initialized, some support something similar for , some want to make it impossible to have a variable that is not initialized. There are alternatives, It isn't necessary to have evaluate to a "value", whether , or . can be considered an error, resulting in the term in question being unmatched. That's the approach of JAVA Jayway, and the implementations that followed Jayway.
NaN is just a placeholder for a value representing “absent" that behaves like NaN: It does not compare equal to anything, including itself. This obviously is a weird value to have, and I believe it was introduced in IEEE 754 as a hack to make it safer to reuse existing code with that new standard, not because having such a value is such a great concept. Right. But that is not relevant to the discussion at hand. “Undefined” again is a placeholder for any non-JSON value representing “absent" that does compare equal to itself. NAME == NAME This is hard to do with throwing an exception. There is nothing wrong with a value representing “absent”. “undefined” and “NaN” were just figures of speech for values representing “absent” that do or do not compare equal to themselves. Grüße, Carsten
Thanks for clarifying. Like SQL NULL then. Do you think so? Personally, I would be fine if both evaluated to no results (although in Jayway they do not.) It's unnatural in some programming languages. A more universal notion is optionality, a result that may or may not have a value. Daniel
Note that by introducing as a value to represent absent, you've effectively extended the JSONPath type system to be more than JSON, JSON does not have the notion of . In JavaScript JSONPath implementations, you can write (returns everything) and (returns nothing), do you plan to support that notation as well? Or do you plan to only allow the construction of an indirectly as the result of evaluating ? Regardless, you've still introduced a non-JSON type. What happens in a future version of JSONPath that supports functions of expressions, like all the current JavaScript ones? Do they accept as an argument, like JavaScript? More things to reason about. Of course, there is plenty of prior experience that bears on these matters. JMESPath stays entirely within the JSON type system as a design principle, and represents the result of as a JSON null value. A couple of JSONPath implementations (a distinct minority) also take that approach. JSONPath implementations that use a scripting language having as a primitive type usually use for absent. The most important and widely used implementations that implement their own expression language, notably Jayway, handle absent as a recoverable error affecting that term.
JSON doesn't have , but it does have a distinction between and absent, which is what represents. It doesn't need to be a "value" that's recognized externally; we intend it as an internal placeholder that indicates the absence. The question we're raising here is, internally to an implementation, how the absence of one property equate to the absence of another property? This will not be a supported syntax in JSON Path. Instead we are using implicit existence: for presence test and for absence test.
We don't have a PR for this issue yet, but I don't think we need to introduce in the specification. This is just a figure of speech to have an reasonable conversation about this issue. Yes. This is good. I don't see a point in providing a literal notation for the absent condition. I would expect so. In my eyes, that would be a bug; a null value is different from absence. Curious: Where are these throws caught ("recoverable")? So far, we have been able to avoid throw/catch, and I wouldn't mind if it stayed that way.
Thinking further on whether needs to have a value or not, would we also expect the following to be true?
Both don't exist, so they are equal, so they are the same, so no.
Argh! I meant would we also expect that expression to be false. But my point here is that the undefined value is acting more than a figure of speech here because when we apply selectors to it (such as [0]), it is preserved. I'm not sure how we could explain that propagation behaviour without there being an undefined value of some sort, even though it may be implicit. Of course, a clean way of defining this would be to say that each selector produces an optional nodelist with an "absent" nodelist which is preserved by other selectors. As was pointed out earlier, we don't need to have an explicit representation for the "absent" nodelist, but we do need to talk about how the "absent" nodelist behaves. Does this help?
Yes, that (empty input empty output) is part of the definition of the path selectors, and maybe needs to be made more explicit there. It is not an aspect of the filter expression language.
Your understanding seems to be closer to c++ , rust , or c# . Something with and properties, considered abstractly. From a conceptual point of view, I think that's better, because it's less Javascriptish, not many consider Javascript to be a great design, and all values remain JSON. Daniel
The nice thing about the JMESPath convention is that it leads to a simple consistent model that stays entirely within the JSON type system. Regarding how this corresponds to user intuition, of all the JMESPath issue logs and all the JMESPath discussions that I've followed, I've never seen this come up. But I think the same can be said of other conventions. Users care about many things, but they don't care about this, whatever the convention is, they figure it out and move on. Most languages would consider having both and as having one thing too many; some (rust, functional languages) would consider that as having two things too many. Daniel
In Javascript, that would result in an exception being thrown, with message "Cannot read properties of undefined"
But this is not JavaScript. From context, this seems to be meant as a part of a filter expressions. LHS and RHS of are two path constructs. These don’t have any values but nodelists; so I don’t know where (or any other surrogate for absent) comes in. The results of these two paths then are used in a comparison, which causes the JSON value at the node selected by each path is obtained. This is where this concept of (absent) comes in, because the nodelists are empty, and therefore there is no such JSON value, so we need to remember that the input to the comparison is something else. Grüße, Carsten
In most prior experience that I know of in JSONPath implementations, the LHS and RHS in filter expressions are expressions that evaluate to single values (null, true, false, string, number, array, object and in some cases undefined), not result lists. These JSONPath implementations group largely into two categories, Implementations that use Javascript/Python/Lua to evaluate filter expressions, where the symbol in the script is substituted with the current node value, and is evaluated as a Javascript/Python/Lua expression, which evaluates to a single value (no result list.) Implementations that evaluate as a JSONPath expression, but using rules so that the result is produced as a single value rather than a result list. Jayway JSONPath is an example, when evaluating in a filter, the slice, union and filter selectors will always produce results wrapped in an array, the wildcard selector will produce results wrapped in an array if there are more than two items, the identifier and index selectors will produce a single value if matched, otherwise a null. It is done that way in the filter even if the final result returned to the user is a result list (Jayway supports both single value and result list returned to the user.) Between the Javascript/Python implementations and Java Jayway , that accounts for most of the JSONPath implementations in use. Do you know of any prior experience where the LHS and RHS in filter expressions evaluate to result lists (node lists)? Thanks, Daniel
In JavaScript ... ... with new optional chaining URL
I wasn't talking about implementations, but about the processing model we have adopted.
Interesting. So, given If resolves to a "result list", , how is the result list containing compared to the single value ? What convention are you following to support comparison between a "result list" and a single value? I apologize if I missed that in the draft. In all the prior experience that I'm familiar with, always evaluates to a single value according to rules. That's true of both implementations that treat as a Javascript like variable, much in fact like it is in the draft, and in implementations where the grammar of is the same as . (For these implementations, is always evaluated to a single value following rules that may differ from evaluating to a result list.) So your approach sounds like something different. Thanks, Daniel
We have rough consensus that absent does equal absent. Tagged as editorial.

Discussion:
Comparisons are restricted to Singular Path values and primitive values (such as number, string, "true", "false", "null"). <del> Comparisons with complex values will fail, i.e. no selection occurs. </del> Data types are not implicitly converted in comparisons.  So ""13 == '13'"" selects no node.

element or member value as the filter selector iterates over the array or object. <del> Comparisons using one of the operators "<", "<=", ">", and ">=" are between numeric values only.  Using these operators to compare other types of values produces a "false" comparison result. </del> <ins> When a path resulting in an empty nodelist appears on either side of a comparison, the result of the comparison is true if and only if the comparison operator is "==", ">=" or "<=" and the other side of the comparison is also a path resulting in an empty nodelist. When no path resulting in an empty nodelist appears on either side of a comparison, any path which appears on either side of the comparison and results in a nodelist consisting of a single node is replaced by the value of the node and then: comparison using one of the operators "==" or "!=" produce a "true" comparison result if and only if the comparison is between: primitive values which satisfy the comparison, or structured values compared using "!=". comparisons using one of the operators "<", "<=", ">", or ">=" produce a "true" comparison result if and only if the comparison is between numeric values which satisfy the comparison. Note that comparisons between structured values, even if the values are equal, produce a "false" comparison result. </ins> The semantics of regular expressions are as defined in I-D.draft- bormann-jsonpath-iregexp.