Draft: draft-ietf-jsonpath-base

Commentary: ( No semantic change ) This removes the weird diamonds we have been tangled in . 
 My argument for a distinction ( related to ) . When we first started discussing path expressions ( , I think ) , we decided that was disallowed as a complete expression , e.g. . This was intended to disambiguate from being either an existence test or a check that the value in is ( e.g. ) . As a result of this decision , the literal was only valid as a ; that is , it can only be used with a comparison operator , and a path such as is syntactically invalid . It also allowed the path to return a node for even though the value that 's returned by is . Without this distinction , would return  , be evaluated as Test - false , and no node would be returned . This means that there must be some distinction between the result of and the literal , although the colloquial meaning for both is " true . " In the interim call , I gave the example that if these are the same , then should be valid . Here 's why . If is the same thing as the literal , then I can do substitution and a valid expression should result . Since is a valid expression , it is implied that is a valid expression by substituting back the other way . But the ABNF forbids such an expression , so we have a contradiction , which implies our original assumption ( " is the same thing as the literal " ) must have been wrong . Pedantically , the literal is just that : a literal . It 's no different than the literal . It should n't carry any meaning of truthiness behind it , but it is commonly used that way . In terms of our expressions , it should be used as a mere token . is asking if the value in is the JSON token . The comparison returns a " true " or " false " result , an actual boolean value rather than just the JSON token , which can then be interpreted as the final result of the expression , or used as an operand for a logical operator , which itself returns a boolean to be interpreted as the final result of the expression . 
 I also want to explore the idea of " lowering " a TestBoolean to a JSONBoolean so that it can be used in a comparison . This idea is that we can go from a TestBoolean " true"/"false " to the JSONBoolean / , respectively , but not the other direction . This does retrict the literals and from being used in a logical expression ( e.g. ) . However , if TestBoolean can be lowered to function as JSONBoolean , then can be lowered to and can be lowered to , still implying that is valid . I think the only recourse here is to keep them distinct ideas . 
 I agree there should be a clear distinction between JSON booleans and the truth values associated with and . I believe the WG consensus supports this , as does the spec . 
 Good . Progress . No . The spec does not support this . The spec says that a function which returns can be used in both and . But you just said above that there needs to be a distinction between these . See my response to in as to why can not represent JSON booleans . I do n't feel like making that argument again . 
 ( Please also see the discussion in and , which is quite useful to find the root cause of this disagreement . ) 
 I think this relates to . An expression boolean ( e.g. the result of ) is not a JSON value . This is something that we decided long ago . However Table 13 describes it as " or " . This is wrong . If we allow to be a subtype of , then paths like and will be valid and produce no parsing errors . In ( see also ) , we decided that this would n't be the case . only operates as a boolean , and only operates as a value . and MUST be distinct in order for this to work . This issue is about the role a function plays within an expression . That role is determined by its return value . 
 Since we do n't allow as a function argument , I do n't think has a type . 
 I do n't see how that 's relevant . I 'm not talking about argument types . I 'm talking about return types and the role of functions within expressions . returns an expression boolean ( not a JSON boolean ) , which is valid where booleans are valid in expressions , i.e. not in comparisons . returns a JSON - like value , which is valid where values are valid in expressions , i.e. only in comparisons . 
 This distinction is implied because of the decisions that we 've already made in the issue and PR linked above that we are n't going to support type coercion that a JSON- is distinct from expression - true 
 From : If we allow to be a subtype of , then the above decisions are contradicted . 
 In the interim we discussed that a function can return one of three kinds of data : a value ( e.g. ) a boolean ( e.g. ) a nodelist ( e.g. a hypothetical function ) Value functions can only be used in a comparison ( i.e. as a ) . This was n't disputed . I argue that boolean functions can only be used as operands in a logical expression . Effectively they are . The value that they return can not be translated / lowered / casted / converted / coerced into the JSON literals and and so they are not . ( This discussion prompted , as linked above . ) We do n't have any functions that return a nodelist currently . The hypothetical function ( or something like it ) that takes a nodelist and returns that nodelist with duplicates removed was proposed during the interim . A use case could be for finding elements with more than x distinct values , like . The key here , though , is that this would only ever be used as : an argument for another function ( as above ) , which is n't part of this discussion a , which is the same as the boolean return scenario ( because nodelists are n't ) . The role of this return type is the same as a boolean return type . What I want to highlight from this discussion is that a boolean is n't a value , so Table 13 is wrong when it lists as a subtype for as a subtype for See also There was an interesting scenario that was mentioned in the interim : . ( This may need to be moved to another issue , but it came up during this discussion , so it 's here . For now , I just want to explore the use case . ) This finds all elements where contains the correct result of whether matches , for example in this data : ( The inverse is arguably more useful , but that aside . ) Importantly , this is only possible if the result of can be translated / lowered / casted / converted / coerced into the JSON literals and to be considered as . ( Again , see . ) I can see a use case for something similar in the test suite actually . Let 's assume we have an function that takes a path and some data and returns the nodelist result of evaluting the path against the data . ( We 'll disregard any security or other practical issues of doing that for this purpose of this example . ) The test suite is represented as an object with a single key that contains an array of scenarios . Each scenario looks something like this : Now suppose we wanted to query this document for all of the test cases which are incorrect . We could use this path : Ideally , this would return an empty nodelist for a compliant implementation . Something like this would be a great  sanity check for the test suite . ( This exact path would imply that nodelists are , which they 're not , but we could work around that by introducing a function or something . This problem does n't apply to the example since that 's attempting to compare things that are more " boolean"-y . ) 
 I 'm not sure what " it " is referring to , but Table 13 says that has abstract instances and . Table 13 does n't mention expression booleans . 
 You ca n't use Table 13 in your argument . The basis of my claim is that the table is wrong . " It " in my comment is the " expression boolean , " or what we started calling TestBoolean . 
 I am trying to understand the second paragraph in the of this issue . After substituting " expression boolean " for " it " , the paragraph reads : I do n't understand how Table 13 says anything at all about expression booleans . Please can you explain ? 
 ... pedantry ... okay . It should be I 'm saying is what we 're now calling TestBoolean , which is not a JSON value . 
 I disagree that represents a " test boolean " . The spec makes this clear : The tricky thing , which is what I suspect we should be focussing on , is in the following : A test expression either tests the existence of a node designated by an embedded query ( see ) or tests the result of a function expression ( see ) . In the latter case , if the function expression is of type or one of its subtypes , it tests whether the result is true ; [ ... ] Note that the special case in this paragraph only applies where the return type of the function expression is defined to be ( or one of its subtypes ) . The special case is not triggered by a return value being a boolean value ( such as ) . In other words , it can be determined before the query is executed . 
 I think this is an important point -- while  values are dynamically typed ( there are n't really JSON types anyway ) , the OptionalBooleanType is statically confined to have three members : Nothing ,  ) , Value(true ) . So it is compatible with an OptionalValueType recipient , and with being used in a comparable . It is also compatible with being used in a test - expr . 
 ( And the point is that the function expression type system is statically typed . ) 
 You keep using the spec as evidence . I 'm arguing that the spec is wrong . You ca n't use it as evidence . I 'm saying that the spec should n't define this way . How is compatible with if it represents a JSON value ? This is the argument in . You 're saying that a value of ( // ) can be used in a . This means that can be used in a because is , which implies is a valid expression . We 've already decided that this is an invalid expression , as linked above , so we have a contradiction . The only way to resolve this contradiction is to make ( and by extension ) akin to 's TestBoolean , not the JSON values and . Taking this further , can not be a subtype of and by extension can not be a subtype of . These are disjoint and operate under different roles in expressions . Additionally , if truly does represent  and , why make a type for it and not make and ? This definition is an incomplete type system . Further , why make the type system stop at functions ? We 've defined these types for functions , but then they do n't extend into the expressions in which they exist , when it 's clearly beneficial to do so ( even if it requires more work ) . 
 Hi Greg , we now know that you want TestBooleans to be distinct from JSON value Booleans . The only argument that you present is that you want them to be different . This is fine ; intuition often is useful in designing things . However , other people have the inverse intuition , so we wo n't get around discussing this with more technical arguments . The purpose of the type system we have been designing is not to replace or augment the JSON type system ( which does n't exist , I 'd argue , but that is a different discussion ) . The purpose is to be able to statically check how functions fit together , and how arguments for functions can be derived from JSONPath filter expressions and how returns from functions can feed back into JSONPath filter expressions . It turns out we have three places where function expressions fit : In a test - expr , in a filter - path , and in a comparable . We would n't need to list filter - path here ; for some reason we have made this one of the two things that can go into a function argument , along with comparable . This anomaly in the grammar can probably be fixed . The current definition of the function expression type system is clouded by the fact that there is both a subtype relationship and a compatibility relationship ( more specifically , a type can be used in place of another type , which involves coercion of the value ) . We care about : nodelists , which can also stand in for a test ( by checking whether the nodelist is empty ) single nodes ( which almost always need to be optional ) , which can stand in for a value values , which can also be used as comparable ; these are often , but not always Optional . Functions like match or search return a special kind of optional value , which can be used in test expressions , and as a comparable , and as an argument for a function that expects just such a an instance and/or a JSONValue . You do n't want them to be used in comparables or as function arguments where a JSONValue can be used . Why . ( Note that you ca n't argue from the grammar , as that does n't express the function argument / return value type system . Of course , you can ask why literal true and literal false can not be used in a logical expression , and I 'd say that this is a little wart , but it also is n't particularly useful . ) 
 No , I 've presented quite a lot of logical evidence that shows they MUST be different . What we have now is rife with contradiction . I agree with this stated purpose . We have failed to fulfil this purpose . What we have failed to do is identify when it 's appropriate for a function to appear in each of these places . An individual function needs to be identified as valid as either or ( XOR if you prefer ) , but it ca n't be valid as both . My evidence is , which ( as currently defined ) is capable of returning  or . These are s. Everyone is saying that can be used as a , but  and can not be used as . ( That 's the contradiction . ) Because has the potential to return these JSON values , the function MUST be restricted to locations where those values are valid , namely . This would mean that MUST be invalid . If returns here , it reduces to , which we have declared invalid in in order to resolve an ambiguity . How are you not seeing this very obvious contradiction ? 
 The solution to making valid is to make it return what calls a " TestBoolean , " optional or not . But that means it 's no longer valid as a and or even become invalid . 
 Note that the first half of this sentence is about the type system , and the second appears to be about the grammar ( which just does n't allow literals as test - expr ) . I tried to clean up the grammar ( without changing it ) over in so that may become more obvious . 
 The grammar and the type system need to align . 
 That is a desirable , but there are some other desirables in conflict with that . The grammar can very well disallow confusing expressions that do not violate the type system . 
 Like what ? What desirable can be so important that it overrides a logical contradiction ? 
 I 'm saying the type system is allowing something that the grammar disallows . 
 What is obvious to you is far from obvious to me . My starting point is that the aspect of the spec being scrutinised by this issue is actually consistent and maybe just needs tweaking or explaining better . I 've to point out what I believe to be the root cause of the confusion . ( I am now essentially AFK for a while , so apologies for any delayed responses . ) 
 The underlying requirement is that we want to convert Nodelists into Booleans when used in a test - expr ( true if non - empty ) . We want to convert OptionalNodes into  ( by looking up the JSON value ) when used in a comparable . We somehow need to map this into the function - expr type system as well . Since a function - argument says whether it is a nodelist or a value , this transfers one to one . Smearing up the type system by equating subtyping with coercion does not help , though . I would prefer to have the conversion from a Nodelist to an OptionalNode be explicit ( possibly supported by a function that does just that ) . Then we do n't have an unclear situation when a Nodelist is offered as an argument to a function that declares this as a Boolean . 
 / I think I have a good explanation of the problem . In this post , I will be using angle brackets to denote a nodelist to distinguish it from a  array and ( code formatting ) to indicate JSON literal values true and false ( italics ) to indicate the operands of a logical operation I would also like to use a couple definitions for expressions : A comparison context is any that involves a comparison operator , like and . A logical context is any that either can be used as the final result of an expression or involves a logical operator , , , or . These are the only two contexts which exists in our expressions . Further , it 's important to recognize that the result of a comparison is used in a logical context . We decided that in a logical context was to be interpreted as an existence test . This means that if an item had an property , regardless of its value , the result of would be a logical true to select both items in To illustrate why , consider an implementation that interprets the value at as the result of the expression . It would consider the  as logical true and select the first item , but it would consider  as logical false and not select the second . With this behavior , there is be no way to select the node with , so we decided this was to always be interpreted as an existence test . This showed that  values , even and , only have meaning in a comparison context . Thus we made a distinction between the JSON literals and and the logical values true and false . We then codified this decision by making  literals ( , , ) and other raw JSON values ( strings , numbers ) only valid in a comparison context using the ABNF . These functions are defined to return , which is defined by Table 13 to be the JSON literals and and the value . Because they return JSON values , and following from the decisions above , they can only be used in a comparison context . That is , they must be compared to another JSON value using a comparison operator to yield a logical result ; they can not be used directly in a logical context . But we also want these functions to be valid in both a comparison context and a logical context . This is the contradiction . They way they 're defined , they can not be used in both contexts . My original suggestion was to have the functions instead return a logical true or false instead of the JSON values . However , this only reverses the contraction rather than solving it . This weekend , I thought about how to define them so that they work in both contexts . I started by analyzing a construct that already does : a path . can exist in both a comparison context and a logical context , and it always returns a nodelist . How that nodelist is interpreted in each context determines the different behaviors it exhibits . To illustrate , let 's take a look at what happens for in a comparison context with ( 1 ) and in a logical context ( 2 ) when evaluating Most importantly , always returns and , respectively , for these items , regardless of the context . a comparison context , the single value is extracted from and -compared with , and the node is selected . is converted to and -compared with , and the node is not selected . a logical context , contains a node , so it evaluates to logical true , and the node is selected . contains no nodes , so it evaluates to logical false , and the node is not selected . returns the same nodelist for both contexts because it 's unaware of that context . It 's just a path evaluating data . While always returns the same nodelist , how that nodelist is interpreted changes with context . Specifically , in a comparison context , is interpreted as JSON . in a logical context , is interpreted as logical true . In order to make these functions valid in both contexts , they need to return something that is valid in both contexts , and the only thing we have that works in both contexts is a nodelist , as demonstrated by . Let 's perform our analysis again for the comparison context with ( 1 ) and the logical context with ( 2 ) . Theoretically , these should select the same nodes , namely nodes that DO NOT match . Starting with the affirmative case where the function identifies a match , let 's say that returns . a comparison context , the single value is extracted from and -compared with , and the node is not selected . a logical context , has nodes so is evaluated to logical true and negated by , and the node is not selected . Good so far . Both paths return the same thing for cases where a match is found . Now , let 's look at the negative case where the function does not identify a match . For this , let 's say that the function returns . a comparison context , the single value is extracted from and -compared with , and the node is selected . a logical context , has nodes so is evaluated to logical true and negated by , and the node is NOT selected . Uh , oh . We have differing results . This does n't work . Let 's try returning for the negative case instead . a comparison context , is converted to and -compared with , and the node is NOT selected . a logical context , has no nodes so is evaluated to logical false and negated by , and the node is selected . Hm ... that does n't work either . In fact , there is NO SINGLE VALUE that can return to make these paths select the same nodes for the non - matching case . The only recourse , then is to restrict to only one of : comparison context : logical context : and ca n't support both . Doing this would mean that when appears outside of its decided context , it will not be considered well - formed , requiring a parsing error ( or compilation error , if you prefer ) . Note that does n't preclude some other function from returning a nodelist in a manner that is consistent between contexts . 
 thanks for the detailed analysis . The current spec defines a special case which produces different behaviour . Section 2.5.5 says ( emphasis ): So , when a function returning an ( or one of its subtypes ) is used in a test expression , this is not treated as an existence test . Instead , the value returned by the function is compared against . So , for the three possible return values , yields true whereas and yield false . On the other hand , when a function returning an ( or one of its subtypes ) is used in a comparison , the value returned by the function is compared to the other side of the comparison . Thus and produce identical nodelists . The crucial thing to note , and which avoids the above contradiction , is that a function returning an ( or one of its subtypes ) is treated differently depending on where it is used . This difference is determined at parse / compile time . During execution , the return value of the function is treated accordingly . 
 I appreciate that the spec says this explicitly . However , I maintain that this special case needs to be removed . This contextual difference in behavior represents an inconsistency in the overall grammar of expressions . Why are functions treated specially ? The underlying mechanics should operate the same , and they do n't . The spec very explicitly says that a JSON / is not valid in a logical context ... unless it comes from a function . Why ? ! Why special - case functions ? A function ( or any expression component ) should only be valid where its return type is valid , that is , where a value of that return type is valid . I should be able to replace the function with a value of its return type and still have a valid expression . But if I do that for , I get which is explicitly invalid . Therefore we have an inconsistent grammar . That statement from the spec you quoted is merely a bandage trying to cover up a more serious problem . In order to have a consistent grammar , this sort of value substitution MUST work . Finally , as the only person who has even attempted to implement this , I find it confusing and difficult , which should be more than enough reason to change it . 
 We do n’t agree . They are not treated specially . It is just a case that only applies to functions . No . It does n’t provide a way to put JSON literals there , so the issue only occurs for function returns . ( Because it is the only case . ) ( If talking about the type system :) . Yes . Maybe we should say “ well - typed ” instead of “ valid ” ... Where you ca n’t notate that value , this is moot . It is not invalid , it is not well - formed . I do n’t follow . You could say that . The original problem is that is not saying whether it talks about the node(s ) or about the value(s ) there . We have made sure this is well - defined in all cases : in a test - expr , it means ( the existence ) of nodes , and in a comparison - expr ( where we restrict the paths to singular ones ) it means the value ( or Nothing if no node ) . Function expressions straddle this boundary , so something needed to be done . Hence the type system , which is not about values . I do n’t follow . Maybe we can learn from this exercise that a mental model that conflates values and types does n’t work too well here .  ,     WHAT A SPECIAL CASE IS ! A special case is precisely a case that only applies to one thing . Precisely ! JSON literals ca n't go there because the values are invalid there . That 's not a reason . You 're saying functions are special - cased because they 're special cases . They should n't be . They should be treated like everything else in the expression . Special casing yields an inconsistent grammar . It 's inconsistent because the same grammar can behave differently . and have the same grammar : and both act in a logical capacity ( i.e. ) . They should behave the same : return a nodelist which is analyzed for contents . But they do n't ; they are inconsistent in their behavior . There 's no reason for them to behave differently except that the spec says that they do . When evaluating an expression such as , I must first analyze the components , , , and . From those , I get values which can be operated on by and . Those values MUST be of the correct type for those operators . The operators , though , have no knowledge of what produced the values . But you 're saying that needs to know if is a function or a path , and depending on which , it changes its behavior . That 's inconsistent behavior . should do one thing , and it should do that thing without context . For the purpose of this discussion , " valid " means that it does n't produce an error and can be evaluated . I 'm not making a distinction between being ABNF - valid and well - formed . I do n't understand what you mean by " notating " the value . The example I gave shows that what would be considered a valid expression ( ) is actually invalid when you consider the return type . The problem is that , within a single context , you 're defining one behavior for a path and another for a function . Functions should behave the same as paths in the same contexts . The spec disagrees with you : " A type is a set of instances . " The type system is very much about values . ( Note that the spec does n't define " instance . " This section is the only place the word is used . This is the example I gave . always finds a match . According to the spec , it should return JSON . Thus the expectation is that will return a nodelist containing all of the nodes of the subject data . Because the return value is always the same , I should be able to substitute the return value into the expression and yield another valid ( and well - formed ) expression that yields the same result . For example , in , I can substitue a logical true ( not JSON ) for and get . Performing that substitution on yields ( that 's the JSON literal ) . This is an invalid expression . It produces an error . Becuase and do not produce the same outcome , we have an inconsistency . It follows that if we are to have a consistent grammar and is invalid ( explicitly via the ABNF ) , must also be invalid . This kind of substitution exists in every programming language and every mathematical convention I 've ever seen . It is the very definition of logical consistency . It is foolish of us to define   expressions in a way that operates in a different manner . As mentioned , I 'm not conflating values and types . The spec clearly defines a type as a set of instances / values . I understand the difference . To accuse me of misunderstanding this is rude , especially knowing that I come from .Net and C # , which are very strictly typed . You 're very clearly holding onto keeping this for some reason despite the many logical arguments I 've made against it . I do n't understand why . Is it pride ? Is it unwillingness to change ? Is it laziness ? For the benefit of the specification and its wide adoption , I urge you to actually consider my arguments and remove this inconsistency from the spec . 
 these are the two options I present . I do n't see a way to have a consistent typing system and allow and in both logical and comparative contexts . The other option I can see is just doing away with the typing system altogether and just checking syntax ( and maybe argument count ) . Then functions can return whatever they like and the expression evaluation system will just run . If it encounters something invalid , then it does n't select that node , just like it would if returned a string in . 
 you have refused to acknowledge that there is anything wrong with the current solution of special - casing function in test expressions , and yet you have also failed to explain why such a special case is acceptable . The only argument you have presented is to quote the spec and say , " The document provides a solution for this . " As I 'm arguing that the document ( and therefore the solution ) is wrong , the document can not be used as evidence . There is a logical and syntactical inconsistency in the solution that the document presents ( " in the document " ) . I have repeatedly offered multiple analyses , explanations , logical arguments , and examples detailing why the current solution is inconsistent and contradictory . Yet it seems you 're not reading them . You have not in any way refuted my objections with why my analyses are incorrect or how the current solution is consistent and non - contradictory ( I re - read the issue to be sure ) , yet you continue to oppose me . I ask you to provide such reasoning or stand down and accept one of my proposals . 
 Yes . Of course . I had problems responding to your critique because it does n't seem to me to be aligned with the document . So the logical next step is to improve the document editorially , and then see whether your critique still applies . Unfortunately , I do n't have much time this week , as I 'm part of a small group that teaches an intensive course .. I ca n't really ask the others to do my work while I tend to this . So I ask for a little more patience . 
 Co - chair hat on . Clearly we are having a communication problem here .  is making two claims : language / meaning of the spec contains unacceptable special - casing and/or inconsistency current version of the spec is unreasonably hard to implement . On ( 1 . ) he has failed to convince the editors ;  has suggested that there is room for editorial improvements to clear the air . My notes : ' m inclined to give the editors a chance to see if they can reduce the discomfort and find us a path forward . confess that after repeated readings of the spec and the discussion , I 'm finding it hard to figure out what is what . In particular I find the language describing the type system to be really opaque . see no particular reason why the input type repertoire and output type repertoire for functions need to be identical . See my other issue about the return values for and . In particular I suspect that strictly limiting the output type repertoire might simplify the lives of spec readers and implementors too . Options such as limiting return types to JSON primitive types ( , , number , string , ) or alternately a strictly - constrained set of node - lists may be worth considering ? ' m probably more concerned about  's concern ( 2 . ) , that the current spec is hard to implement . I have painful personal experience with beautifully crafted specs that are unreasonably hard to code up . Greg , perhaps a little more on the difficulty would be useful ? It does seem to me that any implementation is going to have some special - casing around the functions , since they are a different kind of beast than the rest of the spec . we are unable to come to rough consensus on how to specify the function - extension framework , one plausible path forward is simply to remove it . I think that would be well within the scope of our charter . 
 thank you for taking the time to review this issue . I 've been asking for this and have n't received anthing except " the spec says you 're wrong . " ( See below ) I agree with your statement , but aligning function input with function output is not what this discussion is about . This is about how functions should only be valid where values of their return types would be valid . E.g. A function which returns a  or should only be valid where a JSON or literal would be valid . This makes for a consistent type system . For the current document , this is not the case . I 've tried to explain it above by describing how typed systems should work , but maybe I can rewrite my arguments from an implementation point of view . I 'll work on this and post back . I might be happy removing the type system ( I still need to explore that ) , but I think we need to keep functions . And that means that we still need to address what happens when a function appears in a place it 's not ( should n't be ) expected , like a function that returns a JSON or appearing as an operand for when  or themselves can not appear there . As mentioned in your , we need functions to fill in the gaps for " traditional " syntaxes that do n't fit within the spec 's syntax . You 're right , though : removing functions would make this issue moot , but we 'd also lose some of the functionality that users are used to when working with   . To address the general response that I 've received , I need to remove   from the argument and make the dispute absurd . Suppose we have the image below , and we want to write a document that describes how to reproduce this image . ! There are two scenarios I want to explore : document says , " The sky is red . " I see this and open an issue saying that the specification is wrong , that it should say that the sky is blue . In this scenario , I am correct in saying the spec is wrong . document says , " The sky is blue , " and I open an issue saying that the specification is wrong and suggest that it should say that the sky is red . In this scenario , I am incorrect . In both cases , I lay out an argument . And in both cases , the response should be , " Let 's check the source image , " not " No , the spec says that the sky is [ color ] . It 's fine the way it is . " We check the source image , and the issue is resolved . My point is that if I 'm saying the spec is wrong , the spec ca n't be used as evidence to say that the spec is right . The evidence has to come from an external source . 
 The editors did not have much time in the last week , so that 's why you did n't get a detailed explanation . Responding to each of these reminders does n't increase the speed at which I can work . I 'll submit a PR with an improved description of function expressions that should make the whole thing moot . But I ca n't do this in zero time . I do have empathy for your impatience . We can choose to do so , we can choose not to do so . The only reason you have given so far is that you want that to be the case . I believe that allowing JSON literals in the test - expr grammar would be very confusing . But we could . It just is n't needed . It is indeed hard to introduce a useful function extension without a type system . The requirement on the type system I 'm trying to fulfill is that it meshes with the properties of the non - extended parts of JSONPath , in particular that we can examine well - formed expressions whether they also are well - typed , and that this can be done independently of the actual JSON data that will be fed to the expression . This works best with a type system that is entirely static . Yes . No . Again , that equivalence is a preference from you that I do n't share . I 'm not discussing your blue sky , because we do agree that you do have that preference . We just do n't agree about that preference . If you have a little time left , can you please check ? I do n't think  is getting to this . 
 Understood . I await your detailed comments . There was no such reminder until recently . No , I have given several very detailed logically reasoned explanations backing up my case . Yes , it would . I 'm not advocating for this . I 'm advocating that because " allowing  literals in the test - expr grammar would be very confusing , " we also should n't allow functions that return JSON literals in test - expr grammar . This works best with a type system that is entirely static . I 100 % agree with this . This is not a preference . It is an aspect of typed systems in general . Functions may only appear where their return values may appear . 
 I 'm AFK until Tuesday , but I wonder if  's concerns would be addressed if it was possible to support referential transparency where a function could be replaced , per call ( tricky to implement in general ) , with ( a concrete representation of ) its return value without changing the result of the query . 
 ( 1 ) We would need literals for the whole type system . We ca n't notate Nothing or other node lists at the moment . ( 2 ) We would need to tag the literal with the function return type that is intended . 
 Not sure we would need all those literals . Suppose there was a node whose value was the same as the return value of a function . One referential transparency rule would be that replacing the function call with the path results in the same query result . But I take your point about possibly needing to tag values with types . This makes me wonder how we position the type system as a natural extension of the syntactic rules for non - function expressions . 
 I agree that we do n't need representation of all values in the syntax . For example ,  arrays and objects are values that are not allowed in the syntax . Here 's how I see the type landscape . We have three types : values , boolean , and nodelists . Values include any and all JSON values : arrays , objects , strings , numbers , and the three literals , , and . We do n't need any more precise typing . These can all be considered " values . " Also included as a value is , which we designated to represent the absence of a value , e.g. the value of in the object . Strings , numbers , and the three JSON literals can be expressed in the syntax . JSON objects ,  arrays , and have no representation in the syntax . Booleans are the result of any operation , e.g. , , or . Expressions in their entirety must also result in a boolean . There are two states ( I 'll use " state " to not confuse it with " value " ) for a boolean : true and false . The relation operators , and et al . , take two values ( from above ) as arguments and return a boolean . The logical operators , , , and , take two ( one for ) booleans as arguments and return a boolean . There is no relationship between the boolean states true and false and the JSON literals and . There is no representation for booleans in the syntax . Nodelists result from path evaluation . There is no representation for nodelists in the syntax . A nodelist is convertible to a Value if it has at most a single node . If a nodelist contains a node , the result is the value of that node . If a nodelist contains no nodes , the resulting value is . To enable syntax checking for paths in expressions , we have defined " Singular Path " as an identifiable syntax to ensure that a path can return at most one node and thus may be converted to a Value . A nodelist is always convertible to a Boolean . The resulting state is whether it contains nodes : true if it contains nodes false otherwise Setting up the type system this way covers the entire expression syntax . If we use V for a value , B for a boolean , and SNL and MNL for single nodelist and multiple nodelist respectively , the following are all valid expressions . V = = V B & & B B V ! = V B ! B ! B & & V < V SNL = = V MNL & & B SNL & & B MNL SNL The following are not valid expressions . V B & & V B = = V ! V MNL = = V Note that values and booleans each have their place in the syntax , and one can not be substituted for the other because there is no mapping between them . This is all well - formed and consistent . Now , when we add functions , we need to add them in such a way that they fit into this system . The way to ensure that is to type the function based on its return . This way , we know where in the expression the function is valid . If a function returns a value , it may only be used where a value may be used , i.e. relations . If a function returns a boolean , it may only be used where a boolean may be used , i.e. logical operations and entire expressions . If a function returns a nodelist , it may only be used where a nodelist may be used , i.e. anywhere . While the first two are fairly straightforward , a function returning a nodelist presents an inconsistency : a function returning a nodelist can appear in an expression as a value only if it contains a single node , but we have no syntactic way to ensure that the nodelist it returns has at most a single value . To address this inconsistency , we need to update our conversion rules for nodelists . Instead of : We now use : We also keep the singular path requirement in the syntax because identifying paths that are guaranteed to result in zero - or - one -length nodelists is easy . This also means that MNL = =  is now valid , but because of the singular path requirement it can only occur if the MNL is returned by a function . Finally , because of the conversion rules from nodelist to value and from nodelist to boolean , the same function return may behave differently ( and perhaps unexpectedly ) when a nodelist function appears as value vs as a boolean . I believe this is okay but I advise that we explicitly call it out as a note , possibly with an example as well . 
 ( Note that this removes all of the " optional " types as well . ) 
 here 's what I came up with regarding my experience trying to implement the current document . In implementing expressions , I developed a typed binary tree structure with three kinds of nodes : Value nodes - These represent values and can only be used in comparisons . Logic nodes - These represent logical outcomes and come in two flavors : Comparison nodes - These take two values and an operator . Logical operation nodes - These take one or two logical outcomes and an operator . Nodelist nodes - These represent nodelists and ( prior to functions ) are only returned by paths . ( Note , this is n't actually my code , but it 's similar enough to what I have . ) Functions need to fit into this structure , too , so we create function types for each type of node . With this , I can now represent expressions like . As you can see , everything is typed nicely . Everything gets what it expects . Logic get other logic and comparisons get values . Now , according to the current document , and should return a boolean : which means that with my model , they ca n't appear in a comparison . But the document shows examples where it 's being used both in a test expression and in a comparison : Something is off . I considered that my model was wrong for a while , but I 've made many similar expression - parsing apps previously ( it has been a special project of mine since high school ) , and I 've never run into a case where a value makes sense where logic should be . So I started to explore the type system more deeply . The many comments above have resulted from this exploration . This is certainly the most loosely typed system I 've ever parsed , but it 's still typed , so it should behave accordingly . In   , we say that is valid and returns false because the string and the number are both typed as " values " and those values are not equal . ( In C # , they 're not even comparable because they 're different types , and compilation will fail . ) However , ( which is perfectly legal in C # ) is not valid for   because ( disregarding the ABNF ; I 'm just discussing the theory here ) the logical produced by is not a " value " and ca n't be used with the operator . We encoded this into the ABNF because of this reasoning . So this is the first problem ( and what this issue was created for ): if a is to be used as a " logical " , it can not be a subtype of . They can not be relatable at all because we decided previously that  and literals were merely constant values , not " logicals . " This also applies to functions . If a function is defined to return a , then it can only appear in an expression where a " logical " would appear . Conversely , if a function is defined to return a , then it can only appear in an expression where a " value " would appear . This means must be invalid . The same logic applies , though in reverse , if you take to mean " JSON or , " and is the invalid one in this case . So we ca n't have both . Traditionally , a " function " takes a number of arguments and deterministically and in a context - free manner produces the same result . The document , however says this for when the function exists in a test expression : and also this : This means that the function changes behavior depending on the context , meaning it 's not really a " function . " If you want to claim that there is an implicit conversion from the or that the function returns to the " logical " required by the existence test , then it follows that the literals themselves should also receive that implicit conversion , meaning should be valid , and we already decided that it 's not for the reasons I mentioned before . It always comes down to an intentional separation between " logicals " and  / , and the type system as it exists tries to blur that line . If we want to re - hash that discussion , I 'm sure we 'll come to the same conclusion that they are not the same , nor can they be converted between . To do so would introduce ambiguities and remove functionality . ( This was the reasoning behind the initial decision . )
Paragraph (version A):
A test expression either tests the existence of a node designated by
an embedded query (see extest) or tests the result of a function
expression (see fnex).  In the latter case, if the function
expression is of type "OptionalBooleanType" or one of its subtypes,
it tests whether the result is "true"; if the function expression is
of type "OptionalNodesType" or one of its subtypes, it tests whether
the result is different from "Nothing".
Paragraph (version B):
A test expression either tests the existence of a node designated by
an embedded query (see extest) or tests the result of a function
expression (see fnex).  In the latter case, if the function
expression is of type "ST(OptionalBooleanType)" (see typesys), it
tests whether the result is "true"; if the function expression is of
type "ST(OptionalNodesType)", it tests whether the result is
different from "Nothing".
Paragraph (version A):
A type is a set of instances.  A type is a subtype of another type if
its set of instances (possibly after coercion) is a subset of the set
of instances of the other type.  The subtype relationship is
transitive: for any A, B, and C if B is a subtype of A and C is a
subtype of B, then C is a subtype of A.
Paragraph (version B):
A type is a set of instances.  A type is a subtype of another type if
its set of instances (possibly after coercion) is a subset of the set
of instances of the other type.  The subtype relationship is
transitive: for any A, B, and C if B is a subtype of A and C is a
subtype of B, then C is a subtype of A.  We denote a type "T" and the
transitive closure of all its subtypes as "ST(T)".
Paragraph (version A):
If it occurs as a "filter-path" in a test expression, the function
is defined to have result type "OptionalNodesType" or one of its
subtypes, or to have result type "OptionalBooleanType" or one of
its subtypes.
Paragraph (version A):
If it occurs as a "comparable" in a comparison, the function is
defined to have result type "OptionalNodeOrValueType" or one of
its subtypes.
Paragraph (version B):
If it occurs directly in a test expression, the function is
defined to have a result type in "ST(OptionalNodesType)".  or to
have a result type in "ST(OptionalBooleanType)".
Paragraph (version B):
If it occurs as a "comparable" in a comparison, the function is
defined to have a result type in "ST(OptionalNodeOrValueType)".
Paragraph (version A):
The argument is a "filter-path" or a Singular Path and the
defined type of the argument is "OptionalNodesType".
Paragraph (version B):
The argument is a "filter-path" (which includes Singular Paths)
and the defined type of the argument is "OptionalNodesType".