opened
aspnet-hello
username_0
_From @nathanaeljones on Friday, May 30, 2014 11:56:39 AM_

Could we get some light shed on the whys/hows of [querystring handling logic in vNext](https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNet.PipelineCore/Infrastructure/ParsingHelpers.cs#L558)? Previous [versions](http://referencesource.microsoft.com/#System.Web/xsp/system/Web/HttpValueCollection.cs#159) have been inconsistent at best, and this seems like a great chance to fix things.

An all-encompassing parsing strategy may be possible, but inadvisable, as the logic could not be clearly communicated to developers. I would suggest instead that parsing/serialization formats be named specifically for their appropriate use. 

For example, requests with content-type `application/x-www-form-urlencoded` must follow the [WHATWG parsing/serialization spec](http://url.spec.whatwg.org/#application/x-www-form-urlencoded-0) &mdash; which differs from php, ruby, node, python, classic asp, and each implementation in asp.net vNow, most of which have differing rules on one or more points:
1. Handling of duplicate keys. (error, replace, concatenate, or build list)?
2. Hash serialization. `key[a]=1&key[b]=2` produces `key = {a=1, b=2}` in some implementations
3. [Array serialization](http://stackoverflow.com/questions/6243051/how-to-pass-an-array-within-a-query-string). `a=1&a=2`, `a=1,2`, `a[]=1&a[]=2`, `a[1]=1`, a[2]=2` are all valid ways to represent an array value on different platforms. 
4. Handling of invalid characters (error, or pretend they were URL encoded)?
5. Encoding of reserved characters (Most frameworks fail to url encode reserved characters correctly)
6. Order preservation - some developers (inadvisably) rely on the order of querystring pairs. A recent instance of this is in the latest version of Umbraco. 
7. Case preservation and case sensitivity. 
8. URL decoding pass count (AFAIK, only ASP.NET WebForms messes this up). Ideally,  `+`, `%20` -> ` ` should be the only lossy operation. 

Once we know the team's opinions on querystrings (and paths - and PathInfos), I think there are a lot of developers that would be happy to pitch in with unit tests and compatibility profiles. I imagine that [Javascript/Node](https://github.com/joyent/node/blob/f7ede33f09187cd9b1874982e813380cd292ef17/lib/querystring.js#L142) and [php](https://github.com/php/php-src/blob/419d98b291b4dd36e63f34d72445bd954f3ce3e2/ext/standard/string.c#L4445) compatibility would be the highest priority. 

Of course, this is easiest if we can ensure that we do not lose any data between the network packet and the developer. I'm looking at you, IIS.


_Copied from original issue: aspnet/HttpAbstractions#67_
Publish the spec behind the Micrsoft Querystring Parsing and Serialization rules
issue
created
aspnet-hello
username_0
_From @nathanaeljones on Monday, June 16, 2014 6:49:49 AM_

Also, a compatibility matrix between Uri.(Un)EscapeDataString and the HttpUtility methods would be useful if [we're switching](https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNet.PipelineCore/Infrastructure/ParsingHelpers.cs#L583). [Incorrect space handling is well-known](http://blogs.msdn.com/b/yangxind/archive/2006/11/09/don-t-use-net-system-uri-unescapedatastring-in-url-decoding.aspx), but unicode encoding is a different story. HttpUtility guesses encoding (often wrongly) from the request, and both appear to take very different approaches to actual byte mangling.
comment
created
aspnet-hello
username_0
_From @glennc on Monday, June 23, 2014 3:39:33 PM_

@GrabYourPitchforks Can you talk about this here for now?
comment
created
aspnet-hello
username_0
_From @GrabYourPitchforks on Tuesday, June 24, 2014 8:57:18 AM_

Not quite sure what you mean about IIS and HttpUtility.UrlEncode/UrlDecode munging the encoding.  IIS doesn't touch the query string at all (it just gets forwarded through as raw bytes), and UrlEncode/UrlDecode always assumes UTF8 unless the developer has specified something else in Web.config.

The biggest difference between UnescapeDataString and HttpUtility.UrlDecode is two-fold: (a) UrlDecode can only be used for the query string, not for path segments, due to the way it handles the U+0020 code point; and (b) UrlDecode understands the non-standard %uXXXX format.

We've been working on new encoding / decoding routines which should provide a more uniform and easy-to-understand behavior.  These routines will also support previous customer suggestions like leaving IRI-safe characters unescaped.  I don't have an ETA for when these will be available.
comment
created
aspnet-hello
username_0
_From @davidfowl on Thursday, January 8, 2015 8:51:54 AM_

@GrabYourPitchforks These are available now right?
comment