Rails: Fixing ‘Invalid query parameters: invalid %-encoding’
Tags: rails, ruby
Published: 2021-03-23 02:45:22

**Table of Contents:**
[Problem](#problem)
[A. Drop the query string](#a)
[B. Fix the query string](#b)
[Final word](#final-word)
<a name="problem"></a>
##Problem

```
ActionController::BadRequest
Invalid query parameters: invalid %-encoding (miconazole%202%%20cr)

Rack::QueryParser::InvalidParameterError
invalid %-encoding (miconazole%202%%20cr)

ActionController::BadRequest
ActionView::Template::Error: Invalid query parameters: invalid %-encoding (%2Fsearch%2Fall%Forder%3Ddescending%26page%3D5%26sort%3Dcreated_at)
```


In a full stack apps, sometimes users manually edit query strings in the address bar, and make requests that have invalid encodings.

In a api apps, sometimes front end or mobile engineer forget to encoding the query strings, so they just give raw query strings to your apps, then they left it for years until it became very hard to fixed on the client side, and the management decide the server side should fix it.

These corrupted query strings are not that important, but users are receiving 500 server error pages.

What can we do?

I can offer two choice: Drop the query string OR Fix the query string.

<a name="a"></a>
##A. Drop the query string

I got this from this post right [here](https://jamescrisp.org/2018/05/28/fixing-invalid-query-parameters-invalid-encoding-in-a-rails-app/).

**First**, make a middleware:
```ruby
# You can put this everywhere
# In my case, I put this in app/middlewares/hande_bad_encoding_middleware.rb

class HandleBadEncodingMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    begin
      Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s)
    rescue Rack::Utils::InvalidParameterError
      env['QUERY_STRING'] = ''
    end

    @app.call(env)
  end
end
```
**Second**, use this middleware before `Rack::Runtime`:
```ruby
# config/application.rb

require_relative 'boot'

require 'rails/all'

# If you get NameError: uninitialized constant,
# you have to use require like this one:
require_relative '../app/middlewares/handle_bad_encoding_middleware.rb'

Bundler.require(*Rails.groups)

module YourAppsName
  class Application < Rails::Application

    ...
    # add this:
    config.middleware.insert_before Rack::Runtime, HandleBadEncodingMiddleware
    ...

  end
end
```

The result is, you drop the query string:
```ruby
# from this
"https://your-url.com?query=alcohol%2070%"

# to this
"https://your-url.com"
```

<a name="b"></a>
##B. Fix the query string

I did some research and modify the middleware. This is the step:

**First** (optional), I use gem `rack-utf8_sanitizer` to remove invalid UTF-8 characters from the query strings. This prevents errors like "invalid byte sequence in UTF-8".

I personally prefer `rack-utf8_sanitizer` than `utf8-cleaner` because it is more robust, more option, and less bugs.

**Second**, make a middleware:
```ruby
# You can put this everywhere
# In my case, I put this in app/middlewares/hande_bad_encoding_middleware.rb

class HandleBadEncodingMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    begin
      Rack::Utils.parse_nested_query(env['QUERY_STRING'].to_s)
    rescue Rack::Utils::InvalidParameterError
      params = URI.decode_www_form(env['QUERY_STRING'])
      query_string = URI.encode_www_form(params)
      env['QUERY_STRING'] = query_string
    end

    @app.call(env)
  end
end
```
There are many ways to encode and decode url:
```ruby
URI.escape
URI.unescape
URI.encode_www_form
URI.decode_www_form
URI.encode_www_form_component
URI.decode_www_form_component
CGI.escape
CGI.unescape
```
I've tried them all, and I picked the best for me, which are `URI.encode_www_form` and `URI.decode_www_form`. Probably you have to pick which is best for you.

**Third**, use this middleware before `Rack::Runtime`:
```ruby
# config/application.rb


require_relative 'boot'

require 'rails/all'

# If you get NameError: uninitialized constant,
# you have to use require like this one:
require_relative '../app/middlewares/handle_bad_encoding_middleware.rb'

Bundler.require(*Rails.groups)

module YourAppsName
  class Application < Rails::Application

    ...
    # you add this for the rack-utf8_sanitizer,
    # as described in the gem's readme:
    config.middleware.insert 0, Rack::UTF8Sanitizer
    # add this:
    config.middleware.insert_before Rack::Runtime, HandleBadEncodingMiddleware
    ...

  end
end
```
The result is, you fix the query string:
```ruby
# from this
"https://your-url.com?query=alcohol%2070%"

# to this
"https://your-url.com?query=alcohol%2070%25"
```

<a name="final-word"></a>
##Final Word
If you have any problems or questions, let me know in the comment.

If you are helped, I am glad I can help you.