Repository: Shopify/liquid (Ruby)

Issue:
Unexpected results with `replace` and some escape sequences
I'm not 100% clear on how escape sequences work in Liquid. Sometimes they appear to be a thing, as in `{{ "\\" }}` => `\`, and other times not, as in `{{ "\0" }}` => `\0`.

I _think_ this inconsistency is impacting `replace`. (_And_ `replace_first`, but the rest of this treats `replace` as illustrative.)

Test script:

```ruby
require 'liquid'

def test_render char
  Liquid::Template.parse("
    {%- capture foo %}>\\#{char}b<{% endcapture %}
    {%- capture bar %}a {{ foo }} c{% endcapture %}
    {%- capture baz %}>{{ foo }}<{% endcapture %}
    {{- bar | replace: foo, baz -}}
  ").render
end

def report_result test_char, expected_result, test_result
  pass = expected_result == test_result

  if pass
    puts "\t#{test_char}\t  passed:   #{test_result}"
  else
    puts "🛑\t#{test_char} \t⌜ expected: #{expected_result}\n\t\t⌞ failed:   #{test_result}"
  end
end

control_char = '👍'
control_result = test_render(control_char)
report_result(control_char, control_result, control_result)

(' '..'|').each do |test_char|
  expected_result = control_result.sub(control_char, test_char)
  test_result = test_render(test_char)

  report_result(test_char, expected_result, test_result)
end

```

Result, with ruby 2.3.4p301 and liquid 4.0.0:

```
	👍	  passed:   a >>\👍b<< c
	 	  passed:   a >>\ b<< c
	!	  passed:   a >>\!b<< c
	"	  passed:   a >>\"b<< c
	#	  passed:   a >>\#b<< c
	$	  passed:   a >>\$b<< c
	%	  passed:   a >>\%b<< c
🛑	& 	⌜ expected: a >>\&b<< c
		⌞ failed:   a >>>\&b<b<< c
🛑	' 	⌜ expected: a >>\'b<< c
		⌞ failed:   a >> cb<< c
	(	  passed:   a >>\(b<< c
	)	  passed:   a >>\)b<< c
	*	  passed:   a >>\*b<< c
🛑	+ 	⌜ expected: a >>\+b<< c
		⌞ failed:   a >>b<< c
	,	  passed:   a >>\,b<< c
	-	  passed:   a >>\-b<< c
	.	  passed:   a >>\.b<< c
	/	  passed:   a >>\/b<< c
🛑	0 	⌜ expected: a >>\0b<< c
		⌞ failed:   a >>>\0b<b<< c
🛑	1 	⌜ expected: a >>\1b<< c
		⌞ failed:   a >>b<< c
🛑	2 	⌜ expected: a >>\2b<< c
		⌞ failed:   a >>b<< c
🛑	3 	⌜ expected: a >>\3b<< c
		⌞ failed:   a >>b<< c
🛑	4 	⌜ expected: a >>\4b<< c
		⌞ failed:   a >>b<< c
🛑	5 	⌜ expected: a >>\5b<< c
		⌞ failed:   a >>b<< c
🛑	6 	⌜ expected: a >>\6b<< c
		⌞ failed:   a >>b<< c
🛑	7 	⌜ expected: a >>\7b<< c
		⌞ failed:   a >>b<< c
🛑	8 	⌜ expected: a >>\8b<< c
		⌞ failed:   a >>b<< c
🛑	9 	⌜ expected: a >>\9b<< c
		⌞ failed:   a >>b<< c
	:	  passed:   a >>\:b<< c
	;	  passed:   a >>\;b<< c
	<	  passed:   a >>\<b<< c
	=	  passed:   a >>\=b<< c
	>	  passed:   a >>\>b<< c
	?	  passed:   a >>\?b<< c
	@	  passed:   a >>\@b<< c
	A	  passed:   a >>\Ab<< c
	B	  passed:   a >>\Bb<< c
	C	  passed:   a >>\Cb<< c
	D	  passed:   a >>\Db<< c
	E	  passed:   a >>\Eb<< c
	F	  passed:   a >>\Fb<< c
	G	  passed:   a >>\Gb<< c
	H	  passed:   a >>\Hb<< c
	I	  passed:   a >>\Ib<< c
	J	  passed:   a >>\Jb<< c
	K	  passed:   a >>\Kb<< c
	L	  passed:   a >>\Lb<< c
	M	  passed:   a >>\Mb<< c
	N	  passed:   a >>\Nb<< c
	O	  passed:   a >>\Ob<< c
	P	  passed:   a >>\Pb<< c
	Q	  passed:   a >>\Qb<< c
	R	  passed:   a >>\Rb<< c
	S	  passed:   a >>\Sb<< c
	T	  passed:   a >>\Tb<< c
	U	  passed:   a >>\Ub<< c
	V	  passed:   a >>\Vb<< c
	W	  passed:   a >>\Wb<< c
	X	  passed:   a >>\Xb<< c
	Y	  passed:   a >>\Yb<< c
	Z	  passed:   a >>\Zb<< c
	[	  passed:   a >>\[b<< c
🛑	\ 	⌜ expected: a >>\\b<< c
		⌞ failed:   a >>\b<< c
	]	  passed:   a >>\]b<< c
	^	  passed:   a >>\^b<< c
	_	  passed:   a >>\_b<< c
🛑	` 	⌜ expected: a >>\`b<< c
		⌞ failed:   a >>a b<< c
	a	  passed:   a >>\ab<< c
	b	  passed:   a >>\bb<< c
	c	  passed:   a >>\cb<< c
	d	  passed:   a >>\db<< c
	e	  passed:   a >>\eb<< c
	f	  passed:   a >>\fb<< c
	g	  passed:   a >>\gb<< c
	h	  passed:   a >>\hb<< c
	i	  passed:   a >>\ib<< c
	j	  passed:   a >>\jb<< c
	k	  passed:   a >>\kb<< c
	l	  passed:   a >>\lb<< c
	m	  passed:   a >>\mb<< c
	n	  passed:   a >>\nb<< c
	o	  passed:   a >>\ob<< c
	p	  passed:   a >>\pb<< c
	q	  passed:   a >>\qb<< c
	r	  passed:   a >>\rb<< c
	s	  passed:   a >>\sb<< c
	t	  passed:   a >>\tb<< c
	u	  passed:   a >>\ub<< c
	v	  passed:   a >>\vb<< c
	w	  passed:   a >>\wb<< c
	x	  passed:   a >>\xb<< c
	y	  passed:   a >>\yb<< c
	z	  passed:   a >>\zb<< c
	{	  passed:   a >>\{b<< c
	|	  passed:   a >>\|b<< c
```


Discussion:
The problem is that we were using [String#gsub](http://ruby-doc.org/core-2.3.3/String.html#method-i-gsub) in the replace filter with the assumption that the replacement string was used as the literal replacement text, but the documentation actually says

> If replacement is a String it will be substituted for the matched text. It may contain back-references to the pattern’s capture groups of the form "\d", where d is a group number, or "\k<n>", where n is a group name. If it is a double-quoted string, both back-references must be preceded by an additional backslash. However, within replacement the special match variables, such as &$, will not refer to the current match. If replacement is a String that looks like a pattern’s capture group but is actually not a pattern capture group e.g. "\'", then it will have to be preceded by two backslashes like so "\\'".

This can be seen with plain ruby with

```
irb> "abc".sub("b", "\\`")
=> "aac"
irb> "abc".sub("b", "\\&")
=> "abc"
irb> "abc".sub("b", "\\'")
=> "acc"
```

where those special characters correspond to the [special global variables](http://ruby-doc.org/core-2.3.3/Regexp.html#class-Regexp-label-Special+global+variables) for the string before the match, the match text and the string after the match.

This doesn't seem like intended behaviour.  I'm pretty sure we want to treat the replacement string as the literal text to replace for any matches.
Hi @dylanahsmith 
I also trip over this problem when trying to replace a complete URL.

The following URL could be replaced without problems:
```
https:\/\/app.sendinblue.com\/integrations\/api\/sy\/cart_tracking\/script
```

The following URL which contains `&` could not be replaced:
```
https:\/\/app.sendinblue.com\/integrations\/api\/sy\/cart_tracking\/script?user_connection_id=XXX&ma-key=XXX
```

I tried to escape the & with \. But I could not get it working. How can I replace such URLs in Shopify liquid?



Patch:
diff --git a/lib/liquid/standardfilters.rb b/lib/liquid/standardfilters.rb
index 48ef5c356..7c18c0d7a 100644
--- a/lib/liquid/standardfilters.rb
+++ b/lib/liquid/standardfilters.rb
@@ -201,12 +201,14 @@ def compact(input, property = nil)
 
     # Replace occurrences of a string with another
     def replace(input, string, replacement = ''.freeze)
-      input.to_s.gsub(string.to_s, replacement.to_s)
+      replacement = replacement.to_s
+      input.to_s.gsub(string.to_s) { replacement }
     end
 
     # Replace the first occurrences of a string with another
     def replace_first(input, string, replacement = ''.freeze)
-      input.to_s.sub(string.to_s, replacement.to_s)
+      replacement = replacement.to_s
+      input.to_s.sub(string.to_s) { replacement }
     end
 
     # remove a substring


Test patch:
diff --git a/test/integration/standard_filter_test.rb b/test/integration/standard_filter_test.rb
index 41ca06c6f..9de2106d5 100644
--- a/test/integration/standard_filter_test.rb
+++ b/test/integration/standard_filter_test.rb
@@ -362,8 +362,10 @@ def test_first_last
   def test_replace
     assert_equal '2 2 2 2', @filters.replace('1 1 1 1', '1', 2)
     assert_equal '2 2 2 2', @filters.replace('1 1 1 1', 1, 2)
+    assert_equal "\\& \\& \\& \\&", @filters.replace('1 1 1 1', '1', "\\&")
     assert_equal '2 1 1 1', @filters.replace_first('1 1 1 1', '1', 2)
     assert_equal '2 1 1 1', @filters.replace_first('1 1 1 1', 1, 2)
+    assert_equal '\\& 1 1 1', @filters.replace_first("1 1 1 1", '1', "\\&")
     assert_template_result '2 1 1 1', "{{ '1 1 1 1' | replace_first: '1', 2 }}"
   end