Repository: mvdan/sh
Pull request #687 (2021-03-19 20:01:45)

Add Bash 5.1 parameter transformations
Per the NEWS:

dd. New `U', `u', and `L' parameter transformations to convert to uppercase,
    convert first character to uppercase, and convert to lowercase,
    respectively.

hh. New `K' parameter transformation to display associative arrays as key-
    value pairs.

lollipopman: I am happy to rework this with tests and excluding mksh if needed, just wasn't sure what direction you would prefer?
mvdan: Yeah, tests would be good - there should already be tests covering the other transformations, so you could copy-paste a bit.
lollipopman: > Yeah, tests would be good - there should already be tests covering the other transformations, so you could copy-paste a bit.

Will do, do you want me to prevent them working on mksh?
mvdan: Up to you - I think the current code is slightly wrong too, as mksh doesn't seem to support some older bash transformations like @E. We need tests either way, though.
lollipopman: tests added, let me know if you want any further improvements, thanks!
lollipopman: hmm, I can't reproduce this test failure on Debian sid, I'll need to spend some more time to debug it
mvdan: The tests which run the real bash/mksh/etc require a specific major version, since older or newer versions behave differently. If a different version is installed, they are skipped.

Error inputs can have the ` #NOERR` trailing string to signal that our parser errors, but a real shell's does not. You will need that here, because for better or worse, bash/mksh/etc do not fully parse parameter expansions until they are executed.
lollipopman: > The tests which run the real bash/mksh/etc require a specific major version, since older or newer versions behave differently. If a different version is installed, they are skipped.
> 
> Error inputs can have the ` #NOERR` trailing string to signal that our parser errors, but a real shell's does not. You will need that here, because for better or worse, bash/mksh/etc do not fully parse parameter expansions until they are executed.

Thanks @mvdan that would have taken me quite a while to suss out on my own. I think everything is in place now. Perhaps it would be valuable to add a `TESTING.md` or similar doc to provide guidance on testing? Until you explained that the tests were being skipped I didn't realize that my local testing wasn't exhaustive. It would be great to be able to run all tests locally, perhaps via docker containers? Thanks again
mvdan: Indeed, you're right. A bunch of people have contributed test cases in the past already, so I'm overdue for documenting this a bit.

Patch:
diff --git a/syntax/parser.go b/syntax/parser.go
index d4d96cfe..e65c4c0e 100644
--- a/syntax/parser.go
+++ b/syntax/parser.go
@@ -1379,7 +1379,15 @@ func (p *Parser) paramExpExp() *Expansion {
 			p.curErr("@ expansion operator requires a literal")
 		}
 		switch p.val {
-		case "Q", "E", "P", "A", "a":
+		case "a", "u", "A", "E", "K", "L", "P", "U":
+			if !p.lang.isBash() {
+				p.langErr(p.pos, "this expansion operator", LangBash)
+			}
+		case "#":
+			if p.lang != LangMirBSDKorn {
+				p.langErr(p.pos, "this expansion operator", LangMirBSDKorn)
+			}
+		case "Q":
 		default:
 			p.curErr("invalid @ expansion operator")
 		}


Test patch:
diff --git a/syntax/filetests_test.go b/syntax/filetests_test.go
index 31396302..0c733410 100644
--- a/syntax/filetests_test.go
+++ b/syntax/filetests_test.go
@@ -2421,7 +2421,7 @@ var fileTests = []testCase{
 	},
 	{
 		Strs: []string{`${a@E} ${b@a} ${@@Q} ${!ref@P}`},
-		bsmk: call(
+		bash: call(
 			word(&ParamExp{
 				Param: lit("a"),
 				Exp: &Expansion{
@@ -2453,6 +2453,25 @@ var fileTests = []testCase{
 			}),
 		),
 	},
+	{
+		Strs: []string{`${a@Q} ${b@#}`},
+		mksh: call(
+			word(&ParamExp{
+				Param: lit("a"),
+				Exp: &Expansion{
+					Op:   OtherParamOps,
+					Word: litWord("Q"),
+				},
+			}),
+			word(&ParamExp{
+				Param: lit("b"),
+				Exp: &Expansion{
+					Op:   OtherParamOps,
+					Word: litWord("#"),
+				},
+			}),
+		),
+	},
 	{
 		Strs: []string{`${#foo}`},
 		common: &ParamExp{
diff --git a/syntax/parser_test.go b/syntax/parser_test.go
index 982dcbb2..63f9b7ef 100644
--- a/syntax/parser_test.go
+++ b/syntax/parser_test.go
@@ -1896,6 +1896,42 @@ var shellTests = []errorCase{
 		in:    "echo ${foo@Q}",
 		posix: `1:11: this expansion operator is a bash/mksh feature`,
 	},
+	{
+		in:   "echo ${foo@a}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@u}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@A}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@E}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@K}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@L}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@P}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@U}",
+		mksh: `1:12: this expansion operator is a bash feature`,
+	},
+	{
+		in:   "echo ${foo@#}",
+		bash: `1:12: this expansion operator is a mksh feature #NOERR`,
+	},
 	{
 		in:     "`\"`\\",
 		common: "1:3: reached EOF without closing quote `",