Repository: mbrt/gmailctl (go)

Issue:
Cant get an OAuth token if my publish status is "In production"
First up thanks for this project! 

I have it working in testing mode but I cant get it working if I publish my app. Tried 3 times, following the instructions as per `gmailctl init`. 

When visiting the authorisation link I get the following from the google oauth page:

```
Error 400: invalid_request

You can't sign in to this app because it doesn't comply with Google's OAuth 2.0 policy for keeping apps secure.

You can let the app developer know that this app doesn't comply with one or more Google validation rules.
Request Details
The content in this section has been provided by the app developer. This content has not been reviewed or verified by Google.
If you’re the app developer, make sure that these request details comply with Google policies.
redirect_uri: urn:ietf:wg:oauth:2.0:oob
```

Not sure if this is a recent change or not. I've followed the `init` instructions to the letter, from what I can tell. Any ideas?

Pull request:
Update authorization workflow
Google recently tightened controls on unverified apps. This caused trouble to new users creating their credentials.

The workflow now requires a local webserver and the browser from which the user authenticates needs to run on the same machine as gmailctl, every time the token needs a refresh.

Close #232, #243.

Patch:
diff --git a/.gitignore b/.gitignore
index 1fd50c2..fea05b8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,4 @@
 /gmailctl
 mycfg.jsonnet
 *.snap
+/tmp
diff --git a/README.md b/README.md
index 3cc2809..e300dcc 100644
--- a/README.md
+++ b/README.md
@@ -872,6 +872,34 @@ feature. If you really want to do this, you can manually export your rules with
 `gmailctl export > filters.xml`, upload them by using the Gmail Settings UI and
 select the "apply new filters to existing email" checkbox.
 
+### OAuth2 authentication errors
+
+Gmail APIs require strict controls, even if you are only accessing your own
+data. If you're getting errors similar to:
+
+```
+oauth2: cannot fetch token: 400 Bad Request
+Response: {
+  "error": "invalid_grant",
+  "error_description": "Bad Request"
+}
+```
+
+it's likely your auth token expired. Try refreshing it with:
+
+```bash
+$ gmailctl init --refresh-expired
+```
+
+and follow the instructions on screen.
+
+If this doesn't help, retry the authorization workflow from the start:
+
+```bash
+$ gmailctl init --reset
+$ gmailctl init
+```
+
 ### YAML config is unsupported
 
 gmailctl recently deprecated older config versions (`v1alpha1`, `v1alpha2`).
diff --git a/cmd/gmailctl/localcred/local_provider.go b/cmd/gmailctl/localcred/local_provider.go
index f906950..bb27aaa 100644
--- a/cmd/gmailctl/localcred/local_provider.go
+++ b/cmd/gmailctl/localcred/local_provider.go
@@ -14,40 +14,53 @@ import (
 )
 
 const (
+	// Keep in sync with https://rclone.org/drive/#making-your-own-client-id.
+	// They hit the exact same issues with Google Drive.
 	credentialsMissingMsg = `The credentials are not initialized.
 
 To do so, head to https://console.developers.google.com
 
 1. Create a new project if you don't have one.
-1. Go to 'Enable API and services' and select Gmail.
-2. Go to credentials and create a new one, by selecting 'Help me choose'.
-   2a. Select the Gmail API.
-   2b. Select 'Other UI'.
-   2c. Access 'User data'.
-3. Go to 'OAuth consent screen'.
-   3a. If your account is managed by an organization, you have to
-       select 'Internal' as 'User Type' and Create (otherwise ignore).
-   3b. Set an application name (e.g. 'gmailctl').
-   3c. Update 'Scopes for Google API', by adding:
-       * https://www.googleapis.com/auth/gmail.labels
-       * https://www.googleapis.com/auth/gmail.settings.basic
-5. IMPORTANT: you don't need to submit your changes for verification, as
-   you're only going to access your own data. Save and 'Go back to
-   Dashboard'.
-   5a. Make sure that the 'Publishing status' is set to 'In production'.
-       If it's set to 'Testing', Publish the app and ignore the
-	   verification. Using the testing mode will make your tokens
-	   expire every 7 days and require re-authentication.
-6. Go back to Credentials.
-   6a. Click 'Create credentials'.
-   6b. Select 'OAuth client ID'.
-   6c. Select 'Desktop app' as 'Application type' and give it a name.
-   6d. Create.
-7. Download the credentials file into '%s' and execute the 'init'
+1. Go to 'Enable API and services', search for Gmail and enable it.
+2. Go to 'OAuth consent screen'.
+    2a. If your account is managed by an organization, you have to
+        select 'Internal' as 'User Type'. For individual accounts
+        select 'External'.
+    2b. Set an application name (e.g. 'gmailctl').
+    2c. Use your email for 'User support email' and 'Developer
+        contact information'. Save and continue.
+    3c. Select 'Add or remove scopes' and add:
+        * https://www.googleapis.com/auth/gmail.labels
+        * https://www.googleapis.com/auth/gmail.settings.basic
+    3d. Save and continue until you're back to the dashboard.
+3. You now have a choice. You can either:
+    * Click on 'Publish App' and avoid 'Submitting for
+      verification'. This will result in scary confirmation
+      screens or error messaages when you authorize gmailctl with
+      your account (but for some users it works), OR
+    * You could add your email as 'Test user' and keep the app in
+      'Testing' mode. In this case everything will work, but
+      you'll have to login and confirm the access every week (token
+      expiration).
+4.  Go to Credentials on the left.
+    4a. Click 'Create credentials'.
+    4b. Select 'OAuth client ID'.
+    4c. Select 'Desktop app' as 'Application type' and give it a name.
+    4d. Create.
+5. Download the credentials file into %q and execute the 'init'
    command again.
 
 Documentation about Gmail API authorization can be found
 at: https://developers.google.com/gmail/api/auth/about-auth
+`
+	authMessage = `Go to the following link in your browser and authorize gmailctl:
+
+%v
+
+NOTE that gmailctl runs a webserver on your local machine to
+collect the token as returned from Google. This only runs until
+the token is saved. If your browser is on another machine
+without access to the local network, this will not work.
 `
 )
 
@@ -123,14 +136,16 @@ func openToken(ctx context.Context, auth *api.Authenticator, path string) (*gmai
 }
 
 func setupToken(auth *api.Authenticator, path string) error {
-	fmt.Printf("Go to the following link in your browser then type the "+
-		"authorization code: \n%v\nAuthorization code: ", auth.AuthURL())
-
-	var authCode string
-	if _, err := fmt.Scan(&authCode); err != nil {
-		return fmt.Errorf("unable to retrieve token from web: %w", err)
+	localSrv := newOauth2Server(auth.State)
+	addr, err := localSrv.Start()
+	if err != nil {
+		return errors.WithDetails(fmt.Errorf("starting local server: %w", err),
+			"gmailctl requires a temporary local HTTP server for the authentication flow.")
 	}
+	defer localSrv.Close()
 
+	fmt.Printf(authMessage, auth.AuthURL("http://"+addr))
+	authCode := localSrv.WaitForCode()
 	if err := saveToken(path, authCode, auth); err != nil {
 		return fmt.Errorf("caching token: %w", err)
 	}
@@ -145,7 +160,7 @@ func saveToken(path, authCode string, auth *api.Authenticator) (err error) {
 	}
 	defer func() {
 		e = f.Close()
-		// do not hide more important error
+		// Do not hide more important errors.
 		if err == nil {
 			err = e
 		}
diff --git a/cmd/gmailctl/localcred/oauth2_server.go b/cmd/gmailctl/localcred/oauth2_server.go
new file mode 100644
index 0000000..267f967
--- /dev/null
+++ b/cmd/gmailctl/localcred/oauth2_server.go
@@ -0,0 +1,78 @@
+package localcred
+
+import (
+	"fmt"
+	"net"
+	"net/http"
+)
+
+const successMsg = `Successfully authenticated with Google OAuth2.
+You may now close this page and return to the terminal.
+`
+
+type oauth2Server struct {
+	srv  *http.Server
+	code chan string
+}
+
+func newOauth2Server(expectedState string) *oauth2Server {
+	ch := make(chan string)
+
+	return &oauth2Server{
+		code: ch,
+		srv: &http.Server{
+			Addr: ":0",
+			Handler: http.HandlerFunc(func(resp http.ResponseWriter, req *http.Request) {
+				if err := req.ParseForm(); err != nil {
+					writeError(resp, err, http.StatusBadRequest)
+					return
+				}
+				state := req.Form.Get("state")
+				if state != expectedState {
+					writeError(resp, fmt.Errorf("invalid state: %q", state), http.StatusBadRequest)
+					return
+				}
+				code := req.Form.Get("code")
+				if code == "" {
+					writeError(resp, fmt.Errorf("missing code"), http.StatusBadRequest)
+					return
+				}
+				ch <- code
+				close(ch)
+
+				resp.Header().Add("Content-Type", "text/plain")
+				resp.WriteHeader(200)
+				_, _ = resp.Write([]byte(successMsg))
+			}),
+		},
+	}
+}
+
+// Start the oauth2Server asynchronously.
+func (s *oauth2Server) Start() (string, error) {
+	l, err := net.Listen("tcp", ":0") //nolint:gosec
+	if err != nil {
+		return "", err
+	}
+	go func() {
+		// This always returns an error when the server is closed.
+		_ = s.srv.Serve(l)
+	}()
+	return fmt.Sprintf("localhost:%d", l.Addr().(*net.TCPAddr).Port), nil
+}
+
+// Close shuts down the oauth2Server.
+func (s *oauth2Server) Close() {
+	_ = s.srv.Close()
+}
+
+// WaitForCode waits for the oauth2Server to receive a code.
+// This is a blocking call. Errors are ignored.
+func (s *oauth2Server) WaitForCode() string {
+	return <-s.code
+}
+
+func writeError(resp http.ResponseWriter, err error, code int) {
+	resp.WriteHeader(code)
+	_, _ = resp.Write([]byte(fmt.Sprintf("Error: %v", err)))
+}
diff --git a/go.mod b/go.mod
index 1c174aa..a6b7966 100644
--- a/go.mod
+++ b/go.mod
@@ -9,28 +9,28 @@ require (
 	github.com/pmezard/go-difflib v1.0.0
 	github.com/spf13/cobra v1.4.0
 	github.com/stretchr/testify v1.7.1
-	golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5
-	google.golang.org/api v0.75.0
+	golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a
+	google.golang.org/api v0.73.0
 	gopkg.in/yaml.v2 v2.4.0
 )
 
 require (
-	cloud.google.com/go/compute v1.6.0 // indirect
+	cloud.google.com/go/compute v1.5.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
-	github.com/googleapis/gax-go/v2 v2.3.0 // indirect
+	github.com/googleapis/gax-go/v2 v2.1.1 // indirect
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	go.opencensus.io v0.23.0 // indirect
-	golang.org/x/net v0.0.0-20220412020605-290c469a71a5 // indirect
-	golang.org/x/sys v0.0.0-20220412211240-33da011f77ad // indirect
+	golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
+	golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5 // indirect
 	golang.org/x/text v0.3.7 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20220414192740-2d67ff6cf2b4 // indirect
+	google.golang.org/genproto v0.0.0-20220310185008-1973136f34c6 // indirect
 	google.golang.org/grpc v1.45.0 // indirect
-	google.golang.org/protobuf v1.28.0 // indirect
+	google.golang.org/protobuf v1.27.1 // indirect
 	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
 	sigs.k8s.io/yaml v1.1.0 // indirect
 )
diff --git a/go.sum b/go.sum
index 7c414b0..e73b7f2 100644
--- a/go.sum
+++ b/go.sum
@@ -36,9 +36,8 @@ cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4g
 cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
 cloud.google.com/go/compute v0.1.0/go.mod h1:GAesmwr110a34z04OlxYkATPBEfVhkymfTBXtfbBFow=
 cloud.google.com/go/compute v1.3.0/go.mod h1:cCZiE1NHEtai4wiufUhW8I8S1JKkAnhnQJWM7YD99wM=
+cloud.google.com/go/compute v1.5.0 h1:b1zWmYuuHz7gO9kDcM/EpHGr06UgsYNRpNJzI2kFiLM=
 cloud.google.com/go/compute v1.5.0/go.mod h1:9SMHyhJlzhlkJqrPAc839t2BZFTSk6Jdj6mkzQJeu0M=
-cloud.google.com/go/compute v1.6.0 h1:XdQIN5mdPTSBVwSIVDuY5e8ZzVAccsHvD3qTEz4zIps=
-cloud.google.com/go/compute v1.6.0/go.mod h1:T29tfhtVbq1wvAPo0E3+7vhgmkOYeXjhFvz/FMzPu0s=
 cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
 cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
 cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
@@ -163,10 +162,8 @@ github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+
 github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
 github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
 github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0=
+github.com/googleapis/gax-go/v2 v2.1.1 h1:dp3bWCh+PPO1zjRRiCSczJav13sBvG4UhNyVTa1KqdU=
 github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM=
-github.com/googleapis/gax-go/v2 v2.2.0/go.mod h1:as02EH8zWkzwUoLbBaFeQ+arQaj/OthfcblKl4IGNaM=
-github.com/googleapis/gax-go/v2 v2.3.0 h1:nRJtk3y8Fm770D42QV6T90ZnvFZyk7agSo3Q+Z9p3WI=
-github.com/googleapis/gax-go/v2 v2.3.0/go.mod h1:b8LNqSzNabLiUpXKkY7HAR5jr6bIT99EXz9pXxye9YM=
 github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
 github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
 github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
@@ -301,10 +298,8 @@ golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLd
 golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
 golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/net v0.0.0-20220225172249-27dd8689420f h1:oA4XRj0qtSt8Yo1Zms0CUlsT3KG69V2UGQWPBxujDmc=
 golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
-golang.org/x/net v0.0.0-20220325170049-de3da57026de/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
-golang.org/x/net v0.0.0-20220412020605-290c469a71a5 h1:bRb386wvrE+oBNdF1d/Xh9mQrfQ4ecYhW5qJ5GvTGT4=
-golang.org/x/net v0.0.0-20220412020605-290c469a71a5/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -321,10 +316,8 @@ golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ
 golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
 golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
-golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
+golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a h1:qfl7ob3DIEs3Ml9oLuPwY2N04gymzAW04WsUQHIClgM=
 golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
-golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5 h1:OSnWWcOd/CtWQC2cYSBgbTSJv3ciqd8r54ySIW2y3RE=
-golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -388,10 +381,8 @@ golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220412211240-33da011f77ad h1:ntjMns5wyP/fN65tdBD4g8J5w8n015+iIIs9rtjXkY0=
-golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5 h1:y/woIyUBFbpQGKS0u1aHF/40WUDnek3fPOyD08H5Vng=
+golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -461,9 +452,8 @@ golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
 golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20220411194840-2f41105eb62f h1:GGU+dLjvlC3qDwqYgL6UgRmHXhOOgns0bZu2Ty5mm6U=
-golang.org/x/xerrors v0.0.0-20220411194840-2f41105eb62f/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
 google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
@@ -497,10 +487,8 @@ google.golang.org/api v0.61.0/go.mod h1:xQRti5UdCmoCEqFxcz93fTl338AVqDgyaDRuOZ3h
 google.golang.org/api v0.63.0/go.mod h1:gs4ij2ffTRXwuzzgJl/56BdwJaA194ijkfn++9tDuPo=
 google.golang.org/api v0.67.0/go.mod h1:ShHKP8E60yPsKNw/w8w+VYaj9H6buA5UqDp8dhbQZ6g=
 google.golang.org/api v0.70.0/go.mod h1:Bs4ZM2HGifEvXwd50TtW70ovgJffJYw2oRCOFU/SkfA=
-google.golang.org/api v0.71.0/go.mod h1:4PyU6e6JogV1f9eA4voyrTY2batOLdgZ5qZ5HOCc4j8=
-google.golang.org/api v0.74.0/go.mod h1:ZpfMZOVRMywNyvJFeqL9HRWBgAuRfSjJFpe9QtRRyDs=
-google.golang.org/api v0.75.0 h1:0AYh/ae6l9TDUvIQrDw5QRpM100P6oHgD+o3dYHMzJg=
-google.golang.org/api v0.75.0/go.mod h1:pU9QmyHLnzlpar1Mjt4IbapUCy8J+6HD6GeELN69ljA=
+google.golang.org/api v0.73.0 h1:O9bThUh35K1rvUrQwTUQ1eqLC/IYyzUpWavYIO2EXvo=
+google.golang.org/api v0.73.0/go.mod h1:lbd/q6BRFJbdpV6OUCXstVeiI5mL/d3/WifG7iNKnjI=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -573,13 +561,8 @@ google.golang.org/genproto v0.0.0-20220126215142-9970aeb2e350/go.mod h1:5CzLGKJ6
 google.golang.org/genproto v0.0.0-20220207164111-0872dc986b00/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc=
 google.golang.org/genproto v0.0.0-20220218161850-94dd64e39d7c/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI=
 google.golang.org/genproto v0.0.0-20220222213610-43724f9ea8cf/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI=
-google.golang.org/genproto v0.0.0-20220304144024-325a89244dc8/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI=
+google.golang.org/genproto v0.0.0-20220310185008-1973136f34c6 h1:FglFEfyj61zP3c6LgjmVHxYxZWXYul9oiS1EZqD5gLc=
 google.golang.org/genproto v0.0.0-20220310185008-1973136f34c6/go.mod h1:kGP+zUP2Ddo0ayMi4YuN7C3WZyJvGLZRh8Z5wnAqvEI=
-google.golang.org/genproto v0.0.0-20220324131243-acbaeb5b85eb/go.mod h1:hAL49I2IFola2sVEjAn7MEwsja0xp51I0tlGAf9hz4E=
-google.golang.org/genproto v0.0.0-20220407144326-9054f6ed7bac/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo=
-google.golang.org/genproto v0.0.0-20220413183235-5e96e2839df9/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo=
-google.golang.org/genproto v0.0.0-20220414192740-2d67ff6cf2b4 h1:myaecH64R0bIEDjNORIel4iXubqzaHU1K2z8ajBwWcM=
-google.golang.org/genproto v0.0.0-20220414192740-2d67ff6cf2b4/go.mod h1:8w6bsBMX6yCPbAVTeqQHvzxW0EIFigd5lZyahWgyfDo=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
 google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
@@ -622,9 +605,8 @@ google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGj
 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
+google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ=
 google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
-google.golang.org/protobuf v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw=
-google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
diff --git a/internal/engine/api/api.go b/internal/engine/api/api.go
index 2180058..0a39268 100644
--- a/internal/engine/api/api.go
+++ b/internal/engine/api/api.go
@@ -3,7 +3,9 @@ package api
 import (
 	"fmt"
 	"net/http"
+	"strings"
 
+	"golang.org/x/oauth2"
 	"google.golang.org/api/gmail/v1"
 	"google.golang.org/api/googleapi"
 
@@ -17,6 +19,7 @@ const (
 	gmailUser       = "me"
 	labelTypeSystem = "system"
 	labelDocsURL    = "https://developers.google.com/gmail/api/v1/reference/users/labels#resource"
+	authExpiredURL  = "https://github.com/mbrt/gmailctl#oauth2-authentication-errors"
 )
 
 // NewFromService creates a new GmailAPI instance from the given Gmail service.
@@ -44,7 +47,7 @@ func (g *GmailAPI) ListFilters() (filter.Filters, error) {
 
 	apires, err := g.service.Users.Settings.Filters.List(gmailUser).Do(g.opts...)
 	if err != nil {
-		return nil, err
+		return nil, annotateError(err)
 	}
 	return api.Import(apires.Filter, lmap)
 }
@@ -54,7 +57,7 @@ func (g *GmailAPI) DeleteFilters(ids []string) error {
 	for _, id := range ids {
 		err := g.service.Users.Settings.Filters.Delete(gmailUser, id).Do(g.opts...)
 		if err != nil {
-			return fmt.Errorf("deleting filter %q: %w", id, err)
+			return fmt.Errorf("deleting filter %q: %w", id, annotateError(err))
 		}
 	}
 	return nil
@@ -75,7 +78,7 @@ func (g *GmailAPI) AddFilters(fs filter.Filters) error {
 	for i, gfilter := range gfilters {
 		_, err = g.service.Users.Settings.Filters.Create(gmailUser, gfilter).Do(g.opts...)
 		if err != nil {
-			return fmt.Errorf("creating filter %d: %w", i, err)
+			return fmt.Errorf("creating filter %d: %w", i, annotateError(err))
 		}
 	}
 
@@ -86,7 +89,7 @@ func (g *GmailAPI) AddFilters(fs filter.Filters) error {
 func (g *GmailAPI) ListLabels() (label.Labels, error) {
 	apires, err := g.service.Users.Labels.List(gmailUser).Do(g.opts...)
 	if err != nil {
-		return nil, err
+		return nil, annotateError(err)
 	}
 
 	var res label.Labels
@@ -120,7 +123,7 @@ func (g *GmailAPI) DeleteLabels(ids []string) error {
 	for _, id := range ids {
 		err := g.service.Users.Labels.Delete(gmailUser, id).Do(g.opts...)
 		if err != nil {
-			return fmt.Errorf("deleting label %q: %w", id, err)
+			return fmt.Errorf("deleting label %q: %w", id, annotateError(err))
 		}
 	}
 	return nil
@@ -132,7 +135,7 @@ func (g *GmailAPI) AddLabels(lbs label.Labels) error {
 	for _, lb := range lbs {
 		_, err := g.service.Users.Labels.Create(gmailUser, labelToGmailAPI(lb)).Do(g.opts...)
 		if err != nil {
-			return labelError(fmt.Errorf("creating label %q: %w", lb.Name, err))
+			return annotateError(fmt.Errorf("creating label %q: %w", lb.Name, err))
 		}
 	}
 	return nil
@@ -148,7 +151,7 @@ func (g *GmailAPI) UpdateLabels(lbs label.Labels) error {
 		}
 		_, err := g.service.Users.Labels.Patch(gmailUser, lb.ID, labelToGmailAPI(lb)).Do(g.opts...)
 		if err != nil {
-			return labelError(fmt.Errorf("patching label %q: %w", lb.Name, err))
+			return annotateError(fmt.Errorf("patching label %q: %w", lb.Name, err))
 		}
 	}
 	return nil
@@ -176,11 +179,29 @@ func labelToGmailAPI(lb label.Label) *gmail.Label {
 	}
 }
 
-func labelError(err error) error {
+func annotateError(err error) error {
+	var oerr *oauth2.RetrieveError
+	if errors.As(err, &oerr) {
+		if strings.Contains(string(oerr.Body), "invalid_grant") {
+			return errors.WithDetails(err,
+				"Possible expired token. Try refreshing with `gmailctl init --refresh-expired`",
+				"More help at "+authExpiredURL,
+			)
+		}
+	}
+
 	var gerr *googleapi.Error
-	if errors.As(err, &gerr) && gerr.Code == http.StatusBadRequest {
-		return errors.WithDetails(err,
-			fmt.Sprintf("See the allowed set of color values here: %s", labelDocsURL))
+	if !errors.As(err, &gerr) {
+		return err
+	}
+	if gerr.Code != http.StatusBadRequest {
+		return err
+	}
+	for _, e := range gerr.Errors {
+		if e.Reason == "invalidArgument" && strings.Contains(e.Message, "color palette") {
+			return errors.WithDetails(err,
+				fmt.Sprintf("See the allowed set of color values here: %s", labelDocsURL))
+		}
 	}
 	return err
 }
diff --git a/internal/engine/api/auth.go b/internal/engine/api/auth.go
index ae1724a..949efb1 100644
--- a/internal/engine/api/auth.go
+++ b/internal/engine/api/auth.go
@@ -62,7 +62,8 @@ func (a Authenticator) API(ctx context.Context, token io.Reader) (*GmailAPI, err
 
 // AuthURL returns the URL the user has to visit to authorize the
 // application and obtain an auth code.
-func (a Authenticator) AuthURL() string {
+func (a Authenticator) AuthURL(redirectURL string) string {
+	a.cfg.RedirectURL = redirectURL
 	return a.cfg.AuthCodeURL(a.State, oauth2.AccessTypeOffline)
 }
 
diff --git a/internal/engine/filter/filter.go b/internal/engine/filter/filter.go
index 3ee757c..ffc585e 100644
--- a/internal/engine/filter/filter.go
+++ b/internal/engine/filter/filter.go
@@ -1,9 +1,7 @@
 package filter
 
 import (
-	"bytes"
 	"fmt"
-	"io"
 	"strings"
 
 	"github.com/mbrt/gmailctl/internal/engine/gmail"
@@ -52,8 +50,7 @@ func (f Filter) String() string {
 	w.WriteParam("from", f.Criteria.From)
 	w.WriteParam("to", f.Criteria.To)
 	w.WriteParam("subject", f.Criteria.Subject)
-
-	w.WriteParam("query", indent(f.Criteria.Query, 2))
+	w.WriteParam("query", f.Criteria.Query)
 
 	w.WriteString("  Actions:\n")
 	w.WriteBool("archive", f.Action.Archive)
@@ -70,93 +67,6 @@ func (f Filter) String() string {
 	return w.String()
 }
 
-func indent(query string, level int) string {
-	var indented bytes.Buffer
-	if !indentInternal(strings.NewReader(query), &indented, level+1) {
-		return query
-	}
-	return "\n" + strings.TrimRight(indented.String(), "\n ")
-}
-
-func indentInternal(queryReader io.RuneReader, out *bytes.Buffer, level int) bool {
-	type parseState int
-	const (
-		other parseState = iota
-		skipSpaces
-		inQuotes
-	)
-
-	for i := 0; i < level; i++ {
-		out.Write([]byte("  "))
-	}
-
-	indentationWasNeeded := false
-	writeIndentation := func(n int) {
-		out.WriteByte('\n')
-		for i := 0; i < n; i++ {
-			out.Write([]byte("  "))
-		}
-		indentationWasNeeded = true
-	}
-
-	state := skipSpaces
-
-	for {
-		r, _, err := queryReader.ReadRune()
-		if err != nil {
-			break
-		}
-		switch state {
-		case inQuotes:
-			out.WriteRune(r)
-			if r == '"' {
-				state = other
-			}
-		case skipSpaces, other:
-			switch r {
-			case ' ':
-				if state == skipSpaces {
-					continue
-				}
-				writeIndentation(level)
-
-			case '{', '(':
-				out.WriteRune(r)
-
-				level++
-				writeIndentation(level)
-
-				state = skipSpaces
-
-			case '}', ')':
-				writeIndentation(level - 1)
-				out.WriteRune(r)
-
-				level--
-				writeIndentation(level)
-
-				state = skipSpaces
-
-			case ':':
-				out.WriteByte(':')
-				state = skipSpaces
-
-			case '"':
-				state = inQuotes
-				out.WriteByte('"')
-
-			default:
-				if state != inQuotes {
-					state = other
-				}
-				out.WriteRune(r)
-			}
-		}
-	}
-
-	return indentationWasNeeded
-}
-
 // HasLabel returns true if the given label is used by the filter.
 func (f Filter) HasLabel(name string) bool {
 	return f.Action.AddLabel == name


Test patch:
diff --git a/internal/engine/filter/diff_test.go b/internal/engine/filter/diff_test.go
index 68611d8..d7fc0f5 100644
--- a/internal/engine/filter/diff_test.go
+++ b/internal/engine/filter/diff_test.go
@@ -43,8 +43,7 @@ func TestDiffOutput(t *testing.T) {
 		{
 			ID: "abcdefg",
 			Criteria: Criteria{
-				From:  "someone@gmail.com",
-				Query: "(a b) subject:(foo bar)",
+				From: "someone@gmail.com",
 			},
 			Action: Actions{
 				MarkRead: true,
@@ -55,8 +54,7 @@ func TestDiffOutput(t *testing.T) {
 	new := Filters{
 		{
 			Criteria: Criteria{
-				From:  "{someone@gmail.com else@gmail.com}",
-				Query: "(a c) subject:(foo baz)",
+				From: "{someone@gmail.com else@gmail.com}",
 			},
 			Action: Actions{
 				MarkRead: true,
@@ -71,21 +69,10 @@ func TestDiffOutput(t *testing.T) {
 	expected := `
 --- Current
 +++ TO BE APPLIED
-@@ -1,15 +1,15 @@
+@@ -1,6 +1,6 @@
  * Criteria:
 -    from: someone@gmail.com
 +    from: {someone@gmail.com else@gmail.com}
-     query: 
-       (
-         a
--        b
-+        c
-       )
-       subject:(
-         foo
--        bar
-+        baz
-       )
    Actions:
      mark as read
      categorize as: personal`
@@ -335,45 +322,3 @@ func TestDuplicate(t *testing.T) {
 	// Only one of the two identical filters is present
 	assert.Equal(t, new[1:], fd.Added)
 }
-
-func TestIndent(t *testing.T) {
-	testCases := []struct{ name, query, want string }{
-		{"no_newline_necessary", `from:"foo bar"`, `from:"foo bar"`},
-		{"quotes", `from:"foo bar" "another foo bar" "good thing gmail doesn't support escaping" "re: something"`, `
-  from:"foo bar"
-  "another foo bar"
-  "good thing gmail doesn't support escaping"
-  "re: something"`},
-		{"parens", `(a b) subject:(foo bar "re: something")`, `
-  (
-    a
-    b
-  )
-  subject:(
-    foo
-    bar
-    "re: something"
-  )`},
-		{"negations_and_braces", `-(x {y -z} l)`, `
-  -(
-    x
-    {
-      y
-      -z
-    }
-    l
-  )`},
-		{"unicode", `日(本)語`, `
-  日(
-    本
-  )
-  語`},
-	}
-
-	for _, tc := range testCases {
-		t.Run(tc.name, func(t *testing.T) {
-			got := indent(tc.query, 0)
-			assert.Equal(t, tc.want, got)
-		})
-	}
-}
diff --git a/testdata/valid/02-add-one.diff b/testdata/valid/02-add-one.diff
index 4f0a40d..7a0e563 100644
--- a/testdata/valid/02-add-one.diff
+++ b/testdata/valid/02-add-one.diff
@@ -1,7 +1,7 @@
 Filters:
 --- Current
 +++ TO BE APPLIED
-@@ -1 +1,123 @@
+@@ -1 +1,119 @@
 +* Criteria:
 +    query: replyto:replyer@gmail.com
 +  Actions:
@@ -59,9 +59,7 @@ Filters:
 +    apply label: label2
 +
 +* Criteria:
-+    query: 
-+      cc:peeker@yahoo.com
-+      -subject:"a subject"
++    query: cc:peeker@yahoo.com -subject:"a subject"
 +  Actions:
 +    archive
 +    mark as important
@@ -85,9 +83,7 @@ Filters:
 +    forward to: forward-address@gmail.com
 +
 +* Criteria:
-+    query: 
-+      cc:peeker@yahoo.com
-+      -subject:"a subject"
++    query: cc:peeker@yahoo.com -subject:"a subject"
 +  Actions:
 +    apply label: label2
 +
diff --git a/testdata/valid/03-labels.diff b/testdata/valid/03-labels.diff
index 873ae37..875017d 100644
--- a/testdata/valid/03-labels.diff
+++ b/testdata/valid/03-labels.diff
@@ -1,7 +1,7 @@
 Filters:
 --- Current
 +++ TO BE APPLIED
-@@ -7,28 +7,26 @@
+@@ -5,28 +5,26 @@
      mark as important
      never mark as spam
      mark as read
@@ -31,7 +31,7 @@ Filters:
  * Criteria:
      query: replyto:replyer@gmail.com
    Actions:
-@@ -36,11 +34,10 @@
+@@ -34,11 +32,10 @@
      mark as important
      never mark as spam
      mark as read
@@ -43,7 +43,7 @@ Filters:
  * Criteria:
      query: bcc:bccer@gmail.com
    Actions:
-@@ -48,11 +45,10 @@
+@@ -46,11 +43,10 @@
      mark as important
      never mark as spam
      mark as read
@@ -55,7 +55,7 @@ Filters:
  * Criteria:
      from: someone@gmail.com
    Actions:
-@@ -60,11 +56,10 @@
+@@ -58,11 +54,10 @@
      mark as important
      never mark as spam
      mark as read
@@ -67,7 +67,7 @@ Filters:
  * Criteria:
      query: is:muted
    Actions:
-@@ -72,11 +67,10 @@
+@@ -70,11 +65,10 @@
      mark as important
      never mark as spam
      mark as read
@@ -79,7 +79,7 @@ Filters:
  * Criteria:
      to: someone-else@gmail.com
    Actions:
-@@ -84,45 +78,7 @@
+@@ -82,43 +76,7 @@
      mark as important
      never mark as spam
      mark as read
@@ -109,9 +109,7 @@ Filters:
 -    apply label: label2
 -
 -* Criteria:
--    query: 
--      cc:peeker@yahoo.com
--      -subject:"a subject"
+-    query: cc:peeker@yahoo.com -subject:"a subject"
 -  Actions:
 -    apply label: label2
 -
diff --git a/testdata/valid/05-bigdiff.diff b/testdata/valid/05-bigdiff.diff
index 0db6cb1..eb3b995 100644
--- a/testdata/valid/05-bigdiff.diff
+++ b/testdata/valid/05-bigdiff.diff
@@ -1,43 +1,28 @@
 Filters:
 --- Current
 +++ TO BE APPLIED
-@@ -1,84 +1,95 @@
+@@ -1,82 +1,70 @@
  * Criteria:
 -    to: someone-else@gmail.com
-+    from: spammer2
-   Actions:
--    archive
--    mark as important
--    never mark as spam
--    mark as read
--    star
--    categorize as: social
--    forward to: forward-address@gmail.com
++    query: list:foobaz.mail.com -"action needed"
++  Actions:
 +    delete
- 
- * Criteria:
--    query: is:muted
-+    query: "buy this thing"
++
++* Criteria:
++    query: list:{list3 list1 list4 list6} -to:none@gmail.com
    Actions:
--    archive
+     archive
 -    mark as important
 -    never mark as spam
 -    mark as read
 -    star
 -    categorize as: social
 -    forward to: forward-address@gmail.com
-+    delete
++    categorize as: personal
++    apply label: maillist
  
  * Criteria:
-     query: 
--      cc:peeker@yahoo.com
--      -subject:"a subject"
-+      list:foobaz.mail.com
-+      -"action needed"
-+  Actions:
-+    delete
-+
-+* Criteria:
+-    query: is:muted
 +    to: alias@gmail.com
 +  Actions:
 +    categorize as: promotions
@@ -65,66 +50,20 @@ Filters:
  
  * Criteria:
 -    from: someone@gmail.com
-+    to: pippo+spammy@gmail.com
-+  Actions:
-+    delete
-+
-+* Criteria:
-+    query: bcc:aaaa@gmail.com
-+  Actions:
-+    categorize as: updates
-+
-+* Criteria:
-+    query: 
-+      list:{
-+        list3
-+        list1
-+        list4
-+        list6
-+      }
-+      -to:none@gmail.com
-+  Actions:
-+    apply label: differentlabel
-+
-+* Criteria:
-+    from: spammer1
-+    subject: "spam mail"
-+    query: 
-+      cc:foo@baz.com
-+      bcc:bar@baz.com
-+  Actions:
-+    delete
-+
-+* Criteria:
-+    query: 
-+      list:{
-+        list3
-+        list1
-+        list4
-+        list6
-+      }
-+      -to:none@gmail.com
++    query: list:{list3 list1 list4 list6} -to:none@gmail.com
    Actions:
-     archive
+-    archive
 -    mark as important
 -    never mark as spam
 -    mark as read
 -    star
 -    categorize as: social
 -    forward to: forward-address@gmail.com
-+    categorize as: personal
-+    apply label: maillist
++    apply label: thirdlabel
  
  * Criteria:
 -    query: bcc:bccer@gmail.com
-+    query: 
-+      list:{
-+        list3
-+        list1
-+        list4
-+        list6
-+      }
-+      -to:none@gmail.com
++    to: pippo+spammy@gmail.com
    Actions:
 -    archive
 -    mark as important
@@ -133,11 +72,12 @@ Filters:
 -    star
 -    categorize as: social
 -    forward to: forward-address@gmail.com
-+    apply label: thirdlabel
++    delete
  
--* Criteria:
+ * Criteria:
 -    query: replyto:replyer@gmail.com
--  Actions:
++    query: bcc:aaaa@gmail.com
+   Actions:
 -    archive
 -    mark as important
 -    never mark as spam
@@ -145,10 +85,12 @@ Filters:
 -    star
 -    categorize as: social
 -    forward to: forward-address@gmail.com
--
--* Criteria:
++    categorize as: updates
+ 
+ * Criteria:
 -    query: "something in the body"
--  Actions:
++    from: spammer2
+   Actions:
 -    archive
 -    mark as important
 -    never mark as spam
@@ -156,12 +98,35 @@ Filters:
 -    star
 -    categorize as: social
 -    forward to: forward-address@gmail.com
--
--* Criteria:
++    delete
+ 
+ * Criteria:
 -    query: list:maillist@google.com
--  Actions:
++    query: "buy this thing"
+   Actions:
 -    never mark as important
--
++    delete
+ 
+ * Criteria:
+-    query: cc:peeker@yahoo.com -subject:"a subject"
++    query: list:{list3 list1 list4 list6} -to:none@gmail.com
+   Actions:
+-    archive
+-    mark as important
+-    never mark as spam
+-    mark as read
+-    star
+-    categorize as: social
+-    forward to: forward-address@gmail.com
++    apply label: differentlabel
+ 
++* Criteria:
++    from: spammer1
++    subject: "spam mail"
++    query: cc:foo@baz.com bcc:bar@baz.com
++  Actions:
++    delete
++
 
 Labels:
 --- Current
diff --git a/testdata/valid/06-bigsplit.diff b/testdata/valid/06-bigsplit.diff
index 7171e10..fdc8fe9 100644
--- a/testdata/valid/06-bigsplit.diff
+++ b/testdata/valid/06-bigsplit.diff
@@ -1,95 +1,23 @@
 Filters:
 --- Current
 +++ TO BE APPLIED
-@@ -1,95 +1,72 @@
+@@ -1,70 +1,15 @@
  * Criteria:
-     query: 
-       list:{
--        list3
--        list1
--        list4
--        list6
-+        list40
-+        list41
-+        list42
-+        list43
-+        list44
-+        list45
-+        list46
-+        list47
-+        list48
-+        list49
-+        list50
-       }
--      -to:none@gmail.com
+-    query: bcc:aaaa@gmail.com
++    query: list:{list40 list41 list42 list43 list44 list45 list46 list47 list48 list49 list50}
    Actions:
-     archive
--    categorize as: personal
--    apply label: maillist
+-    categorize as: updates
++    archive
  
  * Criteria:
-     query: 
-       list:{
--        list3
--        list1
--        list4
--        list6
-+        list20
-+        list21
-+        list22
-+        list23
-+        list24
-+        list25
-+        list26
-+        list27
-+        list28
-+        list29
-+        list30
-+        list31
-+        list32
-+        list33
-+        list34
-+        list35
-+        list36
-+        list37
-+        list38
-+        list39
-       }
--      -to:none@gmail.com
+-    query: list:{list3 list1 list4 list6} -to:none@gmail.com
++    query: list:{list20 list21 list22 list23 list24 list25 list26 list27 list28 list29 list30 list31 list32 list33 list34 list35 list36 list37 list38 list39}
    Actions:
--    apply label: differentlabel
-+    archive
+     archive
+-    categorize as: personal
+-    apply label: maillist
  
  * Criteria:
-     query: 
-       list:{
-+        list0
-+        list1
-+        list2
-         list3
--        list1
-         list4
-+        list5
-         list6
-+        list7
-+        list8
-+        list9
-+        list10
-+        list11
-+        list12
-+        list13
-+        list14
-+        list15
-+        list16
-+        list17
-+        list18
-+        list19
-       }
--      -to:none@gmail.com
--  Actions:
--    apply label: thirdlabel
--
--* Criteria:
 -    from: spammer2
 -  Actions:
 -    delete
@@ -100,18 +28,19 @@ Filters:
 -    delete
 -
 -* Criteria:
+-    query: list:{list3 list1 list4 list6} -to:none@gmail.com
+-  Actions:
+-    apply label: differentlabel
+-
+-* Criteria:
 -    from: spammer1
 -    subject: "spam mail"
--    query: 
--      cc:foo@baz.com
--      bcc:bar@baz.com
+-    query: cc:foo@baz.com bcc:bar@baz.com
 -  Actions:
 -    delete
 -
 -* Criteria:
--    query: 
--      list:foobaz.mail.com
--      -"action needed"
+-    query: list:foobaz.mail.com -"action needed"
 -  Actions:
 -    delete
 -
@@ -131,18 +60,19 @@ Filters:
 -    from: notfriend@gmail.com
 -    subject: "hey there"
 -    query: -to:none@gmail.com
++    query: list:{list0 list1 list2 list3 list4 list5 list6 list7 list8 list9 list10 list11 list12 list13 list14 list15 list16 list17 list18 list19}
    Actions:
      archive
 -    star
 -    categorize as: forums
  
 -* Criteria:
--    to: pippo+spammy@gmail.com
+-    query: list:{list3 list1 list4 list6} -to:none@gmail.com
 -  Actions:
--    delete
+-    apply label: thirdlabel
 -
 -* Criteria:
--    query: bcc:aaaa@gmail.com
+-    to: pippo+spammy@gmail.com
 -  Actions:
--    categorize as: updates
+-    delete
 -
diff --git a/testdata/valid/07-deleteall.diff b/testdata/valid/07-deleteall.diff
index ee0a4f4..8a8b47b 100644
--- a/testdata/valid/07-deleteall.diff
+++ b/testdata/valid/07-deleteall.diff
@@ -1,76 +1,19 @@
 Filters:
 --- Current
 +++ TO BE APPLIED
-@@ -1,72 +1 @@
+@@ -1,15 +1 @@
 -* Criteria:
--    query: 
--      list:{
--        list40
--        list41
--        list42
--        list43
--        list44
--        list45
--        list46
--        list47
--        list48
--        list49
--        list50
--      }
+-    query: list:{list40 list41 list42 list43 list44 list45 list46 list47 list48 list49 list50}
 -  Actions:
 -    archive
  
 -* Criteria:
--    query: 
--      list:{
--        list20
--        list21
--        list22
--        list23
--        list24
--        list25
--        list26
--        list27
--        list28
--        list29
--        list30
--        list31
--        list32
--        list33
--        list34
--        list35
--        list36
--        list37
--        list38
--        list39
--      }
+-    query: list:{list20 list21 list22 list23 list24 list25 list26 list27 list28 list29 list30 list31 list32 list33 list34 list35 list36 list37 list38 list39}
 -  Actions:
 -    archive
 -
 -* Criteria:
--    query: 
--      list:{
--        list0
--        list1
--        list2
--        list3
--        list4
--        list5
--        list6
--        list7
--        list8
--        list9
--        list10
--        list11
--        list12
--        list13
--        list14
--        list15
--        list16
--        list17
--        list18
--        list19
--      }
+-    query: list:{list0 list1 list2 list3 list4 list5 list6 list7 list8 list9 list10 list11 list12 list13 list14 list15 list16 list17 list18 list19}
 -  Actions:
 -    archive
 -