Repository: getsops/sops (language: python)

Issue:
Encrypt dotenv file inplace more than once should be failsafe.
Oh no It hit me again. After changing the configuration in *.sops.yaml* I usually run `sops -i -d; sops -i -e` on the encrypted files to apply the changes. (Maybe there is a better way to do this ?)

But if I run `sops -i -e` twice on a file, then the file can not be encrypted afterwards and sops returns the error  **MAC mismatch. File has ...**. Any succeeding `sops -i -e` call on a file should abort if the file is already encrypted. 


Discussion / hints:
I have tested @r10r issue and confirm with SOPS v3.8.1

Error Response
```
$ sops -i -e testing.enc.sops.yaml
The file you have provided contains a top-level entry called 'sops'. This
is generally due to the file already being encrypted. SOPS uses a top-level
entry called 'sops' to store the metadata required to decrypt the file. For
this reason, SOPS can not encrypt files that already contain such an entry.

If this is an unencrypted file, rename the 'sops' entry.

If this is an encrypted file and you want to edit it, use the editor mode,
for example: `sops my_file.yaml`
```

I would suspect this is a current expected output.

You are interacting with the file humanistically so you're getting a humanistic message back. If you just are editing a SOPS encrypted file, just use `sops file`, it will open in an editor and when you exit the editor, it'll save it.

You could also update your command: `sops -i -d testing.enc.sops.yaml; sops -i -e testing.enc.sops.yaml`
To something more safe: `sops -i -d testing.enc.sops.yaml; if [[ $(cat testing.enc.sops.yaml | grep -c 'sops:') -gt 0 ]]; then echo 'file already encrypted'; else sops -i -e testing.enc.sops.yaml;fi`

When interacting with a SOPS encrypted/decrypted file programmatically, you can build this kind of logic into the program, and it is a noop at that point.

Hope this helps ever so slightly while you keep working while waiting on the maintainers to decide how they want to proceed and get back to you.

TTYL,
Peter
@r10r which version of sops are you using? Sops 3.8.1 does not seem to show the behavior you claim.
@silentpete  Thanks for testing this. I'll consider to create a script with the proposed update command.

> @r10r which version of sops are you using? Sops 3.8.1 does not seem to show the behavior you claim.

sops v3.8.1-49-g418ea23e3

I use a development version to be able set yaml indent to 2 to avoid re-indentation for our yaml files.
You should be able to reproduce the issue with the following command sequence:

```bash
echo HELLO=world > foo.env
echo "stores:" > .sops.yaml
sops -i -e foo.env 
sops -i -e foo.env 
sops -i -d foo.env 
```
error message: 

```
MAC mismatch. File has 712A6E486181E64F8949DAF476EAD3FF0924F26D69A28DADCA03B52BA9DF573C0D2E704FC17985EF0763DBF98C0FA8C3C81A20CF322FBB8AD57E2E600070E3C6, computed 1ADDF28E3BCC682983D0FD44D7C1DD81C15D24B22904B9858E6E7CE8D3C1F30C5FDE2F06A37041C0102E40F201B61EB4264C941F427F2A75C051A710727BB853
```

NOTE `echo stores: > .sops.yaml` is required or I get the following exception 

```
panic: runtime error: invalid memory address or nil pointer dereference                                                                                                                        
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xdfa520]                                                                                                                         
                                                                                                                                                                                               
goroutine 1 [running]:                                                                                                                                                                         
github.com/getsops/sops/v3/cmd/sops/common.newDotenvStore(0xf145e0?)                                                                                                                           
        /workdir/cmd/sops/common/common.go:46                                                                                                                                                  
github.com/getsops/sops/v3/cmd/sops/common.StoreForFormat(...)                                                                                                                                 
        /workdir/cmd/sops/common/common.go:162                                                                                                                                                 
github.com/getsops/sops/v3/cmd/sops/common.DefaultStoreForPathOrFormat(0x10bb462?, {0xc000451520, 0xd}, {0x0?, 0x10c35db?})                                                                    
        /workdir/cmd/sops/common/common.go:177 +0x9d                                                                                                                                           
main.inputStore(0xc000187a20, {0xc000451520, 0xd})                                                                                                                                             
        /workdir/cmd/sops/main.go:1085 +0x65                                                                                                                                                   
main.main.func8(0xc000187a20)                                                                                                                                                                  
        /workdir/cmd/sops/main.go:796 +0x770                                                                                                                                                   
github.com/urfave/cli.HandleAction({0xec9b80?, 0x1125d88?}, 0xc00046d6c0?)                                                                                                                     
        /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:524 +0x50
github.com/urfave/cli.(*App).Run(0xc00046d6c0, {0xc0000400c0, 0x4, 0x4})
        /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:286 +0x766
main.main()
        /workdir/cmd/sops/main.go:1018 +0x35be
```

I can try to reproduce the issue with the current HEAD if required. 
Ok, I've tested it with a yaml file and then I'll get the expected error response. So it seems to be limited to dotenv files.
My guess is that it happens both with INI and dotenv files, since both of them employ flattening of metadata.
(wrong tab)
> NOTE `echo stores: > .sops.yaml` is required or I get the following exception
> 
> ```
> panic: runtime error: invalid memory address or nil pointer dereference                                                                                                                        
> [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xdfa520]                                                                                                                         
>                                                                                                                                                                                                
> goroutine 1 [running]:                                                                                                                                                                         
> github.com/getsops/sops/v3/cmd/sops/common.newDotenvStore(0xf145e0?)                                                                                                                           
>         /workdir/cmd/sops/common/common.go:46                                                                                                                                                  
> github.com/getsops/sops/v3/cmd/sops/common.StoreForFormat(...)                                                                                                                                 
>         /workdir/cmd/sops/common/common.go:162                                                                                                                                                 
> github.com/getsops/sops/v3/cmd/sops/common.DefaultStoreForPathOrFormat(0x10bb462?, {0xc000451520, 0xd}, {0x0?, 0x10c35db?})                                                                    
>         /workdir/cmd/sops/common/common.go:177 +0x9d                                                                                                                                           
> main.inputStore(0xc000187a20, {0xc000451520, 0xd})                                                                                                                                             
>         /workdir/cmd/sops/main.go:1085 +0x65                                                                                                                                                   
> main.main.func8(0xc000187a20)                                                                                                                                                                  
>         /workdir/cmd/sops/main.go:796 +0x770                                                                                                                                                   
> github.com/urfave/cli.HandleAction({0xec9b80?, 0x1125d88?}, 0xc00046d6c0?)                                                                                                                     
>         /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:524 +0x50
> github.com/urfave/cli.(*App).Run(0xc00046d6c0, {0xc0000400c0, 0x4, 0x4})
>         /go/pkg/mod/github.com/urfave/cli@v1.22.14/app.go:286 +0x766
> main.main()
>         /workdir/cmd/sops/main.go:1018 +0x35be
> ```
> 
> I can try to reproduce the issue with the current HEAD if required.

I cannot reproduce this part. Can you provide instructions on how to reproduce it, including the exact git commit you have been using?

Fix patch:
diff --git a/cmd/sops/encrypt.go b/cmd/sops/encrypt.go
index 826fa496a..266f48ae3 100644
--- a/cmd/sops/encrypt.go
+++ b/cmd/sops/encrypt.go
@@ -36,7 +36,8 @@ func (err *fileAlreadyEncryptedError) Error() string {
 
 func (err *fileAlreadyEncryptedError) UserError() string {
 	message := "The file you have provided contains a top-level entry called " +
-		"'sops'. This is generally due to the file already being encrypted. " +
+		"'sops', or for flat file formats top-level entries starting with " +
+		"'sops_'. This is generally due to the file already being encrypted. " +
 		"SOPS uses a top-level entry called 'sops' to store the metadata " +
 		"required to decrypt the file. For this reason, SOPS can not " +
 		"encrypt files that already contain such an entry.\n\n" +
@@ -47,10 +48,8 @@ func (err *fileAlreadyEncryptedError) UserError() string {
 }
 
 func ensureNoMetadata(opts encryptOpts, branch sops.TreeBranch) error {
-	for _, b := range branch {
-		if b.Key == "sops" {
-			return &fileAlreadyEncryptedError{}
-		}
+	if opts.OutputStore.HasSopsTopLevelKey(branch) {
+		return &fileAlreadyEncryptedError{}
 	}
 	return nil
 }
diff --git a/sops.go b/sops.go
index c144db005..672d32499 100644
--- a/sops.go
+++ b/sops.go
@@ -567,6 +567,12 @@ type ValueEmitter interface {
 	EmitValue(interface{}) ([]byte, error)
 }
 
+// CheckEncryped is the interface for testing whether a branch contains sops
+// metadata. This is used to check whether a file is already encrypted or not.
+type CheckEncryped interface {
+	HasSopsTopLevelKey(TreeBranch) bool
+}
+
 // Store is used to interact with files, both encrypted and unencrypted.
 type Store interface {
 	EncryptedFileLoader
@@ -574,6 +580,7 @@ type Store interface {
 	EncryptedFileEmitter
 	PlainFileEmitter
 	ValueEmitter
+	CheckEncryped
 }
 
 // MasterKeyCount returns the number of master keys available
diff --git a/stores/dotenv/store.go b/stores/dotenv/store.go
index 9433322ee..88eb0442f 100644
--- a/stores/dotenv/store.go
+++ b/stores/dotenv/store.go
@@ -175,3 +175,15 @@ func isComplexValue(v interface{}) bool {
 	}
 	return false
 }
+
+// HasSopsTopLevelKey checks whether a top-level "sops" key exists.
+func (store *Store) HasSopsTopLevelKey(branch sops.TreeBranch) bool {
+	for _, b := range branch {
+		if key, ok := b.Key.(string); ok {
+			if strings.HasPrefix(key, SopsPrefix) {
+				return true
+			}
+		}
+	}
+	return false
+}
diff --git a/stores/ini/store.go b/stores/ini/store.go
index dd1a77c63..c44b89302 100644
--- a/stores/ini/store.go
+++ b/stores/ini/store.go
@@ -274,3 +274,8 @@ func (store *Store) EmitExample() []byte {
 	}
 	return bytes
 }
+
+// HasSopsTopLevelKey checks whether a top-level "sops" key exists.
+func (store *Store) HasSopsTopLevelKey(branch sops.TreeBranch) bool {
+	return stores.HasSopsTopLevelKey(branch)
+}
diff --git a/stores/json/store.go b/stores/json/store.go
index cbecd8362..2f0f7a721 100644
--- a/stores/json/store.go
+++ b/stores/json/store.go
@@ -357,3 +357,13 @@ func (store *Store) EmitExample() []byte {
 	}
 	return bytes
 }
+
+// HasSopsTopLevelKey checks whether a top-level "sops" key exists.
+func (store *Store) HasSopsTopLevelKey(branch sops.TreeBranch) bool {
+	return stores.HasSopsTopLevelKey(branch)
+}
+
+// HasSopsTopLevelKey checks whether a top-level "sops" key exists.
+func (store *BinaryStore) HasSopsTopLevelKey(branch sops.TreeBranch) bool {
+	return stores.HasSopsTopLevelKey(branch)
+}
diff --git a/stores/stores.go b/stores/stores.go
index e4b17289d..f4f8f2354 100644
--- a/stores/stores.go
+++ b/stores/stores.go
@@ -506,3 +506,13 @@ var ExampleFlatTree = sops.Tree{
 		},
 	},
 }
+
+// HasSopsTopLevelKey returns true if the given branch has a top-level key called "sops".
+func HasSopsTopLevelKey(branch sops.TreeBranch) bool {
+	for _, b := range branch {
+		if b.Key == "sops" {
+			return true
+		}
+	}
+	return false
+}
diff --git a/stores/yaml/store.go b/stores/yaml/store.go
index 4d036f366..86ca00672 100644
--- a/stores/yaml/store.go
+++ b/stores/yaml/store.go
@@ -417,3 +417,8 @@ func (store *Store) EmitExample() []byte {
 	}
 	return bytes
 }
+
+// HasSopsTopLevelKey checks whether a top-level "sops" key exists.
+func (store *Store) HasSopsTopLevelKey(branch sops.TreeBranch) bool {
+	return stores.HasSopsTopLevelKey(branch)
+}


Test patch:
diff --git a/stores/dotenv/store_test.go b/stores/dotenv/store_test.go
index c234d4ea5..6886fbc34 100644
--- a/stores/dotenv/store_test.go
+++ b/stores/dotenv/store_test.go
@@ -80,3 +80,20 @@ func TestEmitEncryptedFileStability(t *testing.T) {
 		previous = bytes
 	}
 }
+
+func TestHasSopsTopLevelKey(t *testing.T) {
+	ok := (&Store{}).HasSopsTopLevelKey(sops.TreeBranch{
+		sops.TreeItem{
+			Key:   "sops",
+			Value: "value",
+		},
+	})
+	assert.Equal(t, ok, false)
+	ok = (&Store{}).HasSopsTopLevelKey(sops.TreeBranch{
+		sops.TreeItem{
+			Key:   "sops_",
+			Value: "value",
+		},
+	})
+	assert.Equal(t, ok, true)
+}
diff --git a/stores/yaml/store_test.go b/stores/yaml/store_test.go
index 41b3004d0..fb53d9ed3 100644
--- a/stores/yaml/store_test.go
+++ b/stores/yaml/store_test.go
@@ -380,4 +380,21 @@ func TestIndent1(t *testing.T) {
 	assert.Nil(t, err)
 	assert.Equal(t, string(INDENT_1_OUT), string(bytes))
 	assert.Equal(t, INDENT_1_OUT, bytes)
-}
\ No newline at end of file
+}
+
+func TestHasSopsTopLevelKey(t *testing.T) {
+	ok := (&Store{}).HasSopsTopLevelKey(sops.TreeBranch{
+		sops.TreeItem{
+			Key:   "sops",
+			Value: "value",
+		},
+	})
+	assert.Equal(t, ok, true)
+	ok = (&Store{}).HasSopsTopLevelKey(sops.TreeBranch{
+		sops.TreeItem{
+			Key:   "sops_",
+			Value: "value",
+		},
+	})
+	assert.Equal(t, ok, false)
+}