Repository: spf13/viper (Go)

Issue:
Error ``Unmarshal`` map when key contains dot
Test code:

```
package main

import (
	"bytes"
	"fmt"
	"github.com/spf13/viper"
)

type Config struct {
	Foo struct {
		Bar map[string]int `mapstructure:"bar"`
	} `mapstructure:"foo"`
}

var content = []byte(`
foo:
  bar:
    "x": 1
    "y": 2
    "z.z": 3
`)

func main() {
	var c1, c2 Config

	viper.SetConfigType("yaml")
	viper.ReadConfig(bytes.NewReader(content))

	viper.Unmarshal(&c1)
	fmt.Println("c1:", c1)

	viper.UnmarshalKey("foo", &c2.Foo)
	fmt.Println("c2:", c2)
}
```

Output:
```
c1: {{map[x:1 y:2]}}
c2: {{map[x:1 y:2 z.z:3]}}
```

As you can see, when the map key contains dot, ``UnmarshalKey`` works correctly but ``Unmarshal`` doesn't.


Discussion:
This still seems to be an issue for me. This test fails depending on v1.3.1:
```go
package main

import (
	"strings"
	"testing"

	"github.com/spf13/viper"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func TestConfigMarshalling(t *testing.T) {
	data := `
root:
  ele.one: value
  eletwo: value`
	reader := strings.NewReader(data)
	viper.SetConfigType("yaml")
	err := viper.ReadConfig(reader)
	require.NoError(t, err)

	// Test if the fields are even correct
	expected := map[string]interface{}{"ele.one": "value", "eletwo": "value"}
	assert.Equal(t, expected, viper.Get("root"))
	var s struct {
		Root map[string]string
	}
	err = viper.UnmarshalKey("root", &s)
	require.NoError(t, err)
}
```
Changing the dot to, say, an underscore fixes the issue.
Also hit this. Seems like if there's a key like "word.other" containing a dot as a map[string]interface{} key, the output turns it into a struct like: 

```
{
    "word": {
        "other": "value"      
    }
}
```

and not
```
{
    "word.other": "value"
}
```

Is that expected? Anyone have pointers to override this to get the former behavior?
Looks like `AllKeys()` returns a list of all the keys it can find as delimited strings:
```golang
// main.go
func main() {
	configString := `
test:
  a: 1 
  b: 2
  c.d: 3
`
	viper.SetConfigType("yaml")
	err := viper.ReadConfig(bytes.NewBufferString(configString))
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
	fmt.Println(viper.AllKeys())
}
```
```
[test.c.d test.a test.b]
```

As noted, the problem comes from keys with dots in as the unmarshalling uses the keys returned by `AllKeys()` and the default key delimiter is a dot, causing it to incorrectly interpret a dot in a key name for a sub key.

I added the following functions to `viper.go` in order to change the delimiter. Alternatively you could just change the default delimiter in `New()`:
```golang
// viper.go
func SetKeyDelim(delim string) { v.keyDelim = delim }
func (v *Viper) SetKeyDelim(delim string) { v.keyDelim = delim }
```
```golang
// main.go
func main() {
	configString := `
test:
  a: 1 
  b: 2
  c.d: 3
`
	viper.SetKeyDelim("\\")
	viper.SetConfigType("yaml")
	err := viper.ReadConfig(bytes.NewBufferString(configString))
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
	fmt.Println(viper.AllKeys())
}
```
```
[test\c.d test\a test\b]
```
~The simplest solution would of course be to change the default but perhaps for the sake of backward compatibility, it would be best to just add the option to change the delimiter.~
Given that the dot is used to allow you to traverse through keys, it would be pragmatic to just add the option to change the delimiter on a case by case basis.

hopefully it will get merged soon!
I will also say that despite it's simplicity, I imagine adding to the API is not going to be as favourable as fixing the implementation of `Unmarshal` in a more thorough way. The fact that keys with dots in can be handled successfully via the `viper.GetStringMap_` methods suggests that there is probably a better way to fix this such that it's only a patch rather than minor change, semantically speaking (even though it would probably be a bigger change). I will continue to look into this but I'm new to both Go and the project so can't promise anything!
I wonder, if we use dots in a key and AutomaticEnv, how will the env variable names look like?
I'll have to look into exactly what `AutomaticEnv` is and what it does, I'm still fairly new to the project but I'm guessing the issue is going to be with environment variables containing dots which is syntactically incorrect ?
Yes. Also #678
@sagikazarmark is it worth opening this again?
I guess it does, since #673 was reverted.
Hey guys, I'm hitting this problem too. Need to get a map with the key being IPs, but it fails.

I've been using this library for quite a long time now for pretty much all my personal and work projects.

Any chance we can get some brainstorming on this?
You could stick your viper module to commit 99520c8 this commit was reverted but you don't need to run on latest :)
`go get github.com/spf13/viper@99520c8`

@mschneider82 thank you for the tip! 
The alternative is to change the key delimiter which will be added in #794, this is on a branch of this repo and I assume will be merged at some point so could also lock the go module to that reference.

As for a more robust solution I've been thinking about a way forward. As discussed in #766, the problem is being unable to determine from a single string, whether a given character should be part of the key name or is treated as the delimiter, eg: `sites.example.com.port` can be interpreted many different ways. The reason it can work for unmarshalling, which is what I tried to fix, is that you're telling viper the config schema in the form of a struct.

I believe the only way to also solve the problem for environment variable and pflag bindings is to provide viper with some sort of schema (like we do for unmarshalling), so that it knows explicitly what to do with a parameter address that might contain delimiters within key names, rather than assuming they're all separate keys. I'm not yet sure if this is something that could be achieved within v1, perhaps providing a schema would be like the feature toggle @sagikazarmark mentions in https://github.com/spf13/viper/pull/794#issue-337317980.
Right, I fully intend to merge #794 soon, but I'm playing with replacing the setter with a constructor/factory function argument. I don't really like setters, and in this case it could actually mess up the internal state of Viper, so that's what blocks the PR from getting merged.
Any workarounds in the meantime?
@WoLfulus With #794 now merged into master, the current workaround is to set your delimiter to something other than a dot.
I am having issues while creating TOML file. 
I set keyDelimiter as "!" but still when key contains "dot".

It add additional double quotes as below:
["x,y"]
instead of [x.y]
I have a very similar problem but with keys with `_` instead. Any key with `_` does not get parsed on `UnmarshalKey`. 
Issues with no activity for 30 days are marked stale and subject to being closed.
This issue has been closed for inactivity. Please re-open if this was a mistake.



Patch:
diff --git a/viper.go b/viper.go
index 5133de792..491799c7c 100644
--- a/viper.go
+++ b/viper.go
@@ -1789,6 +1789,24 @@ outer:
 	return shadow
 }
 
+// Converts a fully qualified map key into a list of relative
+// map keys, allowing for keys to contain the delimiter themselves
+func keyComponents(v *Viper, key string) []string {
+	var result []string
+	components := strings.Split(key, v.keyDelim)
+	for index := 0; index < len(components); index++ {
+		potentialKey := strings.Join(components[0:index], v.keyDelim)
+		if v.Get(potentialKey) != nil {
+			result = append(result, potentialKey)
+		}
+	}
+	result = append(result, key)
+	for i := len(result) - 1; i > 0; i-- {
+		result[i] = strings.Replace(result[i], result[i-1]+v.keyDelim, "", 1)
+	}
+	return result
+}
+
 // AllSettings merges all settings and returns them as a map[string]interface{}.
 func AllSettings() map[string]interface{} { return v.AllSettings() }
 func (v *Viper) AllSettings() map[string]interface{} {
@@ -1801,7 +1819,7 @@ func (v *Viper) AllSettings() map[string]interface{} {
 			// check just in case anything changes
 			continue
 		}
-		path := strings.Split(k, v.keyDelim)
+		path := keyComponents(v, k)
 		lastKey := strings.ToLower(path[len(path)-1])
 		deepestMap := deepSearch(m, path[0:len(path)-1])
 		// set innermost value


Test patch:
diff --git a/viper_test.go b/viper_test.go
index 592551c92..3c15ed26f 100644
--- a/viper_test.go
+++ b/viper_test.go
@@ -13,6 +13,7 @@ import (
 	"os"
 	"os/exec"
 	"path"
+	"path/filepath"
 	"reflect"
 	"runtime"
 	"sort"
@@ -45,6 +46,10 @@ clothing:
 age: 35
 eyes : brown
 beard: true
+emails:
+  steve@hacker.com:
+    created: 01/02/03
+    active: true
 `)
 
 var yamlExampleWithExtras = []byte(`Existing: true
@@ -207,11 +212,16 @@ func initHcl() {
 func initDirs(t *testing.T) (string, string, func()) {
 
 	var (
-		testDirs = []string{`a a`, `b`, `c\c`, `D_`}
+		testDirs = []string{`a a`, `b`, `C_`}
 		config   = `improbable`
 	)
 
+	if runtime.GOOS != "windows" {
+		testDirs = append(testDirs, `d\d`)
+	}
+
 	root, err := ioutil.TempDir("", "")
+	require.NoError(t, err, "Failed to create temporary directory")
 
 	cleanup := true
 	defer func() {
@@ -224,7 +234,7 @@ func initDirs(t *testing.T) (string, string, func()) {
 	assert.Nil(t, err)
 
 	err = os.Chdir(root)
-	assert.Nil(t, err)
+	require.Nil(t, err)
 
 	for _, dir := range testDirs {
 		err = os.Mkdir(dir, 0750)
@@ -415,7 +425,10 @@ func TestEmptyEnv(t *testing.T) {
 	BindEnv("type") // Empty environment variable
 	BindEnv("name") // Bound, but not set environment variable
 
-	os.Clearenv()
+	os.Unsetenv("type")
+	os.Unsetenv("TYPE")
+	os.Unsetenv("name")
+	os.Unsetenv("NAME")
 
 	os.Setenv("TYPE", "")
 
@@ -431,7 +444,10 @@ func TestEmptyEnv_Allowed(t *testing.T) {
 	BindEnv("type") // Empty environment variable
 	BindEnv("name") // Bound, but not set environment variable
 
-	os.Clearenv()
+	os.Unsetenv("type")
+	os.Unsetenv("TYPE")
+	os.Unsetenv("name")
+	os.Unsetenv("NAME")
 
 	os.Setenv("TYPE", "")
 
@@ -491,11 +507,98 @@ func TestSetEnvKeyReplacer(t *testing.T) {
 func TestAllKeys(t *testing.T) {
 	initConfigs()
 
-	ks := sort.StringSlice{"title", "newkey", "owner.organization", "owner.dob", "owner.bio", "name", "beard", "ppu", "batters.batter", "hobbies", "clothing.jacket", "clothing.trousers", "clothing.pants.size", "age", "hacker", "id", "type", "eyes", "p_id", "p_ppu", "p_batters.batter.type", "p_type", "p_name", "foos",
-		"title_dotenv", "type_dotenv", "name_dotenv",
+	ks := sort.StringSlice{
+		"title",
+		"newkey",
+		"owner.organization",
+		"owner.dob",
+		"owner.bio",
+		"name",
+		"beard",
+		"ppu",
+		"batters.batter",
+		"hobbies",
+		"clothing.jacket",
+		"clothing.trousers",
+		"clothing.pants.size",
+		"age",
+		"hacker",
+		"id",
+		"type",
+		"eyes",
+		"p_id",
+		"p_ppu",
+		"p_batters.batter.type",
+		"p_type",
+		"p_name",
+		"foos",
+		"title_dotenv",
+		"type_dotenv",
+		"name_dotenv",
+		"emails.steve@hacker.com.active",
+		"emails.steve@hacker.com.created",
 	}
 	dob, _ := time.Parse(time.RFC3339, "1979-05-27T07:32:00Z")
-	all := map[string]interface{}{"owner": map[string]interface{}{"organization": "MongoDB", "bio": "MongoDB Chief Developer Advocate & Hacker at Large", "dob": dob}, "title": "TOML Example", "ppu": 0.55, "eyes": "brown", "clothing": map[string]interface{}{"trousers": "denim", "jacket": "leather", "pants": map[string]interface{}{"size": "large"}}, "id": "0001", "batters": map[string]interface{}{"batter": []interface{}{map[string]interface{}{"type": "Regular"}, map[string]interface{}{"type": "Chocolate"}, map[string]interface{}{"type": "Blueberry"}, map[string]interface{}{"type": "Devil's Food"}}}, "hacker": true, "beard": true, "hobbies": []interface{}{"skateboarding", "snowboarding", "go"}, "age": 35, "type": "donut", "newkey": "remote", "name": "Cake", "p_id": "0001", "p_ppu": "0.55", "p_name": "Cake", "p_batters": map[string]interface{}{"batter": map[string]interface{}{"type": "Regular"}}, "p_type": "donut", "foos": []map[string]interface{}{map[string]interface{}{"foo": []map[string]interface{}{map[string]interface{}{"key": 1}, map[string]interface{}{"key": 2}, map[string]interface{}{"key": 3}, map[string]interface{}{"key": 4}}}}, "title_dotenv": "DotEnv Example", "type_dotenv": "donut", "name_dotenv": "Cake"}
+	all := map[string]interface{}{
+		"owner": map[string]interface{}{
+			"organization": "MongoDB",
+			"bio":          "MongoDB Chief Developer Advocate & Hacker at Large",
+			"dob":          dob,
+		},
+		"title": "TOML Example",
+		"ppu":   0.55,
+		"emails": map[string]interface{}{
+			"steve@hacker.com": map[string]interface{}{
+				"active":  true,
+				"created": "01/02/03",
+			},
+		},
+		"eyes": "brown",
+		"clothing": map[string]interface{}{
+			"trousers": "denim",
+			"jacket":   "leather",
+			"pants":    map[string]interface{}{"size": "large"},
+		},
+		"id": "0001",
+		"batters": map[string]interface{}{
+			"batter": []interface{}{
+				map[string]interface{}{"type": "Regular"},
+				map[string]interface{}{"type": "Chocolate"},
+				map[string]interface{}{"type": "Blueberry"},
+				map[string]interface{}{"type": "Devil's Food"},
+			},
+		},
+		"hacker": true,
+		"beard":  true,
+		"hobbies": []interface{}{
+			"skateboarding",
+			"snowboarding",
+			"go",
+		},
+		"age":    35,
+		"type":   "donut",
+		"newkey": "remote",
+		"name":   "Cake",
+		"p_id":   "0001",
+		"p_ppu":  "0.55",
+		"p_name": "Cake",
+		"p_batters": map[string]interface{}{
+			"batter": map[string]interface{}{"type": "Regular"},
+		},
+		"p_type": "donut",
+		"foos": []map[string]interface{}{
+			{
+				"foo": []map[string]interface{}{
+					{"key": 1},
+					{"key": 2},
+					{"key": 3},
+					{"key": 4}},
+			},
+		},
+		"title_dotenv": "DotEnv Example",
+		"type_dotenv":  "donut",
+		"name_dotenv":  "Cake",
+	}
 
 	allkeys := sort.StringSlice(AllKeys())
 	allkeys.Sort()
@@ -960,7 +1063,7 @@ func TestDirsSearch(t *testing.T) {
 	err = v.ReadInConfig()
 	assert.Nil(t, err)
 
-	assert.Equal(t, `value is `+path.Base(v.configPaths[0]), v.GetString(`key`))
+	assert.Equal(t, `value is `+filepath.Base(v.configPaths[0]), v.GetString(`key`))
 }
 
 func TestWrongDirsSearchNotFound(t *testing.T) {
@@ -1213,6 +1316,10 @@ clothing:
   pants:
     size: large
   trousers: denim
+emails:
+  steve@hacker.com:
+    active: true
+    created: 01/02/03
 eyes: brown
 hacker: true
 hobbies: