GoogleCloudPlatform/kubernetes — pull request #2490 by ghodss (2014-11-20 03:10:52+00:00)
Title: WIP (do not merge): Upgrade to YAML v2

This is needed because of a serious issue with YAML v1 described at http://blog.labix.org/2014/09/22/announcing-yaml-v2-for-go#typeerrors - basically, YAML v1 does not return any errors if you try to unmarshal YAML or JSON into an incompatible object. This is making [ObjectStream](https://github.com/GoogleCloudPlatform/kubernetes/issues/1905#issuecomment-61691766) development more difficult than it should be.

At this point I am working on porting to v2 but getting some very strange errors from the object conversion code. The big change is that SetYAML/GetYAML have changed to UnmarshalYAML/MarshalYAML, but there may be another change that's now causing this issue. @lavalamp (or others), I would really appreciate if you could check out this PR and try running:

```
go test -run TestEmbeddedObject -v github.com/GoogleCloudPlatform/kubernetes/pkg/runtime
```

And see the errors that come up. Here is the output, including the debug logger output as well as a bunch of my own haphazardly inserted print statements that are included in the PR.

```
$ go test -run TestEmbeddedObject -v github.com/GoogleCloudPlatform/kubernetes/pkg/runtime
=== RUN TestEmbeddedObject
Trying to convert '[runtime_test.EmbeddedTest runtime_test.EmbeddedTestExternal]' to '%!v(MISSING)'
Calling custom conversion of '[runtime.EmbeddedObject runtime.RawExtension]' to '%!v(MISSING)'
Trying to convert '[runtime_test.EmbeddedTest runtime_test.EmbeddedTestExternal]' to '%!v(MISSING)'
Calling custom conversion of '[runtime.EmbeddedObject runtime.RawExtension]' to '%!v(MISSING)'
Calling custom conversion of '[runtime.EmbeddedObject runtime.RawExtension]' to '%!v(MISSING)'
Trying to convert '[runtime.emptyPlugin runtime_test.EmbeddedTestExternal]' to '%!v(MISSING)'
Calling custom conversion of '[runtime.EmbeddedObject runtime.RawExtension]' to '%!v(MISSING)'
"apiVersion":"v1test","kind":"EmbeddedTest","name":"inner","creationTimestamp":null,"id":"inner","object":null,"emptyObject":null},"emptyObject":null}HELLO2: EmbeddedTestSTART-UNMARSHAL
Trying to convert '[runtime_test.EmbeddedTestExternal runtime_test.EmbeddedTest]' to '%!v(MISSING)'
Calling custom conversion of '[runtime.RawExtension runtime.EmbeddedObject]' to '%!v(MISSING)'
RAWEXTENSIONTEXT: apiVersion: v1test
creationTimestamp: null
emptyObject: null
id: inner
kind: EmbeddedTest
name: inner
object: null
HELLO1: apiVersion: v1test
creationTimestamp: null
emptyObject: null
id: inner
kind: EmbeddedTest
name: inner
object: null
HELLO2: EmbeddedTestHELLO1: apiVersion: v1test
creationTimestamp: null
emptyObject: null
id: inner
kind: EmbeddedTest
name: inner
object: null
HELLO2: EmbeddedTestTrying to convert '[runtime_test.EmbeddedTestExternal runtime_test.EmbeddedTest]' to '%!v(MISSING)'
Calling custom conversion of '[runtime.RawExtension runtime.EmbeddedObject]' to '%!v(MISSING)'
RAWEXTENSIONTEXT: HELLO1: HELLO2: /Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/scheme.go:152 (0x107ac8)
(*Scheme).NewObject: debug.PrintStack()
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:193 (0xf61f0)
(*Scheme).New: obj, err := s.raw.NewObject(versionName, typeName)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:112 (0xf5791)
(*Scheme).rawExtensionToEmbeddedObject: inObj, err := scheme.New(inVersion, kind)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:151 (0xf6bf0)
rawExtensionToEmbeddedObject).fm: s.rawExtensionToEmbeddedObject,
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/asm_amd64.s:362 (0x2c2d2)
call64: CALLFN(call64, 64)
/usr/local/Cellar/go/1.3/libexec/src/pkg/reflect/value.go:563 (0xe07c0)
Value.call: call(fn, args, uint32(frametype.size), uint32(retOffset))
/usr/local/Cellar/go/1.3/libexec/src/pkg/reflect/value.go:411 (0xdf467)
Value.Call: return v.call("Call", in)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/converter.go:253 (0x101d63)
(*Converter).convert: ret := fv.Call(args)[0].Interface()
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/converter.go:314 (0x103a7f)

/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/converter.go:241 (0x1017db)
(*Converter).Convert: return c.convert(sv, dv, s)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/scheme.go:213 (0x108095)
(*Scheme).Convert: return s.converter.Convert(in, out, AllowDifferentFieldTypeNames, s.generateConvertMeta(inVersion, outVersion))
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:225 (0xf63a9)
(*Scheme).Convert: return s.raw.Convert(in, out)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:127 (0xf5980)
(*Scheme).rawExtensionToEmbeddedObject: err = scheme.Convert(inObj, outObj)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:151 (0xf6bf0)
rawExtensionToEmbeddedObject).fm: s.rawExtensionToEmbeddedObject,
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/asm_amd64.s:362 (0x2c2d2)
call64: CALLFN(call64, 64)
/usr/local/Cellar/go/1.3/libexec/src/pkg/reflect/value.go:563 (0xe07c0)
Value.call: call(fn, args, uint32(frametype.size), uint32(retOffset))
/usr/local/Cellar/go/1.3/libexec/src/pkg/reflect/value.go:411 (0xdf467)
Value.Call: return v.call("Call", in)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/converter.go:253 (0x101d63)
(*Converter).convert: ret := fv.Call(args)[0].Interface()
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/converter.go:314 (0x103a7f)

/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/converter.go:241 (0x1017db)
(*Converter).Convert: return c.convert(sv, dv, s)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/conversion/decode.go:66 (0x1048ad)
(*Scheme).Decode: err = s.converter.Convert(obj, objOut, 0, s.generateConvertMeta(version, s.InternalVersion))
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/scheme.go:282 (0xf67a0)
(*Scheme).Decode: obj, err := s.raw.Decode(data)
/Users/sam/code/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/runtime/embedded_test.go:70 (0x46f23)
TestEmbeddedObject: decoded, err := s.Decode(wire)
/usr/local/Cellar/go/1.3/libexec/src/pkg/testing/testing.go:422 (0x4186b)
tRunner: test.F(t)
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1445 (0x14b70)
goexit: runtime·goexit(void)
--- FAIL: TestEmbeddedObject (0.00 seconds)
embedded_test.go:68: Wire format is:
"apiVersion":"v1test","kind":"EmbeddedTest","name":"inner","creationTimestamp":null,"id":"inner","object":null,"emptyObject":null},"emptyObject":null}
embedded_test.go:72: Unexpected decode error No type '' for version 'v1test'
FAIL
exit status 1
FAIL github.com/GoogleCloudPlatform/kubernetes/pkg/runtime 0.025s
"apiVersion":"v1test","kind":"EmbeddedTest","name":"outer","creationTimestamp":null,"id":"outer","object":{}
{}
(*Converter).convert: if err := c.convert(sf, df, scope); err != nil {}
(*Converter).convert: if err := c.convert(sf, df, scope); err != nil {}
"apiVersion":"v1test","kind":"EmbeddedTest","name":"outer","creationTimestamp":null,"id":"outer","object":{}
HELLO1: {}
```

There's something calling rawExtensionToEmbeddedObject but I can't figure out what it is and why the RawExtension object that is passed in is not valid.

@nictuku I think you tried this upgrade before in #1481, but ended up aborting? Was there a specific reason for that?

Anyways sorry this is a bit scatterbrained but I've been staring at this code for a few hours and still not sure of what's going on - any help would be appreciated. Thanks!