Change:
This bug was privately disclosed and told to open here. What happened: NewYAMLOrJSONDecoder from has an index out of bounds issue which can cause callers to panic. Download attached reproducer file Run golang test: When run with v0.19.3: Additionally, this can be seen in kubectl directly (below is kubectl 1.20.0): Environment: Kubernetes version (use ): Verified with 1.19, 1.20
/sig api-machinery
This piece attempts to find line+column position of a json syntax error. To do that it assumes that full input is available which is not true: is : is only the first data chunk: The subject happens e.g. when syntax error offset is past buffer size and Test to reproduce the panic: I am not sure it is easily possible to find line+column of json syntax error as full input may not be available. Maybe it should only attempt to find line+column if offset is within the first chunk (i.e. below buffer size).
Line number calculation in and updated in
do you want to submit a patch to with a defensive check to avoid the panic ( compare and )
I have raised a discussion on having line number in the . The consensus there is that it should be possible to count lines with a full input at hand and it does not worth to have it in the core library due to performance impact and maintenance costs. The problem of here is that it attempts to find out line number without having full input at hand. I think the defensive patch is not the right approach because line calculation is incorrect anyway even when there is no panic: begins at a value with a syntax error, i.e. in a stream of two values and syntax error in the second value it would contain only the second value and thus new lines in the first value in the stream would not be counted. I think the following options are there: to stream taken from . This is straightforward but changes interface. I can not find any code that uses field, it is only used in the error message that would change: line number only if offset is within the length of the first data chunk (). I am not sure what would be the unknown line number then, perhaps. line number properly during stream decoding, see This has some performance penalty as line number is calculated in any case even if there is no syntax error, i.e. in the majority of the cases.
cc /triage accepted
This is probably acceptable.
I have opened a PR
Thank you for support.

Discussion:
importpath = "k8s.io/apimachinery/pkg/util/yaml", deps = [ "//staging/src/k8s.io/apimachinery/pkg/util/json:go_default_library", <del> "//vendor/k8s.io/klog/v2:go_default_library", </del> "//vendor/sigs.k8s.io/yaml:go_default_library", ], )

"encoding/json" "fmt" "io" <del> "io/ioutil" </del> "strings" "unicode" jsonutil "k8s.io/apimachinery/pkg/util/json" <del> "k8s.io/klog/v2" </del> "sigs.k8s.io/yaml" )

bufferSize int decoder decoder <del> rawData []byte </del> } type JSONSyntaxError struct { <del> Line int Err  error </del> <ins> Offset int64 Err    error </ins> } func (e JSONSyntaxError) Error() string { <del> return fmt.Sprintf("json: line %d: %s", e.Line, e.Err.Error()) </del> <ins> return fmt.Sprintf("json: offset %d: %s", e.Offset, e.Err.Error()) </ins> } type YAMLSyntaxError struct {

// provide object, or returns an error. func (d *YAMLOrJSONDecoder) Decode(into interface{}) error { if d.decoder == nil { <del> buffer, origData, isJSON := GuessJSONStream(d.r, d.bufferSize) </del> <ins> buffer, _, isJSON := GuessJSONStream(d.r, d.bufferSize) </ins> if isJSON { d.decoder = json.NewDecoder(buffer) <del> d.rawData = origData </del> } else { d.decoder = NewYAMLToJSONDecoder(buffer) } } err := d.decoder.Decode(into) <del> if jsonDecoder, ok := d.decoder.(*json.Decoder); ok { if syntax, ok := err.(*json.SyntaxError); ok { data, readErr := ioutil.ReadAll(jsonDecoder.Buffered()) if readErr != nil { klog.V(4).Infof("reading stream failed: %v", readErr) } js := string(data) // if contents from io.Reader are not complete, // use the original raw data to prevent panic if int64(len(js)) <= syntax.Offset { js = string(d.rawData) } start := strings.LastIndex(js[:syntax.Offset], "n") + 1 line := strings.Count(js[:start], "n") return JSONSyntaxError{ Line: line, Err:  fmt.Errorf(syntax.Error()), } </del> <ins> if syntax, ok := err.(*json.SyntaxError); ok { return JSONSyntaxError{ Offset: syntax.Offset, Err:    syntax, </ins> } } return err

if err == nil { t.Fatal("expected error with json: prefix, got no error") } <del> if !strings.HasPrefix(err.Error(), "json: line 3:") { t.Fatalf("expected %q to have 'json: line 3:' prefix", err.Error()) </del> <ins> const msg = `json: offset 28: invalid character '"' after object key:value pair` if msg != err.Error() { t.Fatalf("expected %q, got %q", msg, err.Error()) </ins> } }