Change:
Our team uses sonobuoy to run the conformormance tests on a cluster and displays the results to the users. We have been using conformance image 1.21.x until now and it sends back progress data to sonobuoy pod and our service reads it from the sonobuoy pod. The progress data included the total number of tests to be run and the completed tests. For 1.21 conformance we were receiving this data properly but when switching to 1.25.x conformance we are getting the total tests as zero. This is causing issues in our UI. Not sure what version of conformance image introduced this but we were using 1.21 and started seeing this issue after moving to 1.25.x. The conformance image versions in between 1.21 -1.25 are not CVE free so we cannot use them. Total number of tests should be non-zero for conformance image 1.25.x. Ideally i think it should come back as 360. provision a 1.25 kubernetes kind (or anything else) cluster Install sonobuoy Run conformance scan on the cluster Check the annotation on the sonobuoy pod . This is set by sonobuoy based on the data it gets back from the e2e conformance pod. The total tests comes as zero I just found it on 1.25.x conformance. Not really sure when this was introduced. Need to make sure back dated versiosn also have this fix in case they have this issue. <details25 cluster with 1.25.x conformance with any sonobuoy version </details<detailsANY </details<details</details<details</details<details</details<detailsConformacne e2e plugin </details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
FYI
Yeah, it seems that it gets the info here I vaguely remember something about a progress custom reporter that was by sonobuoy and we didn't know what was used for
You remember correctly: It looks like that TODO didn't get addressed? I thought it had been. the other thing that was unclear regarding sonobuoy was whether it reads these test statistics from stderr or gets them via HTTP. We removed the stderr output (I think in 1.26) but kept (). This seems to be backed by (sonobuoy sets the parameter), but it would be good to get definite confirmation. Also, does Sonobuoy run tests in parallel? If it does, then these statistics are probably useless or at least will need considerable post-processing.
We depend on for this.
/assign I will take a look as has been fixed and closed
/reopen The fix went in, but let's keep the issue open for a while longer until we really know that it now works as expected. /assign
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
/remove-lifecycle rotten I discussed this further with (see also ) and while a final decision is pending, I am inclined to remove the progress reporting feature because I suspect that it is conceptually broken (to be confirmed) and lacks in-tree support (unit tests, owner). Before I do that, can you clarify: Are you running tests in parallel and does progress reporting work properly, i.e. does it go from 0 to 100%? When you said, "causing issues in our UI", what UI is that?
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
Ping Please see
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Reopen this issue with Mark this issue as fresh with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /close not-planned [1]:
Closing this issue, marking it as "Not Planned". <detailsIn response to : The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closedReopen this issue with Mark this issue as fresh with Offer to help out with [Issue Triage][1] Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
progressReporter.ProcessSpecReport(report) }) <ins> var _ = ginkgo.ReportBeforeSuite(func(report ginkgo.Report) { progressReporter.SetTestsTotal(report.PreRunStats.SpecsThatWillRun) }) </ins> var _ = ginkgo.ReportAfterSuite("Kubernetes e2e suite report", func(report ginkgo.Report) { var err error // The DetailsRepoerter will output details about every test (name, files, lines, etc) which helps

// ProgressReporter is a ginkgo reporter which tracks the total number of tests to be run/passed/failed/skipped. // As new tests are completed it updates the values and prints them to stdout and optionally, sends the updates // to the configured URL. <del> // TODO: Number of test specs is not available now, we can add it back when this is fixed in the Ginkgo V2. // pls see: https://github.com/kubernetes/kubernetes/issues/109744 </del> <ins> // One known limitation of the ProgressReporter it this reporter will not consolidate the reporter from each sub-process // if the tests are run in parallel. // As what's observed the reporter sent before test suite is started will be assembled correctly but each process will report // on its own after each test or suite are completed. // Here is a sample report that 5 testcases are executed totally, and run in parallel with 3 procs, // 3 of them are failed and other 2 passed. // {"msg":"","total":5,"completed":0,"skipped":0,"failed":0} // {"msg":"Test Suite starting","total":5,"completed":0,"skipped":0,"failed":0} // {"msg":"FAILED [sig-node] NoExecuteTaintManager Single Pod doesn't evict pod with tolerations from tainted nodes","total":0,"completed":0,"skipped":1332,"failed":1,"failures":["[sig-node] NoExecuteTaintManager..."]} // {"msg":"FAILED [sig-node] NoExecuteTaintManager Single Pod evicts pods from tainted nodes","total":5,"completed":0,"skipped":2524,"failed":1,"failures":["[sig-node] NoExecuteTaintManager Single Pod evicts pods from tainted nodes"]} // {"msg":"PASSED [sig-node] NoExecuteTaintManager Single Pod removing taint cancels eviction [Disruptive] [Conformance]","total":0,"completed":1,"skipped":1181,"failed":0} // {"msg":"Test Suite completed","total":0,"completed":1,"skipped":2592,"failed":0} // {"msg":"PASSED [sig-node] NoExecuteTaintManager Single Pod eventually evict pod with finite tolerations from tainted nodes","total":0,"completed":1,"skipped":1399,"failed":1,"failures":["[sig-node] NoExecuteTaintManager..."]} // {"msg":"Test Suite completed","total":0,"completed":1,"skipped":1399,"failed":1,"failures":["[sig-node] NoExecuteTaintManager Single Pod doesn't evict pod with tolerations from tainted nodes"]} // {"msg":"FAILED [sig-node] NoExecuteTaintManager Single Pod pods evicted from tainted nodes...","total":5,"completed":0,"skipped":3076,"failed":2,"failures":["[sig-node] NoExecuteTaintManager...","[sig-node] NoExecuteTaintManager..."]} // {"msg":"Test Suite completed","total":5,"completed":0,"skipped":3076,"failed":2,"failures":["[sig-node] NoExecuteTaintManager Single Pod evicts pods from tainted nodes","[sig-node] NoExecuteTaintManager..."]} </ins> type ProgressReporter struct { LastMsg string `json:"msg"` <ins> TestsTotal     int `json:"total"` </ins> TestsCompleted int `json:"completed"` TestsSkipped   int `json:"skipped"` TestsFailed    int `json:"failed"`

reporter.SendUpdates() } <ins> func (reporter *ProgressReporter) SetTestsTotal(totalSpec int) { reporter.TestsTotal = totalSpec reporter.SendUpdates() } </ins> // ProcessSpecReport summarizes the report state and sends the state to the configured endpoint if set. func (reporter *ProgressReporter) ProcessSpecReport(report ginkgo.SpecReport) { testName := strings.Join(report.ContainerHierarchyTexts, " ")