Repository: editorconfig-checker/editorconfig-checker
Pull request #367 (2024-09-05 07:58:42)

feat: provide Codeclimate compatible report fromat
Provide code quality report as JSON according to Code Climate specificiation. This is need when using the editorconfig-checker in GitLab CI.

More details: #365

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker) Report
Attention: Patch coverage is `17.24138%` with `24 lines` in your changes missing coverage. Please review.
> Project coverage is 83.80%. Comparing base [(`d6c17be`)](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/commit/d6c17be4859bf44d4a2f3f45cc4cb9f162c99dea?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker) to head [(`1c5011d`)](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/commit/1c5011d2c61f2b9e0ef043715fa81b9020951c2b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker).
> Report is 39 commits behind head on main.

| [Files with missing lines](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker) | Patch % | Lines |
|---|---|---|
| [pkg/error/format\_codeclimate.go](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?src=pr&el=tree&filepath=pkg%2Ferror%2Fformat_codeclimate.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker#diff-cGtnL2Vycm9yL2Zvcm1hdF9jb2RlY2xpbWF0ZS5nbw==) | 0.00% | [15 Missing :warning: ](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker) |
| [pkg/error/error.go](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?src=pr&el=tree&filepath=pkg%2Ferror%2Ferror.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker#diff-cGtnL2Vycm9yL2Vycm9yLmdv) | 35.71% | [8 Missing and 1 partial :warning: ](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #367      +/-   ##
==========================================
- Coverage   84.89%   83.80%   -1.09%     
==========================================
  Files           8        9       +1     
  Lines         695      636      -59     
==========================================
- Hits          590      533      -57     
+ Misses         81       78       -3     
- Partials       24       25       +1     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/editorconfig-checker/editorconfig-checker/pull/367?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=editorconfig-checker).

tploss: > Looks generally good and like it adheres to the specification you left in the comment. Thanks for turning that ugly `if...elseif..else` into a `switch` - I totally forgot to think about if that feature is available in golang when writing it.. ;)
> 
> However I'd like to see tests for demonstrating it in action and proving correctness.

👍 
Will definitely create some tests. Just have to first understand this *snapshot* system for test data.

> Furthermore I think you should disable the verbose and debug logging when `config.Format == codeclimate` - else anyone trying to debug will surround the JSON output with unparsable noise.

An easier solution from my perspective would be to just log non-standard messages (so debug, error, etc) on stderr instead of stdout. Then one can easily just use `editorconfig-checker --format codeclimate > report.json` but still see all error and debug messages.
Thoughts?

> Also I thought about, if one should maybe turn the codeclimate generation into a method of `ValidationErrors`, which only runs `ConsolidateErrors` and returns the JSON.
> 
> In that regard, I was wondering lately, if it would be better to split `FormatErrors` into per-output-format functions, and change their interface towards `main()` to utilize an io.Writer. But that could easily be another PR once this one is merged.

I was also not really happy with how a single function would grow per added format but did not want to refactor that completely in my PR that just wants to add a single format.
Can we do that as a separate PR afterwards?


klaernie: > Will definitely create some tests. Just have to first understand this _snapshot_ system for test data.

It's actually fairly easy to use: you just demonstrate calling your code in error_test.go and hand the result of calling `FormatErrors` to `s.MatchSnapshot`. Then run `UPDATE_SNAPS=true go test ./...` so it generates the new snapshot.
tploss: > It's actually fairly easy to use: you just demonstrate calling your code in error_test.go and hand the result of calling `FormatErrors` to `s.MatchSnapshot`. Then run `UPDATE_SNAPS=true go test ./...` so it generates the new snapshot.

Done 🙂 
I also verified the test by changing the severity constant to "major" and this subsequently failed the test case.
tploss: The ci is waiting for an approval before running??
I'm not really used to GitHub CI so I don't know if this is normal. @klaernie @mstruebing 
klaernie: Indeed, the CI is waiting to be approved by a maintainer of the repo (so sadly I cannot trigger it).
klaernie: Oh, just noticed you overlooked the same places in the README and cmd/main.go argument help as me, which show the available output formats.
mstruebing: CI already ran so someone needs to have pressed the button.
It's a mechanism to make sure someone who does a PR doesn't break anything or leak any secrets etc.
You could do all kind of bad stuff with Github Actions, this makes sure that at least someone needs to approve the run.
tploss: > Oh, just noticed you overlooked the same places in the README and cmd/main.go argument help as me, which show the available output formats.

Thanks, I have fixed it and also provided a nicely formatted example output 🙂 

Patch:
diff --git a/README.md b/README.md
index b4b687b..6ca2aaf 100644
--- a/README.md
+++ b/README.md
@@ -147,6 +147,25 @@ The following output formats are supported:
       `<file>:<line>:<column>: <type>: <message>`
 - **github-actions**: The format used by GitHub Actions <br/>
       `::error file=<file>,line=<startingLine>,endLine=<endingLine>::<message>`
+- **codeclimate**: The [Code Climate](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types) json format used for [custom quality reports](https://docs.gitlab.com/ee/ci/testing/code_quality.html#implement-a-custom-tool) in GitLab CI
+  ```
+  [
+        {
+              "check_name": "editorconfig-checker",
+              "description": "Wrong indent style found (tabs instead of spaces)",
+              "fingerprint": "e87a958a3960d60a11d4b49c563cccd2",
+              "severity": "minor",
+              "location": {
+                    "path": ".vscode/extensions.json",
+                    "lines": {
+                    "begin": 2,
+                    "end": 2
+                    }
+              }
+        },
+        ...
+  ]
+  ```
 
 ## Configuration
 
diff --git a/cmd/editorconfig-checker/main.go b/cmd/editorconfig-checker/main.go
index 2c01da9..8970f16 100644
--- a/cmd/editorconfig-checker/main.go
+++ b/cmd/editorconfig-checker/main.go
@@ -39,8 +39,8 @@ func init() {
 	flag.BoolVar(&c.ShowVersion, "version", false, "print the version number")
 	flag.BoolVar(&c.Help, "help", false, "print the help")
 	flag.BoolVar(&c.Help, "h", false, "print the help")
-	flag.StringVar(&c.Format, "format", "default", "specify the output format: default, gcc, github-actions")
-	flag.StringVar(&c.Format, "f", "default", "specify the output format: default, gcc, github-actions")
+	flag.StringVar(&c.Format, "format", "default", "specify the output format: default, gcc, github-actions, codeclimate")
+	flag.StringVar(&c.Format, "f", "default", "specify the output format: default, gcc, github-actions, codeclimate")
 	flag.BoolVar(&c.Verbose, "verbose", false, "print debugging information")
 	flag.BoolVar(&c.Verbose, "v", false, "print debugging information")
 	flag.BoolVar(&c.Debug, "debug", false, "print debugging information")
@@ -127,7 +127,9 @@ func main() {
 	errorCount := len(formattedErrors)
 	if errorCount != 0 {
 		config.Logger.PrintLogMessages(formattedErrors)
-		config.Logger.Error(fmt.Sprintf("\n%d errors found", errorCount))
+		if config.Format != "codeclimate" {
+			config.Logger.Error(fmt.Sprintf("\n%d errors found", errorCount))
+		}
 	}
 
 	config.Logger.Verbose("%d files checked", len(filePaths))
diff --git a/pkg/error/error.go b/pkg/error/error.go
index a680863..9bd708c 100644
--- a/pkg/error/error.go
+++ b/pkg/error/error.go
@@ -2,6 +2,7 @@
 package error
 
 import (
+	"encoding/json"
 	"fmt"
 
 	"github.com/editorconfig-checker/editorconfig-checker/v3/pkg/config"
@@ -88,6 +89,8 @@ func ConsolidateErrors(errors []ValidationError, config config.Config) []Validat
 // FormatErrors prints the errors to the console
 func FormatErrors(errors []ValidationErrors, config config.Config) []logger.LogMessage {
 	var formatted_errors []logger.LogMessage
+	var codeclimateIssues []CodeclimateIssue
+
 	for _, fileErrors := range errors {
 		if len(fileErrors.Errors) > 0 {
 			relativeFilePath, err := files.GetRelativePath(fileErrors.FilePath)
@@ -102,7 +105,14 @@ func FormatErrors(errors []ValidationErrors, config config.Config) []logger.LogM
 				fileErrors.Errors = ConsolidateErrors(fileErrors.Errors, config)
 			}
 
-			if config.Format == "gcc" {
+			switch config.Format {
+			case "codeclimate":
+				// codeclimate: A format that is compatible with the codeclimate format for GitLab CI.
+				// https://docs.gitlab.com/ee/ci/testing/code_quality.html#implement-a-custom-tool
+				for _, singleError := range fileErrors.Errors {
+					codeclimateIssues = append(codeclimateIssues, newCodeclimateIssue(singleError, relativeFilePath))
+				}
+			case "gcc":
 				// gcc: A format mimicking the error format from GCC.
 				for _, singleError := range fileErrors.Errors {
 					var lineNo = 0
@@ -111,7 +121,7 @@ func FormatErrors(errors []ValidationErrors, config config.Config) []logger.LogM
 					}
 					formatted_errors = append(formatted_errors, logger.LogMessage{Level: "error", Message: fmt.Sprintf("%s:%d:%d: %s: %s", relativeFilePath, lineNo, 0, "error", singleError.Message)})
 				}
-			} else if config.Format == "github-actions" {
+			case "github-actions":
 				// github-actions: A format dedicated for usage in Github Actions
 				for _, singleError := range fileErrors.Errors {
 					if singleError.LineNumber != -1 {
@@ -124,7 +134,7 @@ func FormatErrors(errors []ValidationErrors, config config.Config) []logger.LogM
 						formatted_errors = append(formatted_errors, logger.LogMessage{Level: "error", Message: fmt.Sprintf("::error file=%s::%s", relativeFilePath, singleError.Message)})
 					}
 				}
-			} else {
+			default:
 				// default: A human readable text format.
 				formatted_errors = append(formatted_errors, logger.LogMessage{Level: "warning", Message: fmt.Sprintf("%s:", relativeFilePath)})
 				for _, singleError := range fileErrors.Errors {
@@ -141,5 +151,15 @@ func FormatErrors(errors []ValidationErrors, config config.Config) []logger.LogM
 			}
 		}
 	}
+
+	if len(codeclimateIssues) > 0 {
+		// marshall codeclimate issues to json
+		codeclimateIssuesJSON, err := json.Marshal(codeclimateIssues)
+		if err != nil {
+			config.Logger.Error("Error creating codeclimate json: %s", err.Error())
+		} else {
+			return []logger.LogMessage{{Level: "", Message: string(codeclimateIssuesJSON)}}
+		}
+	}
 	return formatted_errors
 }
diff --git a/pkg/error/format_codeclimate.go b/pkg/error/format_codeclimate.go
new file mode 100644
index 0000000..09ff4cd
--- /dev/null
+++ b/pkg/error/format_codeclimate.go
@@ -0,0 +1,50 @@
+package error
+
+import (
+	"crypto/md5"
+	"fmt"
+)
+
+// CodeclimageLines represents the lines of an issue in codeclimate format
+type CodeclimateLines struct {
+	Begin int `json:"begin"`
+	End   int `json:"end"`
+}
+
+// CodeclimateLocation represents the location of an issue in codeclimate format
+type CodeclimateLocation struct {
+	Path  string           `json:"path"`
+	Lines CodeclimateLines `json:"lines"`
+}
+
+// CodeclimateIssue represents an issue in codeclimate format
+type CodeclimateIssue struct {
+	Check       string              `json:"check_name"`
+	Description string              `json:"description"`
+	Fingerprint string              `json:"fingerprint"`
+	Severity    string              `json:"severity"`
+	Location    CodeclimateLocation `json:"location"`
+}
+
+const (
+	checkName = "editorconfig-checker"
+	severity  = "minor"
+)
+
+func newCodeclimateIssue(err ValidationError, path string) CodeclimateIssue {
+	toHash := fmt.Sprintf("%s:%d:%d:%s", path, err.LineNumber, err.AdditionalIdenticalErrorCount, err.Message.Error())
+	fingerprint := fmt.Sprintf("%x", md5.Sum([]byte(toHash)))
+	return CodeclimateIssue{
+		Check:       checkName,
+		Description: err.Message.Error(),
+		Fingerprint: fingerprint,
+		Severity:    severity,
+		Location: CodeclimateLocation{
+			Path: path,
+			Lines: CodeclimateLines{
+				Begin: err.LineNumber,
+				End:   err.AdditionalIdenticalErrorCount,
+			},
+		},
+	}
+}


Test patch:
diff --git a/pkg/error/__snapshots__/pathseparator-slash/error_test.snap b/pkg/error/__snapshots__/pathseparator-slash/error_test.snap
index 7df71a4..18c2dca 100755
--- a/pkg/error/__snapshots__/pathseparator-slash/error_test.snap
+++ b/pkg/error/__snapshots__/pathseparator-slash/error_test.snap
@@ -45,3 +45,9 @@
     {Level:"error", Message:"::error file=some/file/with/consecutive/errors,line=6::message kind one"},
 }
 ---
+
+[TestFormatErrors - 4]
+[]logger.LogMessage{
+    {Level:"", Message:"[{\"check_name\":\"editorconfig-checker\",\"description\":\"WRONG\",\"fingerprint\":\"bcd0ed212d202770869048ac50ceea7c\",\"severity\":\"minor\",\"location\":{\"path\":\"proc/cpuinfo\",\"lines\":{\"begin\":1,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"WRONG\",\"fingerprint\":\"8508b1c217914f89d2fbbd0b14eef007\",\"severity\":\"minor\",\"location\":{\"path\":\"proc/cpuinfoNOT\",\"lines\":{\"begin\":1,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"WRONG\",\"fingerprint\":\"3d1d5dbbb6516a1ff6dfd7d463e39c92\",\"severity\":\"minor\",\"location\":{\"path\":\"some/other/path\",\"lines\":{\"begin\":-1,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"WRONG\",\"fingerprint\":\"5047d0cb8e7ab136ef2c8409c7ef3ac3\",\"severity\":\"minor\",\"location\":{\"path\":\"some/other/path\",\"lines\":{\"begin\":1,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"file-level error\",\"fingerprint\":\"d4c61b75e2e08b3f2eb497e4bbe563d1\",\"severity\":\"minor\",\"location\":{\"path\":\"some/file/with/consecutive/errors\",\"lines\":{\"begin\":-1,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"message kind one\",\"fingerprint\":\"57fe25411e900e3f07da07aeea8c0f64\",\"severity\":\"minor\",\"location\":{\"path\":\"some/file/with/consecutive/errors\",\"lines\":{\"begin\":1,\"end\":1}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"message kind one\",\"fingerprint\":\"797f0cfb797675c4bed477a1067e8ef4\",\"severity\":\"minor\",\"location\":{\"path\":\"some/file/with/consecutive/errors\",\"lines\":{\"begin\":4,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"message kind two\",\"fingerprint\":\"72013244ffd1c1406ad764b9de9ce525\",\"severity\":\"minor\",\"location\":{\"path\":\"some/file/with/consecutive/errors\",\"lines\":{\"begin\":5,\"end\":0}}},{\"check_name\":\"editorconfig-checker\",\"description\":\"message kind one\",\"fingerprint\":\"0e3f77d4a6f1ffbcc7b8b8a971679f46\",\"severity\":\"minor\",\"location\":{\"path\":\"some/file/with/consecutive/errors\",\"lines\":{\"begin\":6,\"end\":0}}}]"},
+}
+---
diff --git a/pkg/error/error_test.go b/pkg/error/error_test.go
index ba3e679..a74478f 100644
--- a/pkg/error/error_test.go
+++ b/pkg/error/error_test.go
@@ -324,6 +324,9 @@ func TestFormatErrors(t *testing.T) {
 
 	config3 := config.Config{Format: "github-actions"}
 	s.MatchSnapshot(t, FormatErrors(input, config3))
+
+	config4 := config.Config{Format: "codeclimate"}
+	s.MatchSnapshot(t, FormatErrors(input, config4))
 }
 
 func TestMain(m *testing.M) {