nao1215/csv - Read csv with validation in golang
Tags: showdev, csv, go
Published: 2024-05-12 02:11:14

## What is nao1215/csv package

The [nao1215/csv](https://github.com/nao1215/csv) package is a library that performs validation when loading CSV files. Validation is executed based on the rules specified in the struct tags.

The [nao1215/csv](https://github.com/nao1215/csv) package was developed with inspiration from  [go-playground/validator](https://github.com/go-playground/validator) and [shogo82148/go-header-csv](https://github.com/shogo82148/go-header-csv). I would like to express my gratitude here.


### How to use

Please attach the "validate:" tag to your structure and write the validation rules after it. It's crucial that the "order of columns" matches the "order of field definitions" in the structure. The csv package does not automatically adjust the order.

When using csv.Decode, please pass a pointer to a slice of structures tagged with struct tags. The csv package will perform validation based on the struct tags and save the read results to the slice of structures if there are no errors. If there are errors, it will return them as []error.

```go
package main

import (
	"bytes"
	"fmt"

	"github.com/nao1215/csv"
)

func main() {
	input := `id,name,age
1,Gina,23
a,Yulia,25
3,Den1s,30
`
	buf := bytes.NewBufferString(input)
	c, err := csv.NewCSV(buf)
	if err != nil {
		panic(err)
	}

	type person struct {
		ID   int    `validate:"numeric"`
		Name string `validate:"alpha"`
		Age  int    `validate:"gt=24"`
	}
	people := make([]person, 0)

	errs := c.Decode(&people)
	if len(errs) != 0 {
		for _, err := range errs {
			fmt.Println(err.Error())
		}
	}

	// Output:
	// line:2 column age: target is not greater than the threshold value: threshold=24.000000, value=23.000000
	// line:3 column id: target is not a numeric character: value=a
	// line:4 column name: target is not an alphabetic character: value=Den1s
}
```

### Struct tags

You set the validation rules following the "validate:" tag according to the rules in the table below. If you need to set multiple rules, please enumerate them separated by commas.

#### Validation rule without arguments

| Tag Name          | Description                                       |
|-------------------|---------------------------------------------------|
| boolean           | Check whether value is boolean or not.           |
| alpha             | Check whether value is alphabetic or not           |
| numeric           | Check whether value is numeric or not              |
| alphanumeric     | Check whether value is alphanumeric or not        |
| required          | Check whether value is empty or not                |

#### Validation rule with numeric argument

| Tag Name          | Description                                       |
|-------------------|---------------------------------------------------|
| eq                | Check whether value is equal to the specified value.<br> e.g. `validate:"eq=1"` |
| ne                | Check whether value is not equal to the specified value <br> e.g. `validate:"ne=1"` |
| gt                | Check whether value is greater than the specified value <br> e.g. `validate:"gt=1"` |
| gte               | Check whether value is greater than or equal to the specified value <br> e.g. `validate:"gte=1"` |
| lt                | Check whether value is less than the specified value <br> e.g. `validate:"lt=1"` |
| lte               | Check whether value is less than or equal to the specified value <br> e.g. `validate:"lte=1"` |
| min               | Check whether value is greater than or equal to the specified value <br> e.g. `validate:"min=1"` |
| max               | Check whether value is less than or equal to the specified value <br> e.g. `validate:"max=100"` |
| len 			    | Check whether the length of the value is equal to the specified value <br> e.g. `validate:"len=10"` |
| oneof             | Check whether value is included in the specified values <br> e.g. `validate:"oneof=male female prefer_not_to"` |




## Why I wrote [nao1215/csv](https://github.com/nao1215/csv)

It's the beginning of a long tale of memories.

Have you ever had the task of finding 10 errors within a CSV containing tens of thousands of rows and over 300 columns? I've experienced that several times.
  
I developed [nao1215/sqly](https://github.com/nao1215/sqly)([article is here] (https://dev.to/nchika/sqly-execute-sql-against-csv-json-with-shell-1li6)), which allows executing SQL on CSV files, because I found correcting CSVs to be challenging. However, sqly alone was insufficient.

In Japan, it's common to manage master data using Excel. However, it's not practical for apps to directly read data from Excel. Therefore, it's sometimes necessary to convert Excel to CSV and create apps to import the data into multiple database tables. However, there are cases where incorrect data is present in the CSV (it's strange to have incorrect data in master data, isn't it?), which prevents it from being imported into the database.


![work flow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/184hvl4dfyarsmhu14bf.png)
  
  
I despised this task. 

I just wanted to make it even a bit easier. I had someone else implement a validation process to check the CSV columns and identify where errors were (I didn't do it myself). However, I believed I could write the validation process more generically.

Over a year has passed since I started contemplating this. To bid farewell to this challenge, I secluded myself in my room for a day and created [nao1215/csv](https://github.com/nao1215/csv). It took over a year to accomplish what could have been done in just one day. 

## Next work

I plan to increase the variety of validations and implement faster validation processes. As a side note, I am not currently working with CSVs. I spent a day to resolve lingering regrets from past work. Therefore, the development pace of [nao1215/csv](https://github.com/nao1215/csv) may seem slow.

If everyone could gift GitHub Stars, I believe the development pace would be faster.

Thank you for reading this article.