lil-csv, a 1k JS module to parse and generate CSV files 
Tags: javascript, csv, node, npm
Published: 2021-07-10 12:43:06

I was struggling to find a small JavaScript module to parse CSV (Comma Separated Values) files. All the existing modules have one or more shortcomings:

* Do not work in browsers;
* Large (all I found are at least 10kb min.js.gz).
* Cannot parse to deep objects.
* Cannot generate CSV from deep objects.

I'm pretty sure CSV parsing can be implemented in less than 1Kb min.js.gz. So I did it.

Please meet the powerful [`lil-csv`](https://www.npmjs.com/package/lil-csv).

That's how large it is (v1.3.1):
```
       1465 B: lil-csv.js.gz
       1313 B: lil-csv.js.br
       1315 B: lil-csv.modern.js.gz
       1201 B: lil-csv.modern.js.br
       1480 B: lil-csv.module.js.gz
       1327 B: lil-csv.module.js.br
       1524 B: lil-csv.umd.js.gz
       1359 B: lil-csv.umd.js.br
```

There are tradeoffs though.

1. It doesn't accept streams but only the full CSV file content as a single string. (Also remember: [Premature optimisation is the root of all evil](https://stackify.com/premature-optimization-evil/).)
2. _...you tell me..._

## Simple example

### CSV to JavaScript objects

Let's say you have a CSV like this:

```csv
name,address,course
John Smith,"123 John St, CARLTON",Advanced Calculus
Any Newman,"4a/3a Church Ave, CROYDON",Advanced Calculus
```

Parse it to objects:

```js
import { parse } from "lil-csv";
const objects = parse(fileContents);
console.log(objects);

/*
[
  {
    "name": "John Smith",
    "address": "123 John St, CARLTON",
    "course": "Advanced Calculus"
  },
  {
    "name": "Any Newman",
    "address": "4a/3a Church Ave, CROYDON",
    "course": "Advanced Calculus"
  }
]
*/
```

And stringify them back to CSV string:

```js
import { generate } from "lil-csv";
const string = generate(objects);
console.log(string);

/*
name,address,course
John Smith,"123 John St, CARLTON",Advanced Calculus
Any Newman,"4a/3a Church Ave, CROYDON",Advanced Calculus
*/
```

So, in essence stringifying plus parsing is an idempotent operation:

```js
assert.deepEqual(objects, parse(generate(objects)));
```

### CSV to JavaScript array of arrays

If you just need arrays of strings (not objects) then here is how you do it:

```js
const arrays = parse(fileContents, { header: false });
console.log(arrays);

/*
[
  ["name","address","course"],
  ["John Smith","123 John St, CARLTON","Advanced Calculus"],
  ["Any Newman","4a/3a Church Ave, CROYDON","Advanced Calculus"]
]
*/
```

Stringyfing back to CSV is simple:

```js
const string = generate(arrays, { header: false });
console.log(string);

/*
name,address,course
John Smith,"123 John St, CARLTON",Advanced Calculus
Any Newman,"4a/3a Church Ave, CROYDON",Advanced Calculus
*/
```

## Complex example

### Parsing numbers, dates, booleans

In real world the data is rarely all strings. Often your objects have to have numbers, dates, booleans, etc. Here is how to parse CSV with all kind of data.

Let's parse some strings, dates, numbers and booleans from the following CSV file:

```csv
firstName,lastName,dob,price,completed
John,Smith,1999-01-15,123.00,Y
Alice,Dwarf,1991-11-24,123.00,N
```

Converting custom string to the JS objects, and leaving all other data as strings:

```js
const people = parse(fileContents, {
  header: {
    "*": String,
    dob: v => v ? new Date(v) : null,
    price: v => isNaN(v) ? null : Number(v),
    completed: v => String(v).toUpperCase() === "Y",
  }
});

console.log(people);

/*
[
  {
    "firstName": "John",
    "lastName": "Smith",
    "dob": "1999-01-15T00:00:00.000Z",
    "price": 123.00,
    "completed": true
  },
  {
    "firstName": "Alice",
    "lastName": "Dwarf",
    "dob": "1991-11-24T00:00:00.000Z",
    "price": 123.00,
    "completed": false
  }
]
*/
```

### Generating custom CSV

Here is how you can convert booleans to strings like "Y" and "N", and also convert JS `Date` to calendar dates like "YYYY-MM-DD", and add custom formatting to numbers like "123.00" instead of the default "123":

```js
const string = generate(people, {
  header: {
    "*": String,
    dob: v => v ? new Date(v).toISOString().substr(0, 10) : "",
    price: v => isNaN(v) ? "" : Number(v).toFixed(2),
    completed: v => v ? "Y" : "N",
  }
});

console.log(string);

/*
firstName,lastName,dob,price,completed
John,Smith,1999-01-15,123.55,Y
Alice,Dwarf,1991-11-24,123.55,N
*/
```

## Renaming column headers

### Converting CSV column headers to JS property names

Of course people rarely use JavaScript property names for column headers. You would likely see "Date of birth" in CSV file header instead of "dob". The [`lil-csv`](https://www.npmjs.com/package/lil-csv) is little but powerful. It can can handle that too.

That's how you can rename headers during CSV file parsing and CSV file generation.

You file:

```csv
First name,Last name,Date of birth,Price in dollars,Completed
John,Smith,1999-01-15,123.00,Y
Alice,Dwarf,1991-11-24,123.00,N
```

Renaming each column to a JS object property:

```js
const people = parse(fileContents, {
  header: {
    "First name": "firstName",
    "Last name": "lastName",
    "Date of birth": {
      newName: "dob",
      parse: v => v ? new Date(v) : null,
    },
    "Price in dollars": {
      newName: "price",
      parse: v => isNaN(v) ? null : Number(v),
    },
    Completed: {
      newName: "completed",
      parse: v => String(v).toUpperCase() === "Y",
    },
  }
});

console.log(people);

/*
[
  {
    "firstName": "John",
    "lastName": "Smith",
    "dob": "1999-01-15T00:00:00.000Z",
    "price": 123.00,
    "completed": true
  },
  {
    "firstName": "Alice",
    "lastName": "Dwarf",
    "dob": "1991-11-24T00:00:00.000Z",
    "price": 123.00,
    "completed": false
  }
]
*/
```

### Renaming JS properties to real world column headers

I hope this code is easy to read:

```js
const string = generate(people, {
  header: {
    firstName: "First name",
    lastName: "Last name",
    dob: {
      newName: "Date of birth",
      stringify: v => v ? new Date(v).toISOString().substr(0, 10) : "",
    },
    price: {
      newName: "Price in dollars",
      stringify: v => isNaN(v) ? "" : Number(v).toFixed(2),
    },
    completed: {
      newName: "Completed",
      stringify: v => v ? "Y" : "N",
    },
  }
});

console.log(string);

/*
First name,Last name,Date of birth,Price in dollars,Completed
John,Smith,1999-01-15,123.00,Y
Alice,Dwarf,1991-11-24,123.00,N
*/
```

## Deep objects support!

Here comes the true powers of [`lil-csv`](https://www.npmjs.com/package/lil-csv). You can parse CSV rows **directly** to deep objects like:

```json
{
   order_id: 51234,
   recipient: {
     firstName: "John",
     lastName: "Smith",
     dob: "1999-01-15T00:00:00.000Z",
     address: {
       street: "123 John St, CARLTON",
       country: "AU",
     }
   }
}
```

### Parsing CSV rows as JS deep objects

Let's parse this CSV to the above object:
```csv
ID,First name,Last name,Date of birth,Address,Country
51234,John,Smith,1999-01-15,"123 John St, CARLTON",AU
```

All you need is to **rename headers with dot notation**:

```js
const orders = parse(fileContents, {
  header: {
    ID: {
      parse: Number,
      newName: "order_id",
    },
    "First name": "recipient.firstName",
    "Last name": "recipient.lastName",
    "Date of birth": {
      newName: "recipient.dob",
      parse: v => v ? new Date(v) : null,
    },
    Address: "recipient.address.street",
    Country: "recipient.address.country",
  }
});

console.log(orders);
```

It works similar when generating a CSV file from deep data:

```js
const string = generate(orders, {
  header: {
    order_id: "ID",
    "recipient.firstName": "First name",
    "recipient.lastName": "Last name",
    "recipient.dob": {
      newName: "Date of birth",
      stringify: v => v ? new Date(v).toISOString().substr(0, 10) : "",
    },
    "recipient.address.street": "Address",
    "recipient.address.country": "Country",
  }
});

console.log(string);

/*
ID,First name,Last name,Date of birth,Address,Country
51234,John,Smith,1999-01-15,"123 John St, CARLTON",AU
*/
```

In the above code the "Date of birth" column gets:
1. deeply set/accessed within an object.
2. parsed/stringified in a custom way;
3. renamed;

## Afterword

You get all that power from 1 TCP packet, meaning less than [1460 bytes](https://stackoverflow.com/questions/2613734/maximum-packet-size-for-a-tcp-connection). Or even fewer bytes if you are using only one of the two functions (treeshaking is supported by [`lil-csv`](https://www.npmjs.com/package/lil-csv)).

If you need additional features from `lil-csv` then feel free open an issue here: https://github.com/flash-oss/lil-csv/issues