npm package: @stenway/wsv-io
Description: Read and write WSV files
Author: Stefan John
Latest version: 1.1.0

﻿# WSV-IO

## About this package

This package is the **Node.js-specific part** mentioned in the environment-independent **[WSV](https://www.npmjs.com/package/@stenway/wsv)** package (You will find more information there about WSV in general). This package uses Node.js's **file system module** and offers simple classes to load and save WSV files. It offers stream reader and writer classes to read and write WSV files line-by-line.

If you want to get a first impression on how to use this package, you can watch [this video](https://www.youtube.com/watch?v=RZB0EMhk8hc). But always check the changelog of the presented packages for possible changes, that are
not reflected in the video.

## Getting started

First get the **WSV-IO package** installed with a package manager of your choice.
If you are using NPM just run the following command:
```
npm install @stenway/wsv-io
```

We import the WsvFile class from the WSV IO package
and parse an example WSV string to get a WsvDocument object.
We will pass this document as an argument to the static
method saveSync of the WsvFile class. We also specify the
file name Test.wsv.

```ts
import { WsvDocument } from '@stenway/wsv'
import { WsvFile } from '@stenway/wsv-io'

const filePath = "Test.wsv"
const document = WsvDocument.parse("a b\nc d #My comment")
WsvFile.saveSync(document, filePath)
```
After running the code, the file was created like this:

```
a b
c d #My comment
```
The file will be written with the default ReliableTXT encoding which is UTF-8.

We now change the encoding property of the document to
the ReliableTXT encoding UTF-16 and save the document as
another file. To use the ReliableTxtEncoding enum, we need to install the
[ReliableTXT](https://www.npmjs.com/package/@stenway/reliabletxt) package as well.
```ts
document.encoding = ReliableTxtEncoding.Utf16
WsvFile.saveSync(document, "TestUtf16.wsv")
```

If we wanna save our file and strip all unneccassary whitespace
characters and comments away, we can pass another argument
to the saveSync method.

```ts
WsvFile.saveSync(document, "TestWithoutComment.wsv", false)
```

And now the comment of the original document is gone:
```
a b
c d
```

### Loading WSV files

To load a WSV file, we use the static method loadSync
and we only need to provide the file path. An import aspect
of the load method is, that we will never have to specifiy
the encoding with which the file should be loaded, because
a WSV file is a ReliableTXT file and thus the encoding
is always automatically detected by reading the preamble bytes
at the start of the file.

```ts
const loadedDocument = WsvFile.loadSync("Test.wsv")
const loadedDocumentStr = loadedDocument.toString()
console.log(loadedDocument.encoding)
```
When we run the code, we can see that the file encoding UTF-8 was correctly
detected.

We now do the same with the UTF-16 encoded WSV file:

```ts
const loadedDocumentUtf16 = WsvFile.loadSync("TestUtf16.wsv")
const loadedDocumentUtf16Str = loadedDocumentUtf16.toString()
console.log(loadedDocumentUtf16.encoding)
```

And it will display the correctly detected UTF-16 encoding.

We can also load our first file, but without the comment,
again by specifying the second argument as false:
```ts
const loadedDocumentWithoutComment = WsvFile.loadSync("Test.wsv", false)
const loadedDocumentWithoutCommentStr = loadedDocumentWithoutComment.toString()
```

### Appending to a WSV file

A very helpful method, is the static method appendSync of the
WsvFile class which will either create a new WSV file when the
specified file does not exist, or will append the lines
of a WsvDocument to an existing file. The method will take care
of the encoding, so that there won't be any mismatches. Meaning
you cannot unintentionally append UTF-16 encoded lines, to a UTF-8 encoded file.

```ts
WsvFile.appendSync(loadedDocument, "Append.wsv")
WsvFile.appendSync(loadedDocumentUtf16, "Append.wsv")
```

### Big files

Concerning big WSV files, you can watch the [ReliableTXT NPM package video](https://youtu.be/a7dLaMv6F7Y?si=Pf0lrHSb5FtkqjaF&t=701)
which shows the problem of string and file size limits of V8
and Node.js which lead to certain restrictions when it comes to writing
and reading bigger files in one take. The WSV IO package 
uses the [ReliableTXT IO package](https://www.npmjs.com/package/@stenway/reliabletxt-io) so the same limits apply here as well,
concerning the saveSync and loadSync method. The limitations might change from version to version of Node.js.
With version 20 of Node.js you can say roughly, a 500MB WSV file can be written and read in one take,
but if you need bigger files you should use the stream writer and
reader classes or the appendSync method for writing.

### Streaming classes

We'll now have a look the synchronous WsvStreamWriter
class. We first create a writer object by calling the constructor
and passing the file name as an argument. We then iterate
a hundred times and write a line using the WriteLine method.
After that we close the writer and will execute the example.

```ts
import { SyncWsvStreamReader, SyncWsvStreamWriter } from "@stenway/wsv-io"
import { WsvLine } from "@stenway/wsv"

const writer = SyncWsvStreamWriter.create("Stream.wsv")
try {
	for (let i=0; i<100; i++) {
		writer.writeLine(new WsvLine([i.toString(), "a", "b", "c"]))
	}
} finally {
	writer.close()
}
```

To read the file line by line, we now create a synchronous
WsvStreamReader class. The readLine method returns a WsvLine object
or returns null, if the end of the file was reached.
We count the number of lines and print the number after closing
the reader.

```ts
const reader = SyncWsvStreamReader.create("Stream.wsv")
try {
	let count = 0
	while (true) {
		const line = reader.readLine()
		if (line === null) { break }
		count++
	}
	console.log(`Count: ${count}`)
} finally {
	writer.close()
}
```

## Asynchronous IO

Like the [ReliableTXT IO package](https://www.npmjs.com/package/@stenway/reliabletxt-io) the WSV-IO package
offers synchronous and asynchronous methods and classes. Synchronous methods always have the 'Sync' suffix, like saveSync or loadSync. To use the asynchronous versions just use the save and load methods of the WsvFile class like in the following example:

```ts
const document = WsvDocument.parse("a b\nc")
await WsvFile.save(document, "Test.wsv")
const loadedDocument = await WsvFile.load("Test.wsv")
```

An asynchronous stream reader analogously can be created like this:

```ts
const reader = await WsvStreamReader.create("Stream.wsv")
try {
	const line = await reader.readLine()
} finally {
	await reader.close()
}
```

## BinaryWSV Files

BinaryWSV is the binary representation of WSV documents. It starts with the magic code 'BW1'.
BinaryWSV is made for scenarios, where parsing speed of the textual representation might be a limitation.
Learn more about the format in [this video](https://www.youtube.com/watch?v=vR1bj6sArLU) or read more in
the documentation of the [WSV package](https://www.npmjs.com/package/@stenway/wsv).

This package offers functionality to load and save BinaryWSV files and to stream read and write them line-by-line. The static class BinaryWsvFile offers both synchronous and asynchronous static methods to load, save and append. In the following example the usage of the saveSync and loadSync method is shown:

```ts
const document = WsvDocument.parse("a b\nc")
BinaryWsvFile.saveSync(document, "Test.bwsv")
const loadedDocument = BinaryWsvFile.loadSync("Test.bwsv")
```

The classes and methods are analogous to the ones of the textual representation. A stream writer for a BinaryWSV file would simply look like this:

```ts
const writer = SyncBinaryWsvStreamWriter.create("Stream.bwsv")
try {
	for (let i=0; i<100; i++) {
		writer.writeLine(new WsvLine([i.toString(), "a", "b", "c"]))
	}
} finally {
	writer.close()
}
```