Dynamsoft Service REST API Preview - Scan Documents in Node.js
Tags: node, javascript, programming, desktop
Published: 2023-09-11 01:22:59

Dynamsoft's [Web TWAIN SDK](https://www.dynamsoft.com/web-twain/overview/) has long been a market leader in web-based document scanning, empowering numerous organizations to develop their own document management systems. The SDK consisted of two main components: a Dynamsoft service for scanner management and a JavaScript library for front-end development. Previously, this service could only be accessed via the JavaScript library. With the upcoming release, the Dynamsoft service will expose the core document scanning feature via **REST API**. This new feature enables developers to use various programming languages for document scanning tasks. Now, in addition to **web-based** applications, the SDK can also be used to create **desktop** and **mobile** document scanning apps, as well as **server-side** scanning services. In this article, I will guide you through the process of using the new REST API for document scanning in Node.js.

## Prerequisites
-  Install [Dynamsoft Service REST Preview for Windows](https://www.dynamsoft.com/codepool/downloads/DynamsoftServiceSetup.msi).
    
    At present, the REST API is exclusively available on **Windows**, but support for **Linux (both x64 and ARM64)** and **macOS** is coming soon. 
- Request a [free trial license](https://www.dynamsoft.com/customer/license/trialLicense?product=dwt&source=codepool) for Dynamsoft Service.

## REST API Reference
By default, the REST API's host address is set to `http://127.0.0.1:18622`. To modify this to a LAN IP address, navigate to `http://127.0.0.1:18625/` in your web browser.

![Dynamsoft Service](https://www.dynamsoft.com/codepool/img/2023/09/dynamsoft-service-ip-configuration.png)

| Method | Endpoint        | Description                   | Parameters                         | Response                      |
|--------|-----------------|-------------------------------|------------------------------------|-------------------------------|
| GET    | `/DWTAPI/Scanners`    | Get a list of scanners  | None                               | `200 OK` with scanner list       |
| POST   | `/DWTAPI/ScanJobs`    | Creates a scan job      | `license`, `device`, `config`      | `201 Created` with job ID    |
| GET    | `/DWTAPI/ScanJobs/:id/NextDocument`| Retrieves a document image     | `id`: Job ID   | `200 OK` with image stream    |
| DELETE | `/DWTAPI/ScanJobs/:id`| Deletes a scan job       | `id`: Job ID                      | `200 OK`              |

The parameters for the `/DWTAPI/ScanJobs` endpoint are outlined in the [Dynamic Web TWAIN documentation](https://www.dynamsoft.com/web-twain/docs/info/api/Interfaces.html#DeviceConfiguration), and they control the scanner's behavior.

For instance, you can set the resolution to 200 DPI and the pixel type to color:

```javascript
let parameters = {
    license: "LICENSE-KEY",
    device: device,
};

parameters.config = {
    IfShowUI: false,
    PixelType: 2, // color
    Resolution: 200,
    IfFeederEnabled: false,
    IfDuplexEnabled: false,
};
```

## Developing Node.js Functions to Call the REST API
Install [Axios](https://www.npmjs.com/package/axios) for sending HTTP requests directly from Node.js to RESTful APIs and retrieve their responses.

```bash
npm install axios
```

According to the REST API reference, we implement five functions: `getDevices()`, `scanDocument()`, `deleteJob`, `getImageFiles` and `getImageStreams()`.

```javascript
const axios = require('axios');
const fs = require('fs');
const path = require('path');

module.exports = {
    getDevices: async function (host) {
        return [];
    },
    scanDocument: async function (host, parameters) {
        return '';
    },
    deleteJob: async function (host, jobId) {
    },
    getImageFiles: async function (host, jobId, directory) {
        let images = [];
        return images;
    },
    getImageStreams: async function (host, jobId) {
        let streams = [];
        return streams;
    },
};
```

- `getDevices()` retrieves a list of scanners.
    
    ```javascript
    getDevices: async function (host) {
        devices = [];
        let url = host + '/DWTAPI/Scanners'
        try {
            let response = await axios.get(url)
                .catch(error => {
                    console.log(error);
                });

            if (response.status == 200 && response.data.length > 0) {
                console.log('\nAvailable scanners: ' + response.data.length);
                return response.data;
            }
        } catch (error) {
            console.log(error);
        }
        return [];
    },
    ```

- `scanDocument()` creates a scan job and returns its ID.

    ```javascript
    scanDocument: async function (host, parameters) {
        let url = host + '/DWTAPI/ScanJobs';

        try {
            let response = await axios.post(url, parameters)
                .catch(error => {
                    console.log('Error: ' + error);
                });

            let jobId = response.data;

            if (response.status == 201) {
                return jobId;
            }
            else {
                console.log(response);
            }
        }
        catch (error) {
            console.log(error);
        }


        return '';
    },
    ```

- `deleteJob()` deletes a scan job.

    ```javascript
    deleteJob: async function (host, jobId) {
        if (!jobId) return;

        let url = host + '/DWTAPI/ScanJobs/' + jobId;
        console.log('Delete job: ' + url);
        axios({
            method: 'DELETE',
            url: url
        })
            .then(response => {
                console.log('Deleted:', response.data);
            })
            .catch(error => {
            });
    },
    ```

- `getImageFiles()` retrieves the image files of a scan job.

    ```javascript
    getImageFiles: async function (host, jobId, directory) {
        let images = [];
        let url = host + '/DWTAPI/ScanJobs/' + jobId + '/NextDocument';
        console.log('Start downloading images......');
        while (true) {
            try {
                const response = await axios({
                    method: 'GET',
                    url: url,
                    responseType: 'stream',
                });

                if (response.status == 200) {
                    await new Promise((resolve, reject) => {
                        const timestamp = Date.now();
                        const imagePath = path.join(directory, `image_${timestamp}.jpg`);
                        const writer = fs.createWriteStream(imagePath);
                        response.data.pipe(writer);

                        writer.on('finish', () => {
                            images.push(imagePath);
                            console.log('Saved image to ' + imagePath + '\n');
                            resolve();
                        });

                        writer.on('error', (err) => {
                            console.log(err);
                            reject(err);
                        });
                    });
                }
                else {
                    console.log(response);
                }

            } catch (error) {
                console.error('No more images.');
                break;
            }
        }

        return images;
    },
    ```

- `getImageStreams()` retrieves the image streams of a scan job.

    ```javascript
    getImageStreams: async function (host, jobId) {
        let streams = [];
        let url = host + '/DWTAPI/ScanJobs/' + jobId + '/NextDocument';
        console.log('Start downloading images......');
        while (true) {
            try {
                const response = await axios({
                    method: 'GET',
                    url: url,
                    responseType: 'stream',
                });

                if (response.status == 200) {
                    streams.push(response.data);
                }
                else {
                    console.log(response);
                }

            } catch (error) {
                console.error('No more images.');
                break;
            }
        }

        return streams;
    },
    ```

## Scanning Documents from the Command Line in Terminal
Let's create a `app.js` file for scanning documents from the command line.
1. Import `docscan4nodejs` and `readline`. The `docscan4nodejs` module is what we just implemented, and `readline` is used to read user input from the command line.

    
    ```javascript
    const docscan4nodejs = require("docscan4nodejs")
    const readline = require('readline');
    ```

2. Create a `readline.Interface` instance to read user input from the command line.

    ```javascript
    const rl = readline.createInterface({
        input: process.stdin,
        output: process.stdout
    });

    rl.question(questions, function (answer) {});
    ```

3. Get all available scanners that compatible with **TWAIN**, **SANE**, **ICA**, **WIA**, and **eSCL**.

    ```javascript
    let devices = await docscan4nodejs.getDevices(host);
    ```

4. Select a scanner from the list for scanning documents. A valid license key is required.

    ```javascript
    let parameters = {
        license: "LICENSE-KEY",
        device: devices[index].device,
    };

    parameters.config = {
        IfShowUI: false,
        PixelType: 2,
        //XferCount: 1,
        //PageSize: 1,
        Resolution: 200,
        IfFeederEnabled: false,
        IfDuplexEnabled: false,
    };

    docscan4nodejs.scanDocument(host, parameters).then((jobId) => {
        if (jobId !== '') {
            console.log('job id: ' + jobId);
            (async () => {
                let images = await docscan4nodejs.getImageFiles(host, jobId, './');
                for (let i = 0; i < images.length; i++) {
                    console.log('Image ' + i + ': ' + images[i]);
                }
                await docscan4nodejs.deleteJob(jobId);
            })();
        }

    });
    ```

5. Run the script in the terminal.

    ```bash
    node app.js
    ```

    **Get all available scanners**

    ![scanner list](https://www.dynamsoft.com/codepool/img/2023/09/scanner-source-list.png)
    
    **Acquire a Document**    

    ![command-line document scan](https://www.dynamsoft.com/codepool/img/2023/09/command-line-scan-document.png)

## Implementing Server-side Document Scanning for Web-Based Applications
A more advanced use case involves implementing server-side document scanning for web-based applications. The key benefit of this strategy is that it enables document scanning directly from a web browser, eliminating the need for additional software installations. For instance, if your office has a single document scanner and you wish to share it among multiple colleagues, a web-based application would allow them to initiate scans from their individual computers.

We use `express` to create a web server and `socket.io` to transmit data between the server and the client.

```bash
npm install express socket.io
```

**Node.js Web Server**

1. Initialize `express` and `socket.io`.

    ```javascript
    const express = require('express');
    const path = require('path');
    const fs = require('fs');
    const app = express();
    const http = require('http');
    const server = http.createServer(app);
    const io = require('socket.io')(server);

    const docscan4nodejs = require("docscan4nodejs")
    const { PassThrough } = require('stream');

    app.use(express.static('public'));
    app.use('/node_modules', express.static(__dirname + '/node_modules'));

    const connections = new Map();
    io.on('connection', (socket) => {
        connections.set(socket.id, socket);
        console.log(socket.id + ' connected');

        socket.on('disconnect', () => {
            console.log(socket.id + ' disconnected');
            connections.delete(socket.id);
        });

        socket.on('message', async (message) => {
            
        });

    });

    // Start the server
    const port = process.env.PORT || 3000;

    server.listen(port, '0.0.0.0', () => {
        console.log(`Server running at http://0.0.0.0:${port}/`);
    });

    ```
2. As the `socket.io` connection is established, get the available scanners and send them to the client.

    ```javascript
    io.on('connection', (socket) => {
        ...
        
        docscan4nodejs.getDevices(host).then((scanners) => {
            socket.emit('message', JSON.stringify({ 'devices': scanners }));
        });
    });
    ```

3. When receiving the scan event, initiate a document scan and transmit the resulting image stream to the web client.:
    
    ```javascript
    socket.on('message', async (message) => {
        let json = JSON.parse(message);
        if (json) {
            if (json['scan']) {
                console.log('device: ' + json['scan']);
                let parameters = {
                    license: "LICENSE-KEY",
                    device: json['scan'],
                };

                parameters.config = {
                    IfShowUI: false,
                    PixelType: 2,
                    //XferCount: 1,
                    //PageSize: 1,
                    Resolution: 200,
                    IfFeederEnabled: false,
                    IfDuplexEnabled: false,
                };

                let jobId = await docscan4nodejs.scanDocument(host, parameters);

                if (jobId !== '') {
                    console.log('job id: ' + jobId);
                    let streams = await docscan4nodejs.getImageStreams(host, jobId);
                    for (let i = 0; i < streams.length; i++) {
                        await new Promise((resolve, reject) => {
                            try {
                                const passThrough = new PassThrough();
                                const chunks = [];
    
                                streams[i].pipe(passThrough);
    
                                passThrough.on('data', (chunk) => {
                                    chunks.push(chunk);
                                });
    
                                passThrough.on('end', () => {
                                    const buffer = Buffer.concat(chunks);
                                    socket.emit('image', buffer);
                                    resolve();
                                });
                            }
                            catch (error) {
                                reject(error);
                            }
                        });
                    }
                }
            }
        }
    });
    ```

**Web Client**
1. Establish a `socket.io` connection with the server.

    ```javascript
    const socket = io();
    var data = [];
    var devices = [];
    var selectSources = document.getElementById("sources");
    socket.on('message', (message) => {
    });

    socket.on('image', (buffer) => {
    });
    ```

2. Update the `<select>` element with the available scanners.

    ```javascript
    socket.on('message', (message) => {
        try {
            let json = JSON.parse(message);
            if (json) {
                if (json['devices']) {
                    selectSources.options.length = 0;
                    devices = json['devices'];
                    for (let i = 0; i < devices.length; i++) {
                        console.log('\nIndex: ' + i + ', Name: ' + devices[i]['name']);
                        let option = document.createElement("option");
                        option.text = devices[i]['name'];
                        option.value = i.toString();
                        selectSources.add(option);
                    }
                }
            }
        } catch (error) {
            console.log(error)
        }
    });
    ```

3. Send a scan event to the server when the user clicks the `Scan` button.

    ```javascript
    <button onclick="acquireImage()">Scan Documents</button>
    function acquireImage() {
        if (devices.length > 0 && selectSources.selectedIndex >= 0) {
            socket.emit('message', JSON.stringify({ 'scan': devices[selectSources.selectedIndex]['device'] }));
        }

    }
    ```

4. Display the image stream in the `<img>` element.

    ```javascript
    socket.on('image', (buffer) => {
        // Convert the Buffer into a base64 string
        const base64Image = btoa(
            new Uint8Array(buffer)
                .reduce((data, byte) => data + String.fromCharCode(byte), '')
        );

        // Set the image src to display the image
        let img = document.getElementById('document-image');
        let url = `data:image/jpeg;base64,${base64Image}`;
        img.src = url;

        data.push(url);

        let option = document.createElement("option");
        option.selected = true;
        option.text = url;
        option.value = url;

        let thumbnails = document.getElementById("thumb-box");
        let newImage = document.createElement('img');
        newImage.setAttribute('src', url);
        if (thumbnails != null) {
            thumbnails.appendChild(newImage);
            newImage.addEventListener('click', e => {
                if (e != null && e.target != null) {
                    let target = e.target;
                    img.src = target.src;
                }
            });
        }
    });
    ```

Start the web server on your Windows machine, and then navigate to `http://LAN-IP:18625/` using Safari on macOS.

![server side document scan](https://www.dynamsoft.com/codepool/img/2023/09/server-side-document-scan.png)

## Source Code
[https://github.com/yushulx/dynamsoft-service-REST-API](https://github.com/yushulx/dynamsoft-service-REST-API)