How to Capture Documents On the Go and Collaborate with Web TWAIN Scanning SDK
Tags: webdev, javascript, programming, node
Published: 2023-03-22 01:55:55

Document capture functionality is an essential aspect of any document management system, allowing users to capture digital images of physical documents through a variety of means, including scanners and mobile cameras. Dynamic Web TWAIN helps users easily integrate document capture functionality into their web-based document management system, enabling them to streamline their workflows and improve efficiency. Within this article, a collaboration scenario is presented where documents can be captured via a mobile camera and uploaded as images to a remote document management system built with Dynamic Web TWAIN SDK.


<video src="https://user-images.githubusercontent.com/2202306/226501588-8e7dec62-63d7-4862-8685-00a1129761d4.mp4" controls="controls" muted="muted" style="max-height:640px; min-height: 200px; max-width: 100%;"></video>

## The Scenario: Mobile Capture for Document Management System
Efficient and streamlined document management systems are crucial for organizations to keep up with the fast-paced business world of today. One crucial aspect of such systems is the ability to capture and upload documents quickly and easily. Mobile camera technology offers a more convenient and flexible solution compared to traditional scanners. This scenario presents a common situation where a user captures a document and it is instantly viewable and processable in the organization's document management system.

To achieve this, several Node.js packages can be utilized, including:

- [express](https://www.npmjs.com/package/express) - a web application framework for Node.js
- [multer](https://www.npmjs.com/package/multer) - a Node.js middleware for handling `multipart/form-data`, which is primarily used for uploading files
- [sqlite3](https://www.npmjs.com/package/sqlite3) - a Node.js package that provides an asynchronous API for accessing a SQLite database
- [socket.io](https://www.npmjs.com/package/socket.io) - a Node.js package that enables real-time, bidirectional and event-based communication between the browser and the server
- [qrcode](https://www.npmjs.com/package/qrcode) - a Node.js package that generates QR codes
- [dwt](https://www.npmjs.com/package/dwt) - Dynamic Web TWAIN SDK


## How to Create a Web Document Scanning Application
Dynamic Web TWAIN enables developers to easily integrate document scanning functionality into their web applications with a few lines of JavaScript code. Here are the basic steps involved:
1. Create a new Node.js project and install the required packages.
    ```bash
    npm install express dwt
    ```
2. Set up a web server using Express.
    ```javascript
    const express = require('express');
    const app = express();
    const port = 3000;
    app.listen(port, '0.0.0.0', () => {
        console.log(`Server running at http://0.0.0.0:${port}/`);
    });
    ```

    The ip variable is set to `0.0.0.0`, which tells Express to listen to all available IP addresses. You can also set the port variable to the desired port number.
3. Create a `public` folder for storing static files, such as HTML, CSS, and JavaScript files. Copy [dwt-ui.html](https://github.com/yushulx/dynamic-web-twain-custom-ui/blob/main/dwt-ui.html) to the `public` folder and rename it to `index.html`. 
    
    This file demonstrates how to use Dynamic Web TWAIN API to create a web document scanning application. You need to apply for a [trial license](https://www.dynamsoft.com/customer/license/trialLicense?product=dwt) and replace the license key in `index.html`. 
    
    ```js
    Dynamsoft.DWT.ProductKey = "LICENSE-KEY";
    ```

    Since the resource files of Dynamic Web TWAIN are located in the `node_modules/dwt` folder, you need to configure the `node_modules` folder as a static folder. Add the following code to the `app.js` file:
    ```javascript
    app.use(express.static('public'));
    app.use('/node_modules', express.static(__dirname + '/node_modules'));
    ```
4. A simple web document scanning application is now ready. Run the following command to start the server: 
    ```bash
    node app.js
    ```

    ![Dynamic Web TWAIN hello world](https://www.dynamsoft.com/codepool/img/2023/03/dynamic-web-twain-hello-world.png)

## How to Send and Receive Data in Real-time on the Web
`Socket.IO` is a JavaScript library that enables real-time, bidirectional communication between web clients and servers. It uses a WebSocket protocol under the hood. In this scenario, `Socket.IO` is used to push the file name of the uploaded documents to the web client for display.

Here are the steps to integrate `Socket.IO` into the web client and server:
1. Install the `socket.io` package:
    ```bash
    npm install socket.io
    ```
2. Modify the `app.js` file to create a `Socket.IO` server:
    ```javascript
    const app = express();
    const http = require('http');
    const server = http.createServer(app);
    const port = process.env.PORT || 3000;

    const connections = new Map();
    io.on('connection', (socket) => {
        connections.set(socket.id, socket);
        console.log(socket.id + ' connected');

        socket.on('disconnect', () => {
            console.log(socket.id + ' disconnected');
            connections.delete(socket.id);
        });

        socket.on('message', (message) => {
            console.log('Message received: ' + message);
            socket.emit('message', 'from server');
        });
    });

    server.listen(port, '0.0.0.0', () => {
        console.log(`Server running at http://0.0.0.0:${port}/`);
    });
    ```
    
3. Add the following code to the `index.html` file to create a `Socket.IO` client:
    ```javascript
    <script src="/node_modules/socket.io/client-dist/socket.io.js"></script>
    const socket = io();
    socket.on('message', (data) => {
    });
    ```

## How to Generate a QR Code in JavaScript
QR code is a two-dimensional barcode that can be scanned using a mobile camera. It is a convenient way to share information between devices. In this scenario, a QR code is generated to display the URL of the web document scanning application. This allows users to scan the QR code using their mobile camera and open the web document scanning application on their mobile devices.

To generate a QR code in JavaScript:
1. Include the `qrcode` package in the `index.html` file:
    ```html
    <script src="/node_modules/qrcode/build/qrcode.js"></script>
    ```
2. Create a `canvas` element used for rendering the QR code:
    ```html
    <div class="qr-popup">
        <div class="qr-box">
            <span class="qr-close">&times;</span>
            <canvas id="qr-canvas"></canvas>
        </div>
    </div>
    ```
3. Encode the URL of the document capture and upload page as a QR code and display it in the `canvas` element:
    ```javascript
    const generateBtn = document.getElementById('generate');
    const qrPopup = document.querySelector(".qr-popup");
    const qrClose = document.querySelector(".qr-close");

    generateBtn.addEventListener('click', () => {
        let url = '';
        if (window.location.protocol === 'https:') {
            url = 'https://' + window.location.host + "/mobile.html?socketid=" + socket.id;
        } else {
            url = 'http://' + window.location.host + "/mobile.html?socketid=" + socket.id;
        }
        console.log(url);
        QRCode.toCanvas(document.getElementById('qr-canvas'), url, function (error) {
            if (error) console.error(error)
        })

        qrPopup.style.display = "block";

        qrClose.onclick = function () {
            qrPopup.style.display = "none";
        };
    });
    ```

    
4. Read the QR code using a mobile camera or a QR code reader app.

    ![JavaScript QR code](https://www.dynamsoft.com/codepool/img/2023/03/javascript-qr-code.png)

## How to Upload Images and Save to Database
Install `multer` and `sqlite3` packages:
```bash
npm install multer sqlite3
```

**On the client side**

An input element with type `file` can be used to capture images from a mobile camera. 
```html
<form id='myForm' action="javascript:void(0)" method="POST" enctype="multipart/form-data">
    <input type="file" name="image" accept="image/*">
    <button type="submit">Upload</button>
</form>
```


Using `fetch` API can upload files to a web server. 
```javascript
const form = document.getElementById('myForm');

form.addEventListener('submit', (event) => {
    event.preventDefault();

    const formData = new FormData(form);
    let id = decodeURIComponent(window.location.search.replace(new RegExp("^(?:.*[&\\?]" + encodeURIComponent("socketid").replace(/[\.\+\*]/g, "\\$&") + "(?:\\=([^&]*))?)?.*$", "i"), "$1"));
    formData.append('id', id);

    fetch('/upload', {
        method: 'POST',
        body: formData
    })
        .then(function (response) {
            console.log(response);
            showMessage("Your file has been uploaded.");
        })
        .catch(function (error) {
            console.error(error);
            showMessage(error, 3000);
        });
});
```

**On the server side**
 
The `multer` package is used to save the uploaded images on the server-side. 
```js
const multer = require('multer');
const upload = multer({ dest: 'uploads/' });

app.use('/uploads', express.static(__dirname + '/uploads'));

app.post('/upload', upload.single('image'), (req, res) => {
    const id = req.body.id;
    const { filename, mimetype, size } = req.file;
});
```
By default, Multer assigns a random name to the uploaded file to avoid naming collisions and security vulnerabilities. 

After storing an image file on the disk, you would typically add its filename and MIME type to a database, which would allow you to retrieve the image later on.

```js
const sqlite3 = require('sqlite3').verbose();

app.post('/upload', upload.single('image'), (req, res) => {
    const id = req.body.id;
    const { filename, mimetype, size } = req.file;
    db.run('INSERT INTO images (filename, mimetype, size) VALUES (?, ?, ?)',
        [filename, mimetype, size],
        (err) => {
            if (err) {
                console.error(err);
                return res.sendStatus(500);
            }
            connections.get(id).emit('message', JSON.stringify({ 'filename': filename}));
            res.sendStatus(200);
        });
});
```

Upon receiving a request for an image, the web server will query the associated database to retrieve the file name and MIME type. With this information, the server will utilize the fs package to read the image file and send it to the client.

```js
const fs = require('fs');
app.get('/image/:id', (req, res) => {
    const id = req.params.id;
    const imagePath = path.join(__dirname, 'uploads', id);

    db.all('SELECT mimetype FROM images WHERE filename = ?', [id], (err, rows) => {
        if (err) {
            console.error(err);
            return res.sendStatus(500);
        }

        if (rows.length > 0) {
            let mimetype = rows[0].mimetype;

            fs.readFile(imagePath, (err, data) => {
                if (err) {
                    res.status(404).send('Image not found');
                } else {
                    res.setHeader('Content-Type', mimetype);
                    res.send(data);
                }
            });
        }
        else {
            res.sendStatus(404);
        }
    });

});
```

![document mobile capture](https://www.dynamsoft.com/codepool/img/2023/03/web-document-scanning-mobile-capture.png)

## Source Code
[https://github.com/yushulx/web-twain-document-scan-management/tree/main/examples/mobile_camera](https://github.com/yushulx/web-twain-document-scan-management/tree/main/examples/mobile_camera)