## 3 Algorithms to Improve the Data Write Rate
### Data Write Performance Tests Using Binary Formats
#### 3.2.4 Discussion  
In applications such as AE and vibration monitoring, sensor reading values are frequently acquired in real time as data chunks. Writing several small chunks into a single file requires more time than writing a few large chunks of the same total size. Storing every data chunk in a file as soon as it becomes available leads to a poor write performance. Therefore, Approach 1 is not recommended. To achieve a better write performance, it is recommended to temporarily store the available data in a buffer and wait for more data to arrive before writing all of them into a file. The buffer size should be configured to be sufficiently large to hold data temporarily before writing. Typically, an internal buffer is generated while creating a new file (or opening an existing file). However, an internal buffer is used for both reading and writing; therefore, the write performance is not as good as expected. Instead, it is recommended that a self-created buffer be applied, as described in Approach 2. Using this approach, the average write rate reached the best performance at 162 MB/s in the test applying a buffer size of 512 MB.  
As described above, each write should be performed using a new thread. To avoid access conflicts between the threads, a new file is created for each write. Creating separate threads to write to a single file presents the risk of one thread trying to access the file while another thread is still accessing it, resulting in errors. However, multiple-file parallel writing should be avoided due to poor performance, as demonstrated by the results of Approach 3.  
Compared with the write rate of 16 MB/s when storing data as text files, binary formats provide a considerably faster write rate of 10\(\times\). In addition, the 2-byte binary format can reduce the file size by 4\(\times\) for the same precision. This combination of a faster write rate and a smaller file size results in a 40\(\times\) reduction in the write time or a 40\(\times\) better write performance. A 16-bit resolution requires 2 bytes to store a value, meaning that a  
Figure 7: Data write performance of the multi-threaded write. (**a**) Buffer size = 128 MB and (**b**) buffer size = 256 MB.  
write rate of 162 MB/s provides a measurement ability of 81 MS/s. In the case of a 24-bit resolution, 3 bytes are required to store a value; therefore, the same write rate can offer a measurement ability of 54 MS/s. These abilities satisfy the high sampling rate requirements of AE measurements.  
In this study, the buffer sizes were selected as powers of two, such as 512, 256, or 128 MB, for ease of implementation. However, the tests showed that the difference in the data write rates between the 130 MB and 128 MB configurations was not remarkable. Therefore, it is not mandatory to use a buffer size with a power of 2, as the write rate increases regardless of the buffer size. Additionally, for every configuration in each approach, the writing test was conducted multiple times. There was no remarkable difference between the average write rates of these tests and those described in Table 2. This stability is attributed to the 5 GB of data used in each writing test, which was sufficiently large to ensure generalization.