Real-time parameter evaluation of high-speed microfluidic droplets using continuous spike streams

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 18:25:15

Abstract: Droplet-based microfluidic devices, with their high throughput and low power consumption, have found wide-ranging applications in the life sciences, such as drug discovery and cancer detection. However, the lack of real-time methods for accurately estimating droplet generation parameters has resulted in droplet microfluidic systems remaining largely offline-controlled, making it challenging to achieve efficient feedback in droplet generation. To meet the real-time requirements, it's imperative to minimize the data throughput of the collection system while employing parameter estimation algorithms that are both resource-efficient and highly effective. Spike camera, as an innovative form of neuromorphic camera, facilitates high temporal resolution scene capture with comparatively low data throughput. In this paper, we propose a real-time evaluation method for high-speed droplet parameters based on spike-based microfluidic flow-focusing, named RTDE, that integrates spike camera into the droplet collection system to efficiently capture information using spike stream. To process the spike stream effectively, we develop a spike-based estimation algorithm for real-time droplet generation parameters. To validate the performance of our method, we collected spike-based droplet datasets (SDD), comprising synthetic and real data with varying flow velocities, frequencies, and droplet sizes. Experiments result on these datasets consistently demonstrate that our method achieves parameter estimations that closely match the ground truth values, showcasing high precision. Furthermore, comparative experiments with image-based parameter estimation methods highlight the superior time efficiency of our method, enabling real-time calculation of parameter estimations.

Main paper: 
 related work 2.1 droplet parameter estimation: 
 A droplet microfluidic system is a system capable of generating droplets at a microscale by manipulating fluids. The precise control of droplets relies not only on the properties of the input fluids but also on the design of microfluidic structures [35] . To meet the practical demands of droplet generation, it's often essential to fine-tune input parameters and optimize the design of microfluidic structures based on estimated parameters [1] . In traditional one-dimensional data acquisition systems, data collection typically only allows for quantitative analysis of the components within droplets, posing challenges in acquiring morphological characteristic parameters of the droplets [15] , thus rendering it unsuitable for parameter estimation in droplet microfluidics. Image-based methods offer significant advantages in this scenario. By utilizing captured image data combined with conventional digital image processing methods, relevant parameters of droplet generation can be extracted from the images [18] . However, to achieve continuous data acquisition in high-speed scenarios, the system requires high data throughput.In such cases, it becomes challenging to promptly utilize the collected data for practical parameter estimation, resulting in a lack of synchronization between data collection and parameter estimation, or requiring offline parameter estimation. This asynchrony severely affects the convenience of droplet microfluidic regulation and makes it difficult to promptly respond to fluctuations during actual operation. In recent years, the integration of deep learning into microfluidic applications has shown promising performance in various aspects such as microfluidic design [9, 36, 37] , control [9, 38] , and analysis [39] [40] [41] . However, deep learning training typically requires a large amount of data, and microfluidic experiments involve expensive equipment and complex operations, making data acquisition still a challenging task. This limitation results in the effectiveness of most models remaining limited to simple or specific scenarios, thus hindering widespread application. Additionally, the training and inference processes of these models are often time-consuming, failing to meet the real-time requirements essential for droplet control in microfluidics [1] . Hence, there is an urgent need for a method capable of realizing real-time estimation and feedback of droplet parameters.Spike cameras enable continuous recording of photons in a scene through asynchronous spike emission, offering exceptionally high temporal resolution and dynamic range. The naive approach to estimating droplet parameters from spike streams involves directly reconstructing them into grayscale images and subsequently applying traditional image-based parameter extraction algorithms. Several studies have endeavoured to reconstruct grayscale images from spike streams, which fall into three main categories: statisticbased methods [42] , bio-inspired methods [43, 44] , and deep learning methods [45, 46] . Statistic-based methods mainly include texture from playback (TFP) and texture from inter-spike intervals (TFI), with TFI offering higher temporal resolution compared to TFP. Bio-inspired methods primarily enhance reconstruction quality by incorporating biological principles but struggle to balance noise and motion blur simultaneously. Deep learning-based methods can achieve superior reconstruction quality but typically require extensive labelled datasets for training and still face challenges in generalization. Some research [46] has attempted to achieve reconstruction quality comparable to supervised methods using selfsupervised methods, which do not require labelled data for training and offer greater convenience. However, these methods still require considerable time consumption, making it difficult to match the rapid response and real-time performance requirements of droplet control. Therefore, the development of efficient droplet parameter estimation methods based on the inherent characteristics of spike streams remains profoundly significant. 
spike generation mechanisms: 
 The spike camera converts the photons in space into an electrical current through a photoelectric converter, which is accumulated in capacitors, forming an integral sampling. When the voltage on the capacitor exceeds a set threshold, a spike is fired, followed by a capacitor reset. This process can be represented as follows:EQUATIONwhere Δ𝑡 represents the readout period of the spike camera, 𝐼 (𝑡) denotes the electric current, and 𝜗 is the set threshold. When the current period exceeds this threshold, a spike is fired. Typically, a spike sensor consists of an array of 𝐻 × 𝑊 pixels, and during high-speed acquisition periods, the output spike stream forms an 𝐻 ×𝑊 ×𝑇 matrix. In contrast to event cameras, which focus solely on capturing dynamic information, spike cameras provide a complete record of scene information. The spike stream contains a wealth of spatiotemporal information, thereby offering a more detailed reference for subsequent processing. 
method 4.1 droplet generation frequency evaluation: 
 As spike streams inherently contain rich frequency information, the droplet generation frequency information can be directly extracted from them, as shown in Figure 2 (b). To begin, a rectangular block 𝐵 𝐿 with 𝐻 × 𝑊 pixels is designated at the location where the droplets traverse the microfluidic channel. The channel position can be identified from the image obtained by direct summation of spike streams. Here, the length of 𝐻 should be greater than the width of the microfluidic channel, while the size of 𝑊 should be smaller than the minimum value between the droplet spacing and droplet size. Since the droplet spacing and droplet size are unknown, the 𝑊 is usually set as 5 pixels. Subsequently, the spike streams within this block along the time domain are extracted, and this could be expressed as:EQUATIONwhere 𝑛 ∈ [1, 2, • • • , 𝐻 ×𝑊 ] and 𝑠 𝑖 (𝑡) denotes the spike streams at the 𝑖 -𝑡ℎ point within block 𝐵 𝐿 . To mitigate the impact of background noise on the calculation process, the spike within the block 𝐵 𝐿 is accumulated at each point in time according to the following equation:EQUATIONNext, the cumulative spike data 𝑦 (𝑡) is subjected to Fourier transform to obtain the power spectrum within a designed time range 𝑐𝑇 (where 𝑇 represents the sampling period of the spike camera). This computation could be formulated as follows:𝑝𝑠𝑑 (𝑤) = lim 𝑐→∞ |𝐹 𝑐 (𝜔)| 2 𝑐 = lim 𝑐→∞ 𝑐 -1 𝑛=0 𝑦 (𝑛𝑇 ) 𝑒 -𝑗 2𝜋𝑛𝑤 𝑐 2 𝑐 (4)where 𝑐 is typically set to 20000 in this paper, owing to the 50 𝑢𝑠 for the data readout period of the spike camera. The power spectrum 𝑝𝑠𝑑 (𝑤) primarily comprises the fluctuation frequency 𝑓 𝑛 of background noise and its higher harmonics. The magnitude of this component is generally determined by the illumination intensity and the threshold for spike firing in the spike camera. Furthermore, the dominant frequency pertains to the droplet generation frequency 𝑓 𝑑 , which functions as a carrier wave overlying the background noise frequency 𝑓 𝑛 . By analyzing the illumination intensity in a background area, the frequency 𝑓 𝑛 range associated with background noise in the power spectrum could be determined. Peaks are then sought within the background frequency value 𝑓 𝑛 in the power spectrum. A curve fitting process is performed at the location of the peak, and the fitted curve's peak position corresponds to the droplet generation frequency 𝑓 𝑑 .Based on the principle of signal correlation, the velocity of droplets in microfluidic channels can be evaluated, as shown in Figure 2(b) . This approach entails transforming the droplet velocity into the assessment of phase for spike streams at two distinct positions along the microfluidic channel. Initially, two rectangular blocks 𝐵 𝐿1 and 𝐵 𝐿2 positioned on the microfluidic channel with a spacing of Δ𝑥 should be selected. The channel position can be identified from an image obtained by direct summation of spike streams. These blocks have uniform dimensions 𝐻 * 𝑊 pixels, with constraints akin to those outlined for the block 𝐵 𝐿 in Section 4.1, and the spacing Δ𝑥 should not exceed the length of one droplet cycle, which is typically kept as small as possible, for example, two pixels. Extracting the spike streams from these two blocks could yield two sets of signals, denoted as 𝑦 1 (𝑡) and 𝑦 2 (𝑡), expressed as shown in Eq.(3). Subsequently, we calculate the power spectra of 𝑦 1 (𝑡) and 𝑦 2 (𝑡). By employing the method described in section 4.1, we can determine the droplet generation frequencies, 𝑓 𝑑1 and 𝑓 𝑑2 , corresponding to the signal 𝑦 1 (𝑡) and 𝑦 2 (𝑡), which typically remain reasonably consistent.To mitigate the impact of noise on the computation of the droplet generation signal phase, we further perform the Fourier transform on 𝑦 1 (𝑡) and 𝑦 2 (𝑡), which could be calculated by the following formula:𝐹 1 (𝑤) = ∑︁ 𝑁 -1 𝑛=0 𝑦 1 (𝑛𝑇 ) 𝑒 -𝑗 2𝜋𝑛𝑤 𝑐 (5) 𝐹 2 (𝑤) = ∑︁ 𝑁 -1 𝑛=0 𝑦 2 (𝑛𝑇 ) 𝑒 -𝑗 2𝜋𝑛𝑤 𝑐 (6)where 𝑁 represents the length of the signals. Frequencies other than droplet generation frequencies 𝑓 𝑑1 and 𝑓 𝑑2 are filtered out on the frequency domains 𝐹 1 (𝑤) and 𝐹 2 (𝑤) respectively. This results in frequency domain signals 𝐹 1 (𝑓 𝑑1 ) and 𝐹 2 (𝑓 𝑑2 ), which are then inverse Fourier transformed back to the time domain and yield signals ŷ1 (𝑡) and ŷ2 (𝑡) containing solely the droplet generation frequency. Subsequently, we calculate the cross-correlation of the signal ŷ1 (𝑡)) and ŷ2 (𝑡) . For ŷ1 (𝑡) and ŷ2 (𝑡), we perform a Fourier series expansion, which could be calculated by the following formula:EQUATION)where 𝑓 0 = 0 corresponds to the direct current (DC) component (taking into account that we can filter out the DC components from ŷ1 (𝑡) and ŷ2 (𝑡) and not consider them later). Ψ 𝑖 and Ψ 𝑗 correspond to the intensity of frequency components 𝑓 𝑖 and 𝑓 𝑗 of ŷ1 (𝑡) and ŷ2 (𝑡) respectively. These intensities can be obtained by performing a Fourier transform on ŷ2 (𝑡) and ŷ2 (𝑡). 𝜙 𝑖 and 𝜙 𝑗 correspond to the phases of frequency components 𝑓 𝑖 and 𝑓 𝑗 of ŷ1 (𝑡) and ŷ2 (𝑡) respectively. Further calculate the product of ŷ1 (𝑡) and ŷ2 (𝑡) , which could be expressed as:EQUATIONAccording to the equation 𝑠𝑖𝑛 (𝑎)𝑠𝑖𝑛 (𝑏 ) = -1/2 * [𝑐𝑜𝑠 (𝑎 +𝑏 ) -𝑐𝑜𝑠 (𝑎 -𝑏 ) ], the Eq(9) can be further rewritten as:EQUATIONThe sequence ŷ1 (𝑡 ) × ŷ2 (𝑡 ) is then integrated over the period 𝐸 = 𝑡 𝑚 -𝑡 1 , where 𝑡 1 represents the starting time of the spike stream chosen for analysis, while 𝑡 𝑚 represents the ending time, which could be expressed as:EQUATIONwhen 𝐸 is sufficiently large, the𝑡𝑚 𝑡 =𝑡 1 𝑛 𝑖=1 𝑛 𝑗 ≠𝑖 1 2 Ψ 𝑖 Ψ 𝑗 𝑐𝑜𝑠 (2𝜋 (𝑓 𝑖 - 𝑓 𝑗 )𝑡 + 𝜙 𝑖 -𝜑 𝑗 ) and 𝑡𝑚 𝑡 =𝑡 1 𝑛 𝑖=1 𝑛 𝑗 =1 1 2 Ψ 𝑖 Ψ 𝑗 𝑐𝑜𝑠 (2𝜋 (𝑓 𝑖 + 𝑓 𝑗 )𝑡 + 𝜙 𝑖 + 𝜑 𝑗 )will only accumulate the value of the last incomplete period due to the presence of the cosine function. In comparison to the integral value of the 𝑡𝑚 𝑡 =𝑡 1 𝑛 𝑖=1 𝑛 𝑗 =𝑖 1 2 Ψ 𝑖 Ψ 𝑗 𝑐𝑜𝑠 (2𝜋 (𝑓 𝑖 -𝑓 𝑗 )𝑡 +𝜙 𝑖 -𝜑 𝑗 ), the contribution of these terms could be considered negligible. Therefore, the equation aboveEQUATIONThen, we can obtain the following approximate formula:EQUATIONSince only the droplet generation frequency 𝑓 𝑑 1 and 𝑓 𝑑 2 is present in signals ŷ1 (𝑡 ) and ŷ2 (𝑡 ) respectively, leting 𝛿 = 𝜙 𝑖 -𝜑 𝑖 , we can further obtain:EQUATIONletting 𝑓 = (𝑓 𝑑1 + 𝑓 𝑑2 )/2 , 𝑡 can be calculated by the following equation:EQUATIONFinally, the velocity of droplet generation can be calculated according to the following equation:EQUATIONThe size of the droplets can be directly obtained through the statistical characteristics of the spike stream, as shown in Figure 2 (c). Two lines of length 𝐿, parallel to the microfluidic channel are taken inside the channel, but at a certain distance apart. This distance is typically required to be less than half the size of the droplets, which are typically kept as small as possible, for example, two pixels. Then, the spike streams at each pixel on these lines are obtained, which can be expressed as:EQUATIONwhere 𝑖 ∈ {1, 2} represent different lines, 𝑦 𝑚𝑖 (𝑡 ) denotes the spike stream and 𝑚 ∈ {0, 𝐿}. For each pixel's spike stream, we can obtain the spike interval based on the TFI principle, resulting in:EQUATIONDue to the lower intensity of the droplet's edge compared to the background intensity, differences in spike intervals can arise. Therefore, by setting an appropriate threshold Θ, we can extract the positions where the droplet's edge lies. Taking 𝑑 1𝑖 (𝑡 ) as an example, we can obtain by:EQUATIONwhere F(•) is a function that sets positions that do not meet the condition to zero and 𝑝 1𝑖 (𝑡 ) contains values only at positions where the droplet's edge is located. Then, similarly, based on the TFI principle, we can obtain the interval between adjacent edges 𝑞 1𝑖 (𝑡 ), and further obtain the actual scale that the adjacent edges represent as 𝑙 1𝑖 (𝑡 ) by:EQUATIONIn theory, we can directly derive the size of the droplet from 𝑙 1𝑖 (𝑡 ). However, in practical scenarios where noise is present, obtaining a continuous edge is often challenging. This can lead to biases in single-pixel statistical values.To mitigate this issue, we can enhance the statistical features by aggregating values from a large number of pixels, Applying the same operation to all pixel points on 𝑌 𝑖 , we can obtain:EQUATIONwe then conduct statistical analysis on the values in 𝐿 𝑖 . The two values with the highest frequency are respectively considered as the droplet size and the droplet interval. However, directly deducing this from 𝐿 1 alone is not feasible. Therefore, by comparing the discrepancies in statistical values between 𝐿 1 and 𝐿 2 , along with their relative positions, we can ascertain the droplet size. Typically, as the distance from the centre of the pipeline increases, the two highest statistical values decrease correspondingly to the size of the droplets, while the increase pertains to the spacing between the droplets. The comparison results between the droplet velocities estimated by our method and the droplet flow velocities observed in actual measurements when the oil phase was fixed and the water phase parameters were continuously adjusted. (b) The comparison results between the droplet frequency estimated by our method and the droplet frequency observed in actual measurements when the oil phase was fixed and the water phase parameters were continuously adjusted (c) The comparison results between the droplet size estimated by our method and the droplet size observed in actual measurements when the water phase was fixed and the oil phase parameters were continuously adjusted. 
experiments 5.1 the spike-based droplets datasets: 
 Synthetic Dataset: We simulate droplet morphological features using circles and then simulate the flow process of the droplets by pixel movement to generate a video stream. Subsequently, we assign different sets of parameters including size, flow velocity, and frequency to the droplets, generating multiple sets of data, each containing 20,000 frames. These datasets are then employed as inputs to Spikingsim to simulate spike streams.Real-world Dataset: The real-world data primarily utilized spike cameras mounted on an electric microscope for capturing droplets microfluidics. The microfluidic chip for droplet manipulation has two inputs: oil and water. By shearing water with oil, droplets are formed. Adjusting the input pressure of these two components allows for fine-tuning of droplet generation parameters. In this work, we collected data sets with different input parameters, where the droplets encompass various generation frequencies, flow velocities, and sizes.To validate the accuracy of our spike-based parameter estimation method, we conducted experiments on simulated data. The primary goal was to confirm whether the parameter values estimated by our method were consistent with the settings employed in the simulation. The simulated dataset consisted of three sets of data, each featuring variations in droplet size, frequency, and velocity. Generally, two parameters were held constant while the third one was adjusted. For example, when adjusting droplet size, we kept the droplet's frequency and velocity constant while simulating various droplet sizes. Similar procedures were followed for the other two sets. As shown in Figure 3 , the results of the experiments on simulated data demonstrated that our method's estimated values closely matched the simulated preset values. However, there were slight discrepancies in the estimated values. For instance, when estimating the droplet size, these deviations were partially attributed to the inherent width of the droplet edge, as our statistical analysis only accounted for the outermost positions. Moreover, the finite time resolution of the camera affected the precision of droplet size estimation. Although our method effectively mitigated the influence of noise, some noise near the droplet generation frequency might persist, influencing the accuracy of the final fitting process and resulting in minor deviations in frequency estimation. Similarly, velocity estimation was influenced by fluctuations in frequency, and slight disparities in droplet frequency between adjacent positions could lead to variations in phase difference calculations. Nonetheless, these deviations were within a small range. Overall, these experiments demonstrated that the results of our parameter estimation method were consistent with theoretical expectations.To validate the performance of our parameter estimation method on realworld data, we collected droplet data under different input parameters by adjusting the pressure of the oil and water phases. This included data where the oil phase was fixed while the water phase was continuously adjusted, as well as data where the water phase was fixed while the oil phase was continuously adjusted. The comparison between the droplet parameters estimated by our method and the observed droplet parameters under different input parameters is shown in Figure 4 . It demonstrated that the parameters predicted by our method are consistent with the observed parameters. However, due to the presence of noise, there is slight degree of fluctuation in the predicted results, but they still remain close to the observed values. Comparatively, the calculation of droplet size is influenced by both velocity and time resolution. This results in the size detection resolution being determined by the product of time resolution and velocity. Relative to frequency and velocity, the size detection resolution is larger, making it more robust against noise but also prone to larger detection errors. Furthermore, experimental results reveal relationships between droplet parameters and input parameters. For example, when fixing the water phase (CH2) and adjusting the oil phase (CH1), the droplet velocity gradually decreases  
: 
 the diagnosis of rare cells [2] , early cancer [3] , analysis of enzyme function [4] , the study of phenotypic and genetic diversity at the single-cell leve [5] . Droplet microfluidics [6] , as a commonly used technique for minimizing sample volumes, offers unprecedented throughput (1-10kHz) compared to robotic liquid handling and digital microfluidics [7] . Moreover, it is cost-effective, consumes minimal reagents, and offers enhanced sensitivity [1] due to its large surface-to-volume ratio. Despite these advantages, achieving rapid response and real-time control of droplet microfluidic control systems in high-speed scenarios remains an immensely challenging task [1] . Conventional one-dimensional acquisition methods are often inadequate in capturing comprehensive droplet information, while image-based acquisition methods require a trade-off between data throughput and time-consumption [8] . Therefore, parameter setting for droplet generation control often relies on predictive understanding [9] of multiphase flows or numerical simulations of fluid dynamics [10] . However, these predictions and simulations often involve a degree of uncertainty [9] , leading to an iterative design process. Therefore, the development of a real-time parameter evaluation method is crucial for the design, evaluation, and control of droplet microfluidic platforms.Real-time parameter estimation of microfluidic droplets requires the implementation of a high throughput detection system. These systems may be classified into conventional one-dimensional detection methods [11] [12] [13] and subsequent image-based detection methods [14] [15] [16] [17] . Traditional detection methods. Typically, onedimensional detection methods employ cellular-level optical probes for 'one-by-one' measurements. Sample detection is typically achieved by optical characteristics such as fluorescent light or chromogenic reaction, and is detected using photomultiplier tubes (PMTs) or photodiodes [18] . Although this approach allows for highthroughput and quantitative droplet detection, it often sacrifices spatial resolution in favour of throughput and lacks the capability to record morphological properties of droplets with varying encapsulated contents [1] . Image-based detection methods. Image-based detection techniques can be broadly classified into two primary categories: camera-based methods [16, [19] [20] [21] and scanning-based methods [22] [23] [24] [25] . Camera-based detection systems typically utilize conventional two-dimensional arrays such as CCD or CMOS arrays for data acquisition. These systems are capable of capturing two-dimensional images given a sufficient exposure time. However, the presence of this exposure time poses challenges when applied to high-speed acquisition in droplet microfluidics [18] . Despite attempts to mitigate these challenges by narrowing the field of view or employing spatiotemporal multiplexing strategies [24, 25] , the high-throughput data output often impedes real-time analysis and processing. In contrast, scanning-based methods integrate PMTs with laser spot scanning, followed by post-processing to generate Figure 1 : (a) The spike-based microdroplet acquisition system comprises a microscope with a spike camera and a microfluidic droplet device. The microfluidic device has two-phase inputs: oil and water, which are used to generate droplets by shearing water with oil. (b) The spike camera outputs spike streams, which are then processed in real-time using a spike-based droplet parameter estimation algorithm to extract information regarding droplet frequency, velocity, and size. This information is then fed back to fine-tuning the input parameters.intensity images [8] . In comparison to camera-based methods, scanning methods offer superior bandwidth and lower dark noise. Nevertheless, overall throughput is constrained by scanning speed and achieving real-time analysis and data processing remains a significant challenge [8] . To effectively estimate the real-time parameters of microfluidic droplets, several factors need to be considered: (1) Sampling rate of detection system. In the dynamic environment of droplet microfluidics, evolving high-velocity motion, it is imperative that the data acquisition system swiftly captures the scene information. Comprehensive data collection is a prerequisite for effective and reliable subsequent parameter analyses. (2) Bandwidth of detection system. Efficient use of bandwidth in data collection facilitates real-time analysis and processing, as opposed to the conventional practice of data storage and subsequent processing. (3) Real-time parameter estimation algorithm. Given the collected data, the primary objective is to perform parameter estimation based on the intrinsic characteristics of the data, thus minimizing analysis time and meeting real-time requirements.In recent years, neuromorphic cameras [26] [27] [28] [29] [30] have emerged as a novel type of imaging technology, boasting asynchronous operation at each pixel and outputting data in a discrete format. This unique design enables them to offer remarkable imaging capabilities, particularly excelling in high temporal resolution with relatively small bandwidth. They have shown excellent performance in various high-speed applications. Two common types are event cameras [26, 28, 30] and spike cameras [27, 29] . Event cameras trigger events asynchronously based on set thresholds of light intensity changes, allowing for the capture of dynamic scene information at extremely high temporal resolutions. However, while they excel at capturing dynamic information in scenes, their ability to fully reconstruct scene details remains limited. Although some research endeavours to recover intensity from event streams [31] [32] [33] [34] , the reconstruction performance still exhibits considerable gaps compared to traditional cameras. While event cameras also hold the potential for high-frequency analysis of droplet parameters, in practical applications, the need to obtain high-quality images at any given moment alongside real-time parameter estimation favours spike cameras over event cameras for this task. Spike cameras operate by asynchronously integrating each pixel, firing a spike when a set threshold is reached. They can record rich spatiotemporal information with extremely high temporal resolution and subsequently recover intensity information at any given moment [29] . Compared to other sensors, spike cameras can capture more comprehensive spatiotemporal information in droplet microfluidic scenes with relatively small bandwidth. This facilitates precise estimation of droplet generation parameters in subsequent stages.In this paper, we introduce spike camera to parameter estimation in liquid microfluidies (Figure 1(a) ) for the first time, demonstrating the potential application of spike camera in high-speed microfluidic scenarios. In contrast to the discrete two-dimensional image output of traditional cameras, spike cameras generate continuous spike streams containing rich spatiotemporal information. To fully leverage this advantage, we have developed a spike-based method for real-time estimating droplet parameters (RTED) directly from the characteristics of the spike stream (Figure 1(b) ) . This innovative approach eliminates the need for image reconstruction from spike streams, resulting in outstanding temporal efficiency. When combined with the efficient data acquisition capabilities of the spike camera, real-time estimation and feedback of droplet parameters become feasible. Our method demonstrates high precision through experiments conducted on both simulated and real-world data. Furthermore, comparative experiments with image-based parameter estimation methods highlight the exceptional temporal efficiency of our method, approaching the stringent requirements of real-time computation. In summary, our contributions can be summarized as follows:• We are the first to introduce spike cameras to information acquisition in droplet microfluidics. In contrast to existing methods that require trade-offs between bandwidth and acquisition speed, spike camera enables high temporal resolution spatial information acquisition with relatively low bandwidth. This breakthrough holds promise in overcoming the traditional separation of data acquisition and analysis in droplet microfluidics, achieving real-time estimation and feedback seamlessly integrated into a single system. Ultimately, this will achieve real-time, relatively high-precision parameter estimation of droplet parameters with relatively low bandwidth. • We propose an end-to-end spike-based droplet parameter estimation method, without the need for image reconstruction from spike streams. This method boasts exceptionally high temporal efficiency, capable of meeting real-time processing requirements. Experiments conducted on both synthesized and real-world data consistently demonstrate its superior accuracy. • We collected and open-source the first spike-based droplet datasets (SDD), containing both synthetic and real-world data with varying flow velocities, frequencies, and droplet sizes. 
 
 Appendix:  


Decision: This paper introduces RTDE, a real-time evaluation method for high-speed droplet parameters using a spike camera integrated into a droplet collection system. Leveraging the neuromorphic capabilities of the spike camera, the proposed method captures high temporal resolution scenes with low data throughput. The authors develop a spike-based estimation algorithm to process the spike stream and estimate droplet generation parameters in real-time. Validation of RTDE is conducted on a spike-based droplet dataset (SDD) consisting of synthetic and real data with varying flow velocities, frequencies, and droplet sizes. Experimental results demonstrate that RTDE achieves high precision in parameter estimation and significantly outperforms image-based methods in terms of time efficiency. The RTDE method presents a significant advancement in the field of microfluidics by introducing a novel, efficient, and accurate approach to real-time droplet parameter estimation. The innovative use of spike cameras, coupled with a well-designed spike-based estimation algorithm, addresses a critical challenge in droplet microfluidic systems. The experimental validation is comprehensive, demonstrating the robustness and practical advantages of the method. Despite minor areas for improvement in algorithmic clarity and hardware generalizability, the paper's contributions are substantial and well-aligned with the interests of the ACM MM community. All reviewers give positive reviews, this paper could be accepted.

Reviews: review: paper_summary: paper_summary: The paper presents a novel method for real-time parameter evaluation of high-speed microfluidic droplets using continuous spike streams. The authors introduce spike cameras into droplet microfluidics for the first time, proposing a method named RTDE that integrates spike cameras into the droplet collection system. This method allows for efficient capture of information via spike streams and employs a spike-based estimation algorithm for real-time droplet generation parameters. The performance of RTDE was validated using both synthetic and real-world datasets, demonstrating high precision and superior time efficiency compared to image-based parameter estimation methods. The spike camera's ability to offer high temporal resolution with low data throughput makes it a promising tool for real-time, precise parameter estimation in droplet microfluidics.
strength_weakness: strength_weakness: 1)The paper introduces a pioneering use of spike cameras in microfluidics, showcasing a new direction for real-time parameter estimation.
High Precision: Experiments on both synthetic and real-world data indicate that the proposed method achieves parameter estimations that closely match ground truth values.
2)The spike-based method demonstrates exceptional temporal efficiency, capable of meeting real-time processing requirements.
3)The spike camera's design allows for high temporal resolution with relatively low bandwidth, which is advantageous for real-time applications.
limitations: limitations: 1) Insufficient Justification for Event Cameras' Limitations: The paper argues that spike cameras are preferred over event cameras for high-speed microfluidic droplet parameter estimation due to their ability to capture high-quality images at any given moment alongside real-time parameter estimation. However, the justification provided for event cameras' supposed inadequacies in this context is not sufficiently robust. 

2) Typographical Errors: The manuscript contains typographical errors within the mathematical expressions of Section 4.2, particularly impacting Equations 9 and 13. These mistakes misguide readers who are trying to comprehend the underlying mathematical logic. Incorrect Term Usage in Equation 9: In Equation 9, there is an error where the term \(\hat{y}_1\left(t\right)\) has been mistakenly written as \(\hat{y}_2\left(t\right)\). Equation 13 features a misuse of parentheses that could cause a misinterpretation of the formula.

3) Lack of Consistent Acronym Reference: The manuscript introduces the acronyms M-RTDE and M-SDED only in the conclusion section, with no prior mentions throughout the rest of the paper. This inconsistency in terminology can lead to confusion for readers who may not immediately connect these acronyms to the methods and datasets discussed earlier in the document.
paper_summary: paper_summary: Microfluidic droplet devices have broad applications due to their high throughput and low power consumption. However, the lack of real-time methods for accurately estimating droplet generation parameters has largely kept droplet microfluidic systems under offline control, making effective feedback during droplet generation a challenge. In this paper, the authors integrate a spike camera into the droplet collection system, efficiently capturing information with a spike stream. Based on this, the authors propose a spike-based real-time droplet generation parameter estimation algorithm, achieving real-time estimation of microfluidic droplet parameters.
strength_weakness: strength_weakness: 1. The aim of this paper is to achieve real-time and relatively high-precision parameter estimation for droplets under relatively low bandwidth conditions. Spike cameras, known for their high dynamic resolution, provide an intuitive and effective method for integrating into droplet microfluidic information collection. The experimental results confirm this.
2. For droplet microfluidic information processing under a spike camera, the authors propose an end-to-end spike-based droplet parameter estimation method, which has demonstrated good performance in experiments.
3. The authors have collected and open-sourced the first spike-based droplet dataset (SDD), which I believe contributes to the research community in this field.
limitations: limitations: I see no major flaws in this paper and I am inclined to accept it.

I believe that the method is valid, nevertheless, I am not sure about the extent to which it contributes to the field as a whole, i.e. how widely it is applied and how much it contributes to the field as a whole. I therefore need a reviewer who is more experienced in this area to help me make a judgement. My rating at this stage is ‘Weak Accept’.
paper_summary: paper_summary: The paper introduces a system for real-time parameter estimation of droplet microfluidic control systems in high-speed scenarios. The key innovation of the proposed system is the use of spike cameras, which provide high temporal resolution and dynamic range with a relatively low data rate. To process the spike stream, the authors introduce methods for evaluating droplet generation frequency, velocity, and size. These methods are evaluated using both synthetic and real-world data sets. The results show that the proposed spike-based system parameter estimation is consistent with theoretical expectations and outperforms image-based methods while requiring much less data.
strength_weakness: strength_weakness: - The paper introduces the first spike-based system for real-time parameter estimation of droplet microfluidic control systems. It provides accurate, real-time parameter estimating while requiring significantly less data than state-of-the-art solution.
- The proposed system is evaluated using both synthetic and real-world datasets.
limitations: limitations: -The methods in Section 4 are introduced without clarifying whether they address novel research challenges or are simply a straightforward application of known results.
paper_summary: paper_summary: This paper proposes real-time droplet parameter evaluation using spike camera for microfluidics and Introduces M-RTDE method for real-time droplet parameter estimation.
strength_weakness: strength_weakness: The author claims that (if true) it is the  first to introduce spike cameras to information acquisition in droplet microfluidics.
The proposed spike-based droplet parameter estimation method is interesting.
The spike-based droplet datasets are a significant contribution to the community.
The improvements of experimental results compared to other methods are superior.
limitations: limitations: The paper lacks an introduction to the comparative methods, leading to reader confusion. It's unclear what "Image-based" refers to and whether there are additional methods for comparison. The experiments are also incomplete, needing more datasets and lacking ablation studies.

Can the proposed method balance the advantages of previous imaged-based methods?