# Bio-Inspired Microsystem for Robust Genetic  Assay Recognition

Authors: Jaw-Chyng Lue; Wai-Chi Fang
Journal: Journal of Biomedicine and Biotechnology
 (2008)

---

## Abstract

A compact integrated system-on-chip (SoC) architecture solution for robust, real-time, and on-site genetic analysis has been proposed. This microsystem solution is noise-tolerable and suitable for analyzing the weak fluorescence patterns from a PCR prepared dual-labeled DNA microchip assay. In the architecture, a preceding VLSI differential logarithm microchip is designed for effectively computing the logarithm of the normalized input fluorescence signals. A posterior VLSI artificial neural network (ANN) processor chip is used for analyzing the processed signals from the differential logarithm stage. A single-channel logarithmic circuit was fabricated and characterized. A prototype ANN chip with unsupervised winner-take-all (WTA) function was designed, fabricated, and tested. An ANN learning algorithm using a novel sigmoid-logarithmic transfer function based on the supervised backpropagation (BP) algorithm is proposed for robustly recognizing low-intensity patterns. Our results show that the trained new ANN can recognize low-fluorescence patterns better than an ANN using the conventional sigmoid function.


---

## Body

## 1. Introduction

The development of low-cost portable instruments for rapidly analyzing
genetic assays would significantly  advance the level of medical services
globally. The polymerase chain reaction (PCR) amplification and the capillary electrophoretic (CE)
techniques are often adopted for genetic analysis. A complex system that can
process full PCR amplification and data analysis tasks usually involves
integration of control, optical, thermal, fluid channel, and data acquisition
systems. For example, a portable system
providing full PCR-CE functions was developed earlier for genetic analysis
[1]. The system demonstrated the
feasibility of on-site genetic analysis. However, the expense to build such a system is
considered relatively high. The entire integrated system consists of multiple
PCR chambers, heaters, sensors, solid-state laser excitation light source,
fluorescence detection optics, electronics, CE separation microchannels, and
power supplies. The data was collected
and processed in a portable computer.
Recently, a real low-cost (~10US$) pocket-sized PCR thermocycling
device has been developed based on a smart technique of simultaneously
pseudoisothermally heating multiple zones of a loop channel for PCR
amplification [2]. This thermocycler
does not contain the CE separation, the fluorescence detection, and the data
analysis functions. Theoretically,
multiple PCR amplification results can be rapidly generated in parallel and
displayed simultaneously by using multiple of these low-cost devices.
Therefore, patterns of an array of the PCR resulted samples can potentially be
generated similar to the genetic assay patterns on a microchip. Moreover, the integration of PCR and
electrochemical (EC) transduction functionality on microfabricated
silicon/glass-based devices for DNA amplification and detection was shown successfully [3].
Their microfabricated device needs to operate with external control and data-acquisition
systems.Most of the research efforts for PCR analysis tools were focusing on the
development of the PCR microdevices, the associated thermal systems, the
optical systems, and the data analysis software tools. However, to our best
knowledge, the data acquisition and analysis system for examining PCR samples
or assays is usually a computer equipped with specific PCR analysis software
but not a compact hardware solution.Regarding the goal of building a real compact PCR analysis system that
can rapidly find and analyze the desired genetic patterns, the existing data
acquisition and analysis systems (e.g.,
portable computers and interfaces) are considered relatively large in size and
heavy in weight. In addition, human inspectors cannot recognize the genetic
assay patterns as easily as written characters with explicit meanings. Manual massive PCR data analysis can be very
time consuming. Therefore, people involved in “the human genome project” have
used perceptron-like neural networks for helping to recognize the DNA fragments
with specific functions [4]. For
gene-recognition purpose, a perceptron was first trained by using the datasets
consisting of nucleotide sequences of known functional sites (e.g., transcription initiation sites
(promoters), transcription termination sites (terminators), or splice-junction
sites). Patterns of fragments of the entire DNA sequence were then fed to the
perceptron nucleotide by nucleotide to check if any site of interest appears at
a particular position in the fragment. In the protein coding region recognition
task by Guan et al. [5], their multisensor /neural network successfully
identified 96% of the 17,576 sequence positions as coming from coding or
noncoding regions. In the splice junction recognition experiment, the resulting
recognition rate was 99% for acceptor junctions and 96% for donor junctions.Nonetheless, if the genetic analysis task needs to be conducted on a
hazardous or dangerous field (e.g.,
potentially disease-contagious environment), a compact, autonomous, and even
disposable PCR data analysis system would be preferred. Therefore, by taking advantages of the VLSI
microfabrication technologies and artificial neural network theories, we
proposed a microsystem consisting of a unique optical configuration setup, a differential
logarithm sensor-processor array chip, and an ANN SoC processor chip for fast
recognizing and analyzing the PCR prepared genetic patterns.In typical PCR amplification procedure, a dual-labeled (i.e., for sample and reference channels)
assay design is commonly used for identifying differentially expressed genes. This method also reduces the sources of
variability/noise due to aspects of individual spot that affects both specimens
similarly [6]. In order to accurately
calculate the density of the sample DNA material in a particular dot/well after
the PCR amplification, the integral of the total fluorescence intensity (presumably
representing the density of the DNA materials inside the dot/well) from the topological
profile of the dot/well is usually computed.
The logarithmic value of the ratio of the two intensities of the
fluorescent-dye-labeled specimens (one for the sample specimen, the other for
the reference specimen) measured from the same dot/well is calculated based on
the fluorescence assay image. The ratio of the two intensities would provide
the normalized population of the genetic material in the dot/well disregarding
the initial population density before PCR amplification. The logarithmic
operation would amplify the small signals. In most of the commercial available
solutions, the fluorescence assay image is usually scanned by a color scanner
with high resolution and then transferred to a computer for image analysis. The
profile analysis software usually computes the normalized intensity of dot/well
after dot/well sequentially.The intensity of fluorescence light is usually relatively low. Using higher excitation light intensity can
lead to brighter fluorescence patterns. Increasing
the integral of detection time can enhance the received fluorescence patterns. However,
lower power consumption and faster detection are preferred. Furthermore, some fixed-pattern noises in the
input pattern may exist (e.g., fixed
pattern noises created by scattered lights, nonuniformity of the responsivity
of the detector array). These noises may introduce errors to the measurement of
the density of the DNA materials.In order to fast parallel-process the data and resolve the ambiguity
induced by the noises in the data analysis task, a trained artificial neural
network is considered a solution. The parallel processing capability comes from
the nature of the ANN’s multiple input architecture [7–9]. In contrast
with the conventional sequential data analysis methods, the data analysis
throughput would be increased linearly as the number of dot/well increases. Regarding
potential noises, the ANN will automatically take the noises into calculation
in the latter recognition phase because the ANN can learn from the training
patterns that contain the fixed pattern noises in the learning phase. In
addition, because of the natural capability of associating noisy input patterns
with output index of a trained ANN, noises introduced by other factors will not
significantly affect the ANN’s recognition capability. In conjunction with the
natural capabilities of an ANN listed above, a signal amplification stage that
can augment the low-fluorescence input before the ANN stage would help the ANN to
acquire data more reliably, and thus result in a more robust data analysis
capability of the entire system.


## 2. Biochip Module Architecture

We proposed a hardware microsystem that is suitable for real-time,
on-site, robust genetic fluorescence data analysis (Figure1). This envisioned biochip module architecture consists
of an on-chip assay with an array of clusters of dual-labeled genetic
dots/wells, a dual-color beams module, an imaging lens, a bioimaging optoelectronic
microchip with coated color filters (Figure 2),
a parallel analog data-transfer bus (optional depending on the implementation
method), and an artificial neural network (ANN) module for image analysis.Figure 1
(a) Hierarchical diagram of the proposed biochip microsystem for genetic assay recognition. (b) Schematics of a three-layered ANN and the preceding differential logarithm stage. The system of dual-labeled gene assay, dual-color beam module, and imaging lens is not shown in this schematic diagram. The thin-film color filters coated on top of the sample and reference channels are represented by the red and green boxes.Figure 2
(right) Proposed layout of a 15-by-15 unit cell array of the differential logarithm circuitry (2.2 mm by 2.2 mm), (top left) an enlarged view of the layout of a single cell with pseudo thin-film monochromatic filter layers, and (bottom left) the schematic diagram of a single unit cell.The operational function of each
module is explained below along the optical and electrical signal pathways. The
dual-labeled genetic dots/wells are simultaneously excited by two monocromatic
excitation beams (e.g., 532 nm with a
bandwidth of 10 nm from a green diode laser pointer source for the cyanine Cy3 dye, and 635 nm with a bandwidth
of 10 nm from a red solid-state diode laser source for the cyanine Cy5 dye) according to the receiving
bandwidths of the sample and the reference channels. The assay can be either
front-side or backside illuminated as long as a clear fluorescence image of the
dot/well array is generated. Two fluorescence patterns with different peak
wavelengths are produced (e.g., peak
value at 570 nm from the Cy3 dye and peak value at 670 nm from the Cy5 dye, the
two spectral profiles are highly distinguishable), and imaged onto the
bioimaging chip through an imaging lens. Each unit on the bioimaging chip
contains two sensor channels. One sensor is coated with a thin-film microfilter
for wavelength A (e.g., 580 nm with a
narrow transmission bandwidth of approximately 40 nm of a deposited thin-film filter
[10]). The other sensor is coated with a thin-film micro-filter for wavelength
B (e.g., 675 nm with a transmission
bandwidth of 40 nm). The two optical signals are received simultaneously and
processed separately through the posterior electrical circuits. A bioimaging
chip made of an array of differential logarithm circuitry was designed (Figure 2 (bottom left)). Two
analog photoelectric voltage signals produced by the separated logarithmic
amplifier circuit channels are fed continuously into the differential pair
circuit. The difference of the two input
voltages is then represented by the output voltage from the differential pair
circuit. The calculation of the logarithm of the ratio of the sample to the reference
fluorescence intensities is effectively accomplished in this chip. The analog
output from each unit in the bioimaging chip is then sent to the posterior
artificial neural network module through the parallel data bus.The ANN stage is responsible for filtering and recognizing the desired assay
cluster patterns. Fixed pattern noises and noises caused by the nonlinear
circuitry are expected to be accommodated after the ANN is trained. Either unsupervised or supervised learning
algorithm can be adopted to train the ANN.
The ANN in the biochip module architecture can be implemented by either
hardware or software. In this work, we provide a hardware implementation (i.e., a weight-reconfigurable winner-take-all
ANN chip suitable for the Kohonen self-organized filter algorithm [7]) for the
unsupervised version, and a computer-simulated ANN (i.e., a feedforward ANN using the back-propagation (BP) learning algorithm
[8, 9]) for the supervised version. In
the hardware implementation, the weight values are reconfigurable and stored in
memory devices. By adopting a massively paralleled neural computing paradigm
and a mixed-signal deep submicro fabrication technology, the ANN can be
implemented on a single VLSI chip. A row/column parallel data flow architecture
is used to connect all on-chip systems, and to reduce data bandwidth
limitations due to conventional data bus architectures.Because the weak fluorescence signals are enhanced by the imaging chip
and automatically analyzed by the noise-tolerable neural network module, the entire
architecture system is expected to robustly conduct the recognition task.


## 3. Hardware Module Implementation

### 3.1. Bioimaging Chip and Its Nonlinear Circuitry

The proposed bioimaging chip consists of an array
of differential logarithm processor unit and row/column readout circuit. A prototype
layout of an array of15×15
differential logarithm unit is shown in Figure 2.
Each unit contains two size-matched
logarithmic amplifier circuits and a differential pair circuit. As described in the previous section, each
unit produces an analog voltage output to represent the difference between the
logarithms of the sample (experimental) input and the reference (control)
input.The key logarithmic amplifier circuit is
designed after the works of Chamberlain and Lee [11] and Mead [12], but with an additional n-well/p-sub junction
layer for effectively isolating cross-talk noises among the logarithmic
amplifier processor unit array.Chamberlain and Lee first adopted the intrinsic vertical n-p junction (n+-diffusion/p-sub) under the source terminal of an
NPN transistor to function as a reverse-biased photodiode. A wide dynamic range
silicon photodetector can be implemented [11]. Mead has a similar design by
using a vertical parasitic bipolar transistor for photosensing part [12]. This
vertical bipolar is a natural byproduct structure in the CMOS process. The p+-diffusion/n-well/p-sub structure is the PNP bipolar transistor existing in any PMOS
transistor in an isolated section of n-well
region. This design is the fundamental
building block of his silicon retina.The optoelectronic logarithmic amplifier
circuit for each channel in this work was fabricated by using the MOSIS AMI 1.5-μm
5-V ABN BiCMOS n-well process as
shown in Figure 3 [13]. The photodetector is made of a vertical n-well/p-base/n+-emission
bipolar detector. Two diode-connected NMOS transistors are connected in series
with the bipolar detector. The detecting area collects the fluorescence inputs.
Mainly due to the diode configured NMOS
transistors, the V-I characteristic of this normally “OFF” and low-power
circuit behaves logarithmically while operating in the subthreshold region, and
similar to a square root curve (~ IDS) while operating in the saturation region. This
logarithmic amplifier circuit can detect light intensity as low as
approximately 10 nW and consumes energy from 100 nW to 2 μW (VDD:
5 Volts) depending on the incident intensity and wavelength.Figure 3
Output voltage of a single channel of saturated logarithmic circuit as function of the input optical power (input wavelength: 830 nm). An optical micrograph of the single-channel logarithmic amplifier circuit and its correspondent schematic circuit diagram are shown.


### 3.2. Weight-Reconfigurable Artificial Neural  Processor Chip

The SoC architecture design of the weight-reconfigurable ANN processor
consists of an input neurons array, a programmable synapse weight matrix, an
array of output neurons, a winner-take-all module, and a membership encoder [14, 15].
The input neurons array has M input
neurons that are used to buffer the input vector. Each input vector has M analog components (generating from the
preceding bioimaging chip). The
programmable synapse weight matrix is composed of M×N synapse cells for
the NM-dimensional codevectors. The output neuron array is composed of N summing neurons with selectable
sigmoid or sigmoid-logarithmic transfer functions. The winner-take-all module consists of N competitive circuit cells, which
perform parallel comparison among N inverted distortion values and choose a single winner. The membership encoder
circuit is an N-to-n decoder that uses binary codes to
encode N classes.The ANN processor works as a learning accelerator in the learning phase
at a time complexityO(1) for
processing each learning iteration. Its programmable weight matrix can be
either generated by using the on-chip self-organization learning procedure or
be uploaded by the BP training subsystem [15]. The ANN processor also realizes
a full-search vector quantization process for each input vector at a time
complexity O(1) in the recognition
phase.This ANN processor can also support the
multiple winner-take-all scenario (e.g.,
more than one classes
that the input assay pattern may belong to, or multiple desired patterns that
the input assay pattern are similar to). After a winning pattern (the most
likelihood) was picked out from theN prestored classes/codevectors in the recognition task according to a particular
analog input vector, the associated circuitry of this winning pattern will be
disabled in the next recognition iteration. Therefore, a second-winner pattern
(the second likelihood) can be chosen later according to the same input vector.
By repeating the procedure stated above, multiple-winner patterns could be
chosen for the current input pattern eventually.The ANN chip can learn unsupervised if the selforganization learning
procedure is adopted. In this case, the ANN chip can perform on-chip learning
in the learning phase. For the supervised learning version (e.g., back-propagation algorithm or its
variations), the weight update procedure usually involves complex computations
that require further signal processing circuits in order to achieve the on-chip
learning purpose. Further real estates on chip are then required to accommodate
the circuits.A prototype ANN SoC chip using a scalable 2-μm 5-V CMOS
technology was designed, fabricated, and tested. Its chip layout and design
features are shown in Figures 4 and 5, respectively. This prototype chip includes 25 input
neurons, 25×64 weight cells, 64
output summing neurons, 64 winner-take-all cells, and a 64-to-8 membership
encoder. The estimated power dissipation is 50 mW at 10 MHz. Its equivalent computation power is about 16
giga-operations per second.Figure 4
The optical micrograph of the prototype ANN chip that is wire-bonded to a ceramic package. The silicon chip die size is4.6mm×6.8mm.Figure 5
A system-on-chip architecture design for the winner-take-all selforganization artificial neural network chip.An engineering version of the ANN SoC SiP (silicon intellectual property)
has been under development to enable the proposed miniaturized PCR
system-on-chip design using the TSMC 130-nm 1.2-V CMOS technology. The scalable
ANN prototype chip can be converted into a design containing 100 input neurons,100×256 weight cells, 256 output summing neurons,
256 WTA cells, and a 256-to-8 membership decoder. The envisioned chip size is approximately 1.2mm×2mm. Its estimated power dissipation is about 120 mW at a 100D vector throughput
rate of 100 MHz. Its equivalent computation power is about 2.5 tera-operations
per second. Because of this ANN SoC SiP design, the feasibility of the proposed
low-power, real-time, and on-site PCR assay analysis on an integrated
microsystem becomes promising. The proposed microsystem would be useful
especially in a scenario of finding desired or suspicious biopatterns in a
massive amount of data.


## 3.1. Bioimaging Chip and Its Nonlinear Circuitry

The proposed bioimaging chip consists of an array
of differential logarithm processor unit and row/column readout circuit. A prototype
layout of an array of15×15
differential logarithm unit is shown in Figure 2.
Each unit contains two size-matched
logarithmic amplifier circuits and a differential pair circuit. As described in the previous section, each
unit produces an analog voltage output to represent the difference between the
logarithms of the sample (experimental) input and the reference (control)
input.The key logarithmic amplifier circuit is
designed after the works of Chamberlain and Lee [11] and Mead [12], but with an additional n-well/p-sub junction
layer for effectively isolating cross-talk noises among the logarithmic
amplifier processor unit array.Chamberlain and Lee first adopted the intrinsic vertical n-p junction (n+-diffusion/p-sub) under the source terminal of an
NPN transistor to function as a reverse-biased photodiode. A wide dynamic range
silicon photodetector can be implemented [11]. Mead has a similar design by
using a vertical parasitic bipolar transistor for photosensing part [12]. This
vertical bipolar is a natural byproduct structure in the CMOS process. The p+-diffusion/n-well/p-sub structure is the PNP bipolar transistor existing in any PMOS
transistor in an isolated section of n-well
region. This design is the fundamental
building block of his silicon retina.The optoelectronic logarithmic amplifier
circuit for each channel in this work was fabricated by using the MOSIS AMI 1.5-μm
5-V ABN BiCMOS n-well process as
shown in Figure 3 [13]. The photodetector is made of a vertical n-well/p-base/n+-emission
bipolar detector. Two diode-connected NMOS transistors are connected in series
with the bipolar detector. The detecting area collects the fluorescence inputs.
Mainly due to the diode configured NMOS
transistors, the V-I characteristic of this normally “OFF” and low-power
circuit behaves logarithmically while operating in the subthreshold region, and
similar to a square root curve (~ IDS) while operating in the saturation region. This
logarithmic amplifier circuit can detect light intensity as low as
approximately 10 nW and consumes energy from 100 nW to 2 μW (VDD:
5 Volts) depending on the incident intensity and wavelength.Figure 3
Output voltage of a single channel of saturated logarithmic circuit as function of the input optical power (input wavelength: 830 nm). An optical micrograph of the single-channel logarithmic amplifier circuit and its correspondent schematic circuit diagram are shown.


## 3.2. Weight-Reconfigurable Artificial Neural  Processor Chip

The SoC architecture design of the weight-reconfigurable ANN processor
consists of an input neurons array, a programmable synapse weight matrix, an
array of output neurons, a winner-take-all module, and a membership encoder [14, 15].
The input neurons array has M input
neurons that are used to buffer the input vector. Each input vector has M analog components (generating from the
preceding bioimaging chip). The
programmable synapse weight matrix is composed of M×N synapse cells for
the NM-dimensional codevectors. The output neuron array is composed of N summing neurons with selectable
sigmoid or sigmoid-logarithmic transfer functions. The winner-take-all module consists of N competitive circuit cells, which
perform parallel comparison among N inverted distortion values and choose a single winner. The membership encoder
circuit is an N-to-n decoder that uses binary codes to
encode N classes.The ANN processor works as a learning accelerator in the learning phase
at a time complexityO(1) for
processing each learning iteration. Its programmable weight matrix can be
either generated by using the on-chip self-organization learning procedure or
be uploaded by the BP training subsystem [15]. The ANN processor also realizes
a full-search vector quantization process for each input vector at a time
complexity O(1) in the recognition
phase.This ANN processor can also support the
multiple winner-take-all scenario (e.g.,
more than one classes
that the input assay pattern may belong to, or multiple desired patterns that
the input assay pattern are similar to). After a winning pattern (the most
likelihood) was picked out from theN prestored classes/codevectors in the recognition task according to a particular
analog input vector, the associated circuitry of this winning pattern will be
disabled in the next recognition iteration. Therefore, a second-winner pattern
(the second likelihood) can be chosen later according to the same input vector.
By repeating the procedure stated above, multiple-winner patterns could be
chosen for the current input pattern eventually.The ANN chip can learn unsupervised if the selforganization learning
procedure is adopted. In this case, the ANN chip can perform on-chip learning
in the learning phase. For the supervised learning version (e.g., back-propagation algorithm or its
variations), the weight update procedure usually involves complex computations
that require further signal processing circuits in order to achieve the on-chip
learning purpose. Further real estates on chip are then required to accommodate
the circuits.A prototype ANN SoC chip using a scalable 2-μm 5-V CMOS
technology was designed, fabricated, and tested. Its chip layout and design
features are shown in Figures 4 and 5, respectively. This prototype chip includes 25 input
neurons, 25×64 weight cells, 64
output summing neurons, 64 winner-take-all cells, and a 64-to-8 membership
encoder. The estimated power dissipation is 50 mW at 10 MHz. Its equivalent computation power is about 16
giga-operations per second.Figure 4
The optical micrograph of the prototype ANN chip that is wire-bonded to a ceramic package. The silicon chip die size is4.6mm×6.8mm.Figure 5
A system-on-chip architecture design for the winner-take-all selforganization artificial neural network chip.An engineering version of the ANN SoC SiP (silicon intellectual property)
has been under development to enable the proposed miniaturized PCR
system-on-chip design using the TSMC 130-nm 1.2-V CMOS technology. The scalable
ANN prototype chip can be converted into a design containing 100 input neurons,100×256 weight cells, 256 output summing neurons,
256 WTA cells, and a 256-to-8 membership decoder. The envisioned chip size is approximately 1.2mm×2mm. Its estimated power dissipation is about 120 mW at a 100D vector throughput
rate of 100 MHz. Its equivalent computation power is about 2.5 tera-operations
per second. Because of this ANN SoC SiP design, the feasibility of the proposed
low-power, real-time, and on-site PCR assay analysis on an integrated
microsystem becomes promising. The proposed microsystem would be useful
especially in a scenario of finding desired or suspicious biopatterns in a
massive amount of data.


## 4. Simulations and Empirical Results

### 4.1. Numerical Simulations of Biosignature and  Optical Character
Recognition

In this
section, two pattern-recognition tasks were computer simulated to demonstrate
the feasibility of using an ANN for our proposed biochip module architecture. A
novel sigmoid-logarithmic function is also integrated within the learning
algorithm (i.e., back-propagation
algorithm) to demonstrate the capability of recognizing relatively dim
patterns. The study in this section will
assist our future circuit design and may contribute to the new techniques for
medical image processing.In most
of the fluorescence spectroscopy applications, the fluorescence patterns
usually have relatively low intensities and are difficult to analyze. We know
that high-excitation intensities and long exposure time can lead to stronger
fluorescence signals. However, low-energy consumption and fast detection are the
design goals for our biochip module architecture. Therefore, if the posterior
ANN of our biochip architecture can analyze dim fluorescence patterns better,
we can potentially use relatively lower energy and shorter time to conduct the
analysis task.Regarding
the neural network learning algorithm, the simplest transfer function that we
can use in the algorithm is a linear ramp function (e.g., linear slope between 1 and −1, flat and continuous outside [1, −1]).
However, higher recognition capability can be achieved by using
nonlinear transfer function in the neural network learning algorithm.The
nonlinear sigmoid (logistic) transfer function is usually adopted in artificial
neural network models because its derivative can be easily obtained
algebraically. For example, we defineA(h) as a sigmoid function:(1)A(h)=11+exp(−h).The first derivative ofA(h) can easily be calculated by using the
identity A′(h)=A(A−1). Therefore, computation complexity and cost
of hardware or software can be reduced.In addition, a single-layer feedforward
network (SLFN) with any bounded continuous nonconstant activation (transfer) function
or arbitrarily bounded activation (transfer) function with unequal limits at the
infinities can form decision regions with arbitrary shapes [16–18]. A
multilayer perceptron architecture naturally consists an SLFN in its structure.
As long as a function has unequal upper bond, lower bond, and monotonic
behavior, it can be used as a transfer function.For the above computational advantage and
theoretical reasons, we proposed a novel piecewise sigmoid-logarithmic function
that also yields similar mathematical identities and computational benefits:(2)A(h)={11+exp(−h),h<−2,−αln(β(δ−h)),−2≤h<0,αln(β(h+δ)),0≤h<2,11+exp(−h),2≤h.In this
piecewise function,α=0.050095635,β=1000,δ=0.01, and h is the net weighted input to the transfer function A(h).
The central part (net input ranging from −2 to 2) of the original sigmoid function was
replaced by two asymmetric pieces of logarithmic curves.To demonstrate
the capability of recognizing dim patterns by using a feedforward ANN with
sigmoid-logarithmic transfer function, a simple pseudo genetic assay analysis
task and an optical character pattern-recognition task were simulated. MATLAB programs were created to train an ANN
and examine its performance.A
100-100-2 (100 inputs, 100 hidden neurons, and 2 output neurons) artificial
feedforward neural network was chosen to perform both recognition tasks. For the
biosignature recognition, 20 patterns/clusters on a microchip genetic assay
were prepared (Figure6).
For simplicity, we used the pixellated image of a fluorescence image of the
sample material to represent a normalized (i.e., after taking
logarithmic value of the ratio of the sample to reference signals) but noisy
image for the analog input to the ANN. Additional seven datasets by rescaling
the gray level of the original dataset with different rescaling factors were
also prepared (factors: 1/3.16, 1/10, 1/31.6, 1/100, 1/316, 1/1000, 1/10000 of
the original gray level). Noticing that,
both brightness and contrast levels of these new biopatterns were reduced by
the rescaling factor. The three desired biopatterns (solid framed in Figure 6 that were randomly
picked) are what we were searching in a scenario of finding the designed PCR
assay cluster pattern of the subject with certain disease.(a) Fluorescence image of a sampled microarray of cDNA, Cy3 dye, and Cy5 dye mix (only Cy5 red fluorescence is shown) [19]. Each cluster consists of a 10×10 grid of sample dots. Each dot corresponds to the location of a cDNA probe to which mRNA from the cells of interest have been bound. (b) Pixellated images of the clusters from the left microarray photo. Each cluster consists of a 10×10 pixel array that mimics a normalized biosignature pattern. The first three randomly picked desired patterns (enclosed in the solid line frame) have indices (0, 0), (0, 1), and (1, 0), accordingly. The rest unwanted patterns share index (1, 1).
(a)(b)Similarly, in the optical character-recognition
task, a dataset containing 100 different alphanumeric letters 1, 2, 3, and 4
was prepared first (as shown in Figure7).
By using the same rescaling factors (as listed in the biopattern
recognition), we generated the other seven datasets that contain dimmer hand script
patterns.Figure 7
The picture of 100 different patterns of alphanumeric letters 1, 2, 3, and 4 used in the optical character recognition experiment.In both
experiments, the digitized biopattern and character datasets were used for both
training and testing the artificial neural network. In contrast to the
traditional method of preparing independent training and test datasets, the
test datasets were assigned to be identical to the training datasets in order
to examine the feasibility of the proposed ANN model with the sigmoid-logarithmic
function.For simplicity, the intensities of the high-resolution
pixels of each original fluorescence dot in Figure6(a) were averaged and replaced by a single super pixel
in the pixellated image in Figure 6(a). If the patterns with the lowest resolution (one pixel for one dot/well)
can be correctly recognized by the ANN, the patterns with higher resolution
should be recognized by the ANN with higher recognition accuracy. To economically
implement the bioimaging chip, one neuron unit would be sufficient for
receiving all the lights emitting from the fluorescence profile of the imaging dot/well. The results of this simulated ANN model would
assist the future design of the proposed architecture. The average of the
intensities of the pixels of the original fluorescence patterns is considered simple
yet physically reasonable.The
back-propagation training using sigmoid-logarithmic transfer function and
gradient descent method was conducted to find a convergent weight configuration
(with fixed learning rateη=0.03). A criterion is employed to count the
percentage of training data that has been learned with an error less than 20%. In
order to guide the weight configuration closer to a convergence condition, the
BP training using regular sigmoid function was conducted first. The BP training
using sigmoid-logarithmic function was conducted afterwards.The entire procedure of the BP training
algorithm using sigmoid-logarithmic transfer function is described as follows.(i)
Prepare the input patterns for the
feedforward multilayer perceptron (MLP) neural network.(ii)
Assign the target values for the
associated input patterns.(iii)
Use the input patterns to train
the multilayer perceptron with sigmoid transfer function until the criterion
value becomes close to one. Now the weight configuration is closer to a
convergence condition for latter training.(iv)
Use the weight values obtained in
the previous step as the initial weight condition for training the multilayer
perceptron with the logarithmic-sigmoid transfer function. The regular BP algorithm using the gradient
descent method is again adopted. After
the criterion becomes one, the training is finished.(v)
Use this trained MLP with
logarithmic-sigmoid transfer function to recognize the test data set. Examine
the recognition accuracy.The detailed
conditions and pseudocodes of the BP algorithm with sigmoid and
logarithmic-sigmoid transfer function are provided as the following.The initial weight values for the first
weight matrixW and second weight matrix V were given randomly. The range of
these random weight values was set between −1 and 1. However, the range of the
trained weight values was unlimited. The
input vector X was the vectorized pixellated pattern. The associated target
values D were given. The learning rate (coefficient in front of the gradient
partial derivative) is parameter η.
The number of iterations is parameter iter. The training data set was used for testing as well to verify
the feasibility of this proposed algorithm.The
pseudocode for the regular back-propagation algorithm using sigmoid transfer
function is listed in
Algorithm1.The pseudocode for
the back-propagation algorithm using the piecewise sigmoid-logarithmic transfer
function is listed in
Algorithm2.


### 4.2. Simulations Results

The
result of recognizing all of the normalized genetic assay datasets by the
trained 100-input-100-hidden-neuron-2-output-neuron network is shown in Table1 (BIO). The network using new
transfer function can still find one desired biosignature in the dataset with
factor 1/100 of the original gray level. However, the network using
conventional sigmoid transfer functions cannot distinguish biopatterns well in
the datasets with factor 1/3.16 of the original gray level or below. Consistent recognition accuracy was obtained
for the optical character recognition (OCR) tasks (Table 1 (OCR)). The network using new transfer function can
recognize 32 characters in the dataset with factor 1/100 of the original gray
level but not the network with regular sigmoid function. The recognition
accuracies for datasets with a factor below 1/316 become constant because the
network tends to recognize one category perfectly according to the converged
weight configuration in this particular experiment.Table 1
Biosignature
and OCR recognition results (Unit: counts of patterns correctly recognized in
one test dataset. OCR: each test
dataset contains 100 characters. BIO: biosignature recognition task,
each test dataset contains 20 patterns.)
Gray
level to original dataTransfer function (learning rate)Hybrid
sigmoid-logarithmic1/(1+exp(−h))1/(1+exp(−5h))(η=0.03)(η=0.03)(η=0.0018)BIOOCRBIOOCRBIOOCR1 (original)2064207920631/3.16205864813651/1020581251371/31.620471251251/1001932———251/3161726————1/10001725————1/100001725————


## 4.1. Numerical Simulations of Biosignature and  Optical Character
Recognition

In this
section, two pattern-recognition tasks were computer simulated to demonstrate
the feasibility of using an ANN for our proposed biochip module architecture. A
novel sigmoid-logarithmic function is also integrated within the learning
algorithm (i.e., back-propagation
algorithm) to demonstrate the capability of recognizing relatively dim
patterns. The study in this section will
assist our future circuit design and may contribute to the new techniques for
medical image processing.In most
of the fluorescence spectroscopy applications, the fluorescence patterns
usually have relatively low intensities and are difficult to analyze. We know
that high-excitation intensities and long exposure time can lead to stronger
fluorescence signals. However, low-energy consumption and fast detection are the
design goals for our biochip module architecture. Therefore, if the posterior
ANN of our biochip architecture can analyze dim fluorescence patterns better,
we can potentially use relatively lower energy and shorter time to conduct the
analysis task.Regarding
the neural network learning algorithm, the simplest transfer function that we
can use in the algorithm is a linear ramp function (e.g., linear slope between 1 and −1, flat and continuous outside [1, −1]).
However, higher recognition capability can be achieved by using
nonlinear transfer function in the neural network learning algorithm.The
nonlinear sigmoid (logistic) transfer function is usually adopted in artificial
neural network models because its derivative can be easily obtained
algebraically. For example, we defineA(h) as a sigmoid function:(1)A(h)=11+exp(−h).The first derivative ofA(h) can easily be calculated by using the
identity A′(h)=A(A−1). Therefore, computation complexity and cost
of hardware or software can be reduced.In addition, a single-layer feedforward
network (SLFN) with any bounded continuous nonconstant activation (transfer) function
or arbitrarily bounded activation (transfer) function with unequal limits at the
infinities can form decision regions with arbitrary shapes [16–18]. A
multilayer perceptron architecture naturally consists an SLFN in its structure.
As long as a function has unequal upper bond, lower bond, and monotonic
behavior, it can be used as a transfer function.For the above computational advantage and
theoretical reasons, we proposed a novel piecewise sigmoid-logarithmic function
that also yields similar mathematical identities and computational benefits:(2)A(h)={11+exp(−h),h<−2,−αln(β(δ−h)),−2≤h<0,αln(β(h+δ)),0≤h<2,11+exp(−h),2≤h.In this
piecewise function,α=0.050095635,β=1000,δ=0.01, and h is the net weighted input to the transfer function A(h).
The central part (net input ranging from −2 to 2) of the original sigmoid function was
replaced by two asymmetric pieces of logarithmic curves.To demonstrate
the capability of recognizing dim patterns by using a feedforward ANN with
sigmoid-logarithmic transfer function, a simple pseudo genetic assay analysis
task and an optical character pattern-recognition task were simulated. MATLAB programs were created to train an ANN
and examine its performance.A
100-100-2 (100 inputs, 100 hidden neurons, and 2 output neurons) artificial
feedforward neural network was chosen to perform both recognition tasks. For the
biosignature recognition, 20 patterns/clusters on a microchip genetic assay
were prepared (Figure6).
For simplicity, we used the pixellated image of a fluorescence image of the
sample material to represent a normalized (i.e., after taking
logarithmic value of the ratio of the sample to reference signals) but noisy
image for the analog input to the ANN. Additional seven datasets by rescaling
the gray level of the original dataset with different rescaling factors were
also prepared (factors: 1/3.16, 1/10, 1/31.6, 1/100, 1/316, 1/1000, 1/10000 of
the original gray level). Noticing that,
both brightness and contrast levels of these new biopatterns were reduced by
the rescaling factor. The three desired biopatterns (solid framed in Figure 6 that were randomly
picked) are what we were searching in a scenario of finding the designed PCR
assay cluster pattern of the subject with certain disease.(a) Fluorescence image of a sampled microarray of cDNA, Cy3 dye, and Cy5 dye mix (only Cy5 red fluorescence is shown) [19]. Each cluster consists of a 10×10 grid of sample dots. Each dot corresponds to the location of a cDNA probe to which mRNA from the cells of interest have been bound. (b) Pixellated images of the clusters from the left microarray photo. Each cluster consists of a 10×10 pixel array that mimics a normalized biosignature pattern. The first three randomly picked desired patterns (enclosed in the solid line frame) have indices (0, 0), (0, 1), and (1, 0), accordingly. The rest unwanted patterns share index (1, 1).
(a)(b)Similarly, in the optical character-recognition
task, a dataset containing 100 different alphanumeric letters 1, 2, 3, and 4
was prepared first (as shown in Figure7).
By using the same rescaling factors (as listed in the biopattern
recognition), we generated the other seven datasets that contain dimmer hand script
patterns.Figure 7
The picture of 100 different patterns of alphanumeric letters 1, 2, 3, and 4 used in the optical character recognition experiment.In both
experiments, the digitized biopattern and character datasets were used for both
training and testing the artificial neural network. In contrast to the
traditional method of preparing independent training and test datasets, the
test datasets were assigned to be identical to the training datasets in order
to examine the feasibility of the proposed ANN model with the sigmoid-logarithmic
function.For simplicity, the intensities of the high-resolution
pixels of each original fluorescence dot in Figure6(a) were averaged and replaced by a single super pixel
in the pixellated image in Figure 6(a). If the patterns with the lowest resolution (one pixel for one dot/well)
can be correctly recognized by the ANN, the patterns with higher resolution
should be recognized by the ANN with higher recognition accuracy. To economically
implement the bioimaging chip, one neuron unit would be sufficient for
receiving all the lights emitting from the fluorescence profile of the imaging dot/well. The results of this simulated ANN model would
assist the future design of the proposed architecture. The average of the
intensities of the pixels of the original fluorescence patterns is considered simple
yet physically reasonable.The
back-propagation training using sigmoid-logarithmic transfer function and
gradient descent method was conducted to find a convergent weight configuration
(with fixed learning rateη=0.03). A criterion is employed to count the
percentage of training data that has been learned with an error less than 20%. In
order to guide the weight configuration closer to a convergence condition, the
BP training using regular sigmoid function was conducted first. The BP training
using sigmoid-logarithmic function was conducted afterwards.The entire procedure of the BP training
algorithm using sigmoid-logarithmic transfer function is described as follows.(i)
Prepare the input patterns for the
feedforward multilayer perceptron (MLP) neural network.(ii)
Assign the target values for the
associated input patterns.(iii)
Use the input patterns to train
the multilayer perceptron with sigmoid transfer function until the criterion
value becomes close to one. Now the weight configuration is closer to a
convergence condition for latter training.(iv)
Use the weight values obtained in
the previous step as the initial weight condition for training the multilayer
perceptron with the logarithmic-sigmoid transfer function. The regular BP algorithm using the gradient
descent method is again adopted. After
the criterion becomes one, the training is finished.(v)
Use this trained MLP with
logarithmic-sigmoid transfer function to recognize the test data set. Examine
the recognition accuracy.The detailed
conditions and pseudocodes of the BP algorithm with sigmoid and
logarithmic-sigmoid transfer function are provided as the following.The initial weight values for the first
weight matrixW and second weight matrix V were given randomly. The range of
these random weight values was set between −1 and 1. However, the range of the
trained weight values was unlimited. The
input vector X was the vectorized pixellated pattern. The associated target
values D were given. The learning rate (coefficient in front of the gradient
partial derivative) is parameter η.
The number of iterations is parameter iter. The training data set was used for testing as well to verify
the feasibility of this proposed algorithm.The
pseudocode for the regular back-propagation algorithm using sigmoid transfer
function is listed in
Algorithm1.The pseudocode for
the back-propagation algorithm using the piecewise sigmoid-logarithmic transfer
function is listed in
Algorithm2.


## 4.2. Simulations Results

The
result of recognizing all of the normalized genetic assay datasets by the
trained 100-input-100-hidden-neuron-2-output-neuron network is shown in Table1 (BIO). The network using new
transfer function can still find one desired biosignature in the dataset with
factor 1/100 of the original gray level. However, the network using
conventional sigmoid transfer functions cannot distinguish biopatterns well in
the datasets with factor 1/3.16 of the original gray level or below. Consistent recognition accuracy was obtained
for the optical character recognition (OCR) tasks (Table 1 (OCR)). The network using new transfer function can
recognize 32 characters in the dataset with factor 1/100 of the original gray
level but not the network with regular sigmoid function. The recognition
accuracies for datasets with a factor below 1/316 become constant because the
network tends to recognize one category perfectly according to the converged
weight configuration in this particular experiment.Table 1
Biosignature
and OCR recognition results (Unit: counts of patterns correctly recognized in
one test dataset. OCR: each test
dataset contains 100 characters. BIO: biosignature recognition task,
each test dataset contains 20 patterns.)
Gray
level to original dataTransfer function (learning rate)Hybrid
sigmoid-logarithmic1/(1+exp(−h))1/(1+exp(−5h))(η=0.03)(η=0.03)(η=0.0018)BIOOCRBIOOCRBIOOCR1 (original)2064207920631/3.16205864813651/1020581251371/31.620471251251/1001932———251/3161726————1/10001725————1/100001725————


## 5. Conclusion

A new optoelectronic
multichip microsystem for real-time field applicable robust dual-label PCR
assay analysis was proposed. This microsystem architecture contains a front-end
bioimage chip for analog signal conversion and augmentation, and an artificial
neural network for the autonomous data analysis purpose. A differential
logarithmic bioimage chip is designed and presented. The typical data analysis
procedure of taking logarithm of the ratio of the normalized post-PCR sample
intensity is conducted effectively in this differential logarithmic bio-image
chip. A single channel logarithmic circuit of the differential logarithmic
bioimage chip was designed, fabricated, and characterized. The weak
fluorescence signals can be amplified by this logarithmic amplifier circuit for
easier data analysis. Regarding the ANN
subsystem, an unsupervised hardware version: a weight-reconfigurable winner-take-all
ANN SoC chip suitable for selforganized Kohonen filter algorithm, and a
supervised software version: a computer-simulated ANN using back-propagation
algorithm with a novel sigmoid-logarithmic transfer function is presented. The back-propagation
neural network learning algorithm using the sigmoid-logarithmic function was successfully
simulated. The simulation results show that a trained ANN using this new
transfer function can classify low-fluorescence patterns better than using the
conventional sigmoid transfer function. This software model might be applicable to
other medical image processing tasks. In
summary, by integrating the optical setup, the bioimage chip, and the
artificial neural network processor with excellent performances and advantages
listed previously, we can envision the success of using this compact
microsystem to conduct on-site, real-time, noise-tolerable, and high-throughput
dual-labeled genetic expression analysis efficiently.Algorithm 1
Define
input vector to the first layer that contains inputpatternsX with −1 biasFori=1 to iterCalculate hidden neuron output usingsigmoid functionDefine output vector from the hidden layerthat contains hidden neuron output and −1 biasCalculate the final output using sigmoid functionCheck the criterion of percentage of the input datathat has an error less than 20%If all input data have errors less than 20%, stopthe trainingCompute the first back-propagation error setCompute the second back-propagation error setUpdate the second weight matrixUpdate the first weight matrixEndAlgorithm 2
Define
input vector that contains input patternsXwith −1 biasFori=1 to iterCalculate the hidden neuron output according
to wherethe net weighted input is falling in the range of thepiecewise sigmoid-logarithmic functionDefine output vector from the hidden layer
thatcontains hidden neuron output and −1 biasCalculate the final neuron output, the
first back-propagation error set, and the second back-propagationerror set according
to where the net weighted input isfalling in the range of the piecewise
sigmoid-logarithmicfunctionCheck the criterion of percentage of the
input datathat has an
error less than 20%If all input data have errors less than
20%,stop the trainingUpdate the second weight matrixUpdate the first weight matrixEnd



---

*Source: 259174-2008-06-11.xml*