Building information systems based on neural networks

title: Training Algorithm with Incomplete Data for Feed-Forward Neural Networks
abstract: A new algorithm is developed to train feed-forward neural networks for non-linear input-to-output mappings with small incomplete data in arbitrary distributions. The developed Training-EStimation-Training (TEST) algorithm consists of 3 steps, i.e., (1) training with the complete portion of the training data set, (2) estimation of the missing attributes with the trained neural networks, and (3) re-training the neural networks with the whole data set. Error back propagation is still applicable to estimate the missing attributes. Unlike other training methods with missing data, it does not assume data distribution models which may not be appropriate for small training data. The developed TEST algorithm is first tested for the Iris benchmark data. By randomly removing some attributes from the complete data set and estimating the values latter, accuracy of the TEST algorithm is demonstrated. Then it is applied to the Diabetes benchmark data, of which about 50% contains missing attributes. Compared with other existing algorithms, the proposed TEST algorithm results in much better recognition accuracy for test data.

title: Merging Back-propagation and Hebbian Learning Rules for Robust Classifications
abstract: By imposing saturation requirements on hidden-layer neural activations, a new learning algorithm is developed to improve robustness on classification performance of a multi-layer Perceptron. Derivatives of the sigmoid functions at hidden-layers are added to the standard output error with relative significance factors, and the total error is minimized by the steepest-descent method. The additional gradient-descent terms become Hebbian, and this new algorithm merges two popular learning algorithms, i.e., error back-propagation and Hebbian learning rules. Only slight modifications are needed for the standard back-propagation algorithm, and additional computational requirements are negligible. This saturation requirement effectively reduces output sensitivity to the input, which results in improved robustness and better generalization for classifier networks. Also distributed representations at hidden-layers are successfully suppressed to accomplish efficient utilization of hidden neurons. Computer simulations demonstrates much faster learning convergence as well as improved robustness for classifications and hetero-associations of binary patterns.

title: Out-of-Vocabulary Rejection based on Selective Attention Model
abstract: Based on a psychological selective attention theory a new algorithm is developed to provide reliable out-of-vocabulary (OOV) rejection for speech recognition systems in noisy environments. The developed attention model is based on Broadbent's ‘early filtering’ theory, and the attention adaptation process utilizes a gradient-descent error minimization algorithm with error backpropagation rule. The developed model is applied to isolated-word recognition tasks, and much higher in-vocabulary recognition rates are achieved with the same out-of-vocabulary rejection rates.

title: On the Efficient Speech Feature Extraction Based on Independent Component Analysis
abstract: A new efficient code for speech signals is proposed. To represent speech signals with minimum redundancy we use independent component analysis to adapt features (basis vectors) that efficiently encode the speech signals. The learned basis vectors are sparsely distributed and localized in both time and frequency. Time-frequency analysis of basis vectors shows the property similar with the critical bandwidth of human auditory system. Our results suggest that the obtained codes of speech signals are sparse and biologically plausible.

title: A modified infomax algorithm for blind signal separation
abstract: We present a new algorithm to perform blind signal separation (BSS), which takes a trade-off between the ordinary gradient infomax algorithm and the natural gradient infomax algorithm. Analyzing the algorithm, we show that desired equilibrium points are locally stable by choosing appropriate score functions and step sizes. The algorithm provides better performance than the ordinary gradient algorithm, and it is free from approximation error and the small-step-size restriction of the natural gradient algorithm. In simulations on convolved mixtures, the algorithm provides much better performance than the other algorithms while requiring less computation.

title: Noise-robust automatic speech recognition using mainlobe-resilient time-frequency quantile-based noise estimation
abstract: In standard speech recognition systems in which training data are clean speech, the presence of background noise in received signal can severely deteriorate the recognition performance. This paper presents a simple noise-robust speech recognition system based on a modified noise spectral estimation method called mainlobe-resilient time-frequency quantile-based noise estimation (M-R T-F QBNE), which focuses on the mainlobes at harmonic frequencies. We estimate the global signal-to-noise ratio (SNR) and select a recognition model, which is best matched to the SNR operating range. Experimental results show that the recognition accuracy of the proposed recognition system is higher than that of the AURORA2 clean training baseline by 23%. Compared to multicondition training, the proposed method achieves comparable recognition accuracy.

title: An Efficient Neighbor Knowledge Based Broadcasting for Mobile Ad Hoc Networks
abstract: Although a flooding is the most commonly used method for one-to-all communications, it often results in huge traffic in mobile ad hoc networks. Several optimization schemes have been proposed to alleviate this problem. However, none of them offers a complete solution and has their own limitations, such as, lack of reliability, high latency or excessive control packet overhead. In this paper, we propose the FONIAH (Flooding based on One- hop Neighbor Information and Adaptive Holding) scheme to overcome these limitations.

title: Statistical Shape Modelling of the Levator Ani with Thickness Variation
abstract: The levator ani is vulnerable to injury during childbirth and effective surgical intervention requires full knowledge of the morphology and mechanical properties of the muscle structure. The shape of the levator ani and regional thickening during different levels of physiological loading can provide an indication of pelvic floor dysfunction. This paper describes a coupled approach for shape and thickness statistical modelling based on harmonic embedding for volumetric analysis of the levator ani. With harmonic embedding, the dynamic information extracted by the statistical modelling captures shape and thickness variation of the levator ani across different subjects and during varying levels of stress. With this study, we demonstrate that the derived model is compact and physiologically meaningful, demonstrating the practical value of the technique.

title: Mammogram Analysis Using Two-Dimensional Autoregressive Models: Sufficient or Not?
abstract: Two-dimensional (2–D) autoregressive (AR) models have been used as one of the methods to characterise the textures of tumours in mammograms. Previously, the 2–D AR model coefficients were estimated for the block containing the tumour and the blocks in its 3 × 3 neighbourhood. In this paper, the possibility of having the estimated set of AR model coefficients of the block containing the tumour as a unique set of AR model coefficients for the entire mammogram is looked into. Based on the information given from the MiniMammography database, the possible number of blocks of the same size of the block containing the tumour is obtained from the entire mammogram and for each block a set of AR model coefficients is estimated using a method that combines both the Yule-Walker system of equations and the Yule-Walker system of equations in the third-order statistical domain. These sets of AR model coefficients are then compared. The simulation results show that 98.6% of the time we can not find another set of AR model coefficients representing the blocks of pixels in the possible neighbourhood of the entire mammogram for the data (95 mammograms with 5 of them having two tumours) available in the MiniMammography database.

title: Embedding Technical Analysis Into Neural Network Based Trading Systems
abstract: We have recently proposed a promising trading system for the S P 500 index which consists of a feature selection component and a simple filter for data preprocessing two specialized neural networks for return prediction and a rule base for prediction integration. The objective of this study is to explore if including additional knowledge for more sophisticated data filter ing and return integration leads to further improvements in the system. The new system uses a well known technical indicator to split the data and an additional indicator for reducing the number of unprofitable trades. Several system combinations are explored and tested over a 5 year trading period. The most promising system yielded an annual rate of return ARR of 15.99% with 54 trades. This compares favorably to the ARR for the buy and hold strategy 11.05% and to the best results obtained using the system with no technical analysis knowl edge embedded 13.35% with 126 trades.