Miracle-dz/UniFAR-FTUs huggingface dataset:
Blind Men and the Elephant : Piecing Together Hadoop for Diagnosis

Google’s MapReduce framework enables distributed, data-intensive, parallel applications by decomposing a massive job into smaller (Map and Reduce) tasks and a massive data-set into smaller partitions, such that each task processes a different partition in parallel. However, performance problems in a distributed MapReduce system can be hard to diagnose and to localize to a specific node or a set of nodes. On the other hand, the structure of large number of nodes performing similar tasks naturally affords us opportunities for observing the system from multiple viewpoints. We present a “Blind Men and the Elephant” (Blimey) framework in which we exploit this structure, and demonstrate how problems in a MapReduce system can be diagnosedůů by corroborating the multiple viewpoints. More specifically, we present algorithms within the Blimey framework based on OS-level performance counters, on white-box metrics extracted from logs, and on application-level heartbeats. We show that our Blimey algorithms are able to capture a variety of faults including resource hogs and application hangs, and to localize the fault to subsets of slave nodes in the MapReduce system. In addition, we discuss how the diagnostic algorithms’ outcomes can be further synthesized in a repeated application of the Blimey approach. We present a simple supervised learning technique which allows us to identify a fault if it has been previously observed. Keywords-MapReduce, Hadoop, Failure Diagnosis
Ice‐jam flood risk assessment and mapping

In northern regions, river ice‐ jam flooding can be more severe than open‐water flooding causing property and infrastructure damages, loss of human life and adverse impacts on aquatic ecosystems. Very little has been performed to assess the risk induced by ice‐related floods because most risk assessments are limited to open‐water floods. The specific objective of this study is to incorporate ice‐jam numerical modelling tools (e.g. RIVICE, Monte‐Carlo simulation) into flood hazard and risk assessment along the Peace River at the Town of Peace River (TPR) in Alberta, Canada. Adequate historical data for different ice‐jam and open‐water flooding events were available for this study site and were useful in developing ice‐affected stage‐frequency curves. These curves were then applied to calibrate a numerical hydraulic model, which simulated different ice jams and flood scenarios along the Peace River at the TPR. A Monte‐Carlo analysis was then carried out to acquire an ensemble of water level profiles to determine the 1 : 100‐year and 1 : 200‐year annual exceedance probability flood stages for the TPR. These flood stages were then used to map flood hazard and vulnerability of the TPR. Finally, the flood risk for a 200‐year return period was calculated to be an average of $32/m2/a ($/m2/a corresponds to a unit of annual expected damages or risk). Copyright © 2016 John Wiley & Sons, Ltd.
LASSO-Patternsearch algorithm with application to ophthalmology data

The LASSO-Patternsearch is proposed, as a two-stage procedure to identify clusters of multiple risk factors for outcomes of interest in large demographic studies, when the predictor variables are dichotomous or take on values in a small finite set. Many diseases are suspected of having multiple interacting risk factors acting in concert, and it is of much interest to uncover higher order interactions when they exist. The method is related to Zhang et al(2004) except that variable flexibility is sacrificed to allow entertaining models with high as well as low order interactions among multiple predictors. A LASSO is used to select important patterns, being applied conservatively to have a high rate of retention of true patterns, while allowing some noise. Then the patterns selected by the LASSO are tested in the framework of (parametric) generalized linear models to reduce the noise. Notably, the patterns are those that arise naturally from the log linear expansion of the multivariate Bernoulli density. Separate tuning procedures are proposed for the LASSO step and then the parametric step and a novel computational algorithm for the LASSO step is developed to handle the large number of unknowns in the problem. The method is applied to data from the Beaver Dam Eye Study and is shown to expose physiologically interesting interacting risk factors. In a study of progression of myopia in an older cohort, it is found in this group that the risk for smokers is reduced by taking vitamins, while the risk for non-smokers is independent of the “taking vitamins” variable, which is in agreement with the general result that smoking reduces the absorption of vitamins, and certain vitamins have been
KNET: A General Framework for Learning Word Embedding using Morphological Knowledge

Neural network techniques are widely applied to obtain high-quality distributed representations of words, i.e., word embeddings, to address text mining, information retrieval, and natural language processing tasks. Recently, efficient methods have been proposed to learn word embeddings from context that captures both semantic and syntactic relationships between words. However, it is challenging to handle unseen words or rare words with insufficient context. In this paper, inspired by the study on word recognition process in cognitive psychology, we propose to take advantage of seemingly less obvious but essentially important morphological knowledge to address these challenges. In particular, we introduce a novel neural network architecture called KNET that leverages both contextual information and morphological word similarity built based on morphological knowledge to learn word embeddings. Meanwhile, the learning architecture is also able to refine the pre-defined morphological knowledge and obtain more accurate word similarity. Experiments on an analogical reasoning task and a word similarity task both demonstrate that the proposed KNET framework can greatly enhance the effectiveness of word embeddings.
Alleviating Mode Collapse in GAN via Diversity Penalty Module

The vanilla GAN (Goodfellow et al. 2014) suffers from mode collapse deeply, which usually manifests as that the images generated by generators tend to have a high similarity amongst them, even though their corresponding latent vectors have been very different. In this paper, we introduce a pluggable diversity penalty module (DPM) to alleviate mode collapse of GANs. It reduces the similarity of image pairs in feature space, i.e., if two latent vectors are different, then we enforce the generator to generate two images with different features. The normalized Gram matrix is used to measure the similarity. We compare the proposed method with Unrolled GAN (Metz et al. 2016), BourGAN (Xiao, Zhong, and Zheng 2018), PacGAN (Lin et al. 2018), VEEGAN (Srivastava et al. 2017) and ALI (Dumoulin et al. 2016) on 2D synthetic dataset, and results show that the diversity penalty module can help GAN capture much more modes of the data distribution. Further, in classification tasks, we apply this method as image data augmentation on MNIST, Fashion- MNIST and CIFAR-10, and the classification testing accuracy is improved by 0.24%, 1.34% and 0.52% compared with WGAN GP (Gulrajani et al. 2017), respectively. In domain translation, diversity penalty module can help StarGAN (Choi et al. 2018) generate more accurate attention masks and accelarate the convergence process. Finally, we quantitatively evaluate the proposed method with IS and FID on CelebA, CIFAR-10, MNIST and Fashion-MNIST, and the results suggest GAN with diversity penalty module gets much higher IS and lower FID compared with some SOTA GAN architectures.
dp -GAN : Alleviating Mode Collapse in GAN via Diversity Penalty Module

The vanilla GAN [5] suffers from mode collapse deeply, which usually manifests as that the images generated by generators tend to have a high similarity amongst them, even though their corresponding latent vectors have been very different. In this paper, we introduce a pluggable block called diversity penalty ( dp ) to alleviate mode collapse of GANs. It is used to reduce the similarity of image pairs in feature space, i.e., if two latent vectors are different, then we enforce the generator to generate two images with different features. The normalized Gram Matrix is used to measure the similarity. We compare the proposed method with Unrolled GAN [17], BourGAN [26], PacGAN [14], VEEGAN [23] and ALI [4] on 2D synthetic dataset, and results show that our proposed method can help GAN capture more modes of the data distribution. Further, we apply this penalty term into image data augmentation on MNIST, Fashion-MNIST and CIFAR-10, and the testing accuracy is improved by 0.24%, 1.34% and 0.52% compared with WGAN GP [6], respectively. Finally, we quantitatively evaluate the proposed method with IS and FID on CelebA, CIFAR-10, MNIST and Fashion-MNIST. Results show that our method gets much higher IS and lower FID compared with some current GAN architectures.
Real-Time Contactless Heart Rate Estimation from Facial Video

With the increase in health consciousness, noninvasive body monitoring has aroused interest among researchers. As one of the most important pieces of physiological information, researchers have remotely estimated heart rates from facial videos in recent years. Although progress has been made over the past few years, there are still some limitations, such as the increase in processing time with accuracy and the lack of comprehensive and challenging datasets for use and comparison. Recently, it was shown that heart rate information can be extracted from facial videos by spatial decomposition and temporal filtering. Inspired by this, a new framework is introduced in this thesis for remotely estimating the heart rate under realistic conditions by combining spatial and temporal filtering and a convolutional neural network. Our proposed approach exhibits better performance compared with that of the benchmark on the MMSE-HR dataset in terms of both the average heart rate estimation and short-term heart rate estimation. High consistency in short-term heart rate estimation is observed between our method and the ground truth.
GDoF of the MISO BC: Bridging the gap between finite precision CSIT and perfect CSIT

This work bridges the gap between sharply contrasting results on the degrees of freedom of the K user broadcast channel where the transmitter is equipped with K transmit antennas and each of the K receivers is equipped with a single antenna. This channel has K DoF when channel state information at the transmitter (CSIT) is perfect, but as shown recently, it has only 1 DoF when the CSIT is limited to finite precision. By considering the full range of partial CSIT assumptions parameterized by β ∈ [0,1], such that the strength of the channel estimation error terms scales as ~ SNR<sup>-β</sup> relative to the channel strengths which scale as ~ SNR, it is shown that this channel has 1 - β + Kβ DoF. For K = 2 users with arbitrary β<sub>ij</sub> parameters, the DoF are shown to be 1 + min<sub>i,j</sub> β<sub>ij</sub>. To explore diversity of channel strengths, the results are further extended to the symmetric Generalized Degrees of Freedom setting where the direct channel strengths scale as ~ SNR and the cross channel strengths scale as ~ SNR<sup>α</sup>, α ∈ [0,1], β ∈ [0,α]. Here, the roles of α and β are shown to counter each other on equal terms, so that the sum GDoF value in the K user setting is (α - β) + K(1 - (α-β )) and for the 2 user setting with arbitrary β<sub>ij</sub>, is 2 - α + min<sub>i,j</sub> β<sub>ij</sub>.
Fixed Points of Generalized Approximate Message Passing With Arbitrary Matrices

The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.
Coherent Motion Segmentation in Moving Camera Videos using Optical Flow Orientations-Supplementary material

In this supplementary material, we present some additional details and results for our paper. The sampling procedure used in our graphical model is in Section 2. The entire set of 46 orientation fields used as the “library” FOFs are presented in Section 3. Comparison of our segmentation results to various other methods is presented in Section 4. In Section 5, we compare our model to other models with optical flow orientations as the input. In addition to this document, the supplementary material includes sample video results for all 37 videos that we tested. Video results are discussed in Section 6.
HANTAVIRUS IN RODENTS OF SAN DIEGO COUNTY IN RELATION TO THE EL NIÑO-SOUTHERN OSCILLATION (ENSO)

OF THE THESIS Hantavirus in Rodents of San Diego County in Relation to the El Nino-Southern Oscillation (Enso) by Sarah Elizabeth Cokely Master of Public Health with a Concentration in Environmental Health San Diego State University, 2011 Hantavirus Pulmonary Syndrome (HPS) is a viral infection carried by rodents and transmitted to humans. Increases in rodent population following El Nino precipitation in the Four Corners region of the United States and resulting increases in HPS cases have been studied; however, no such studies exist for the San Diego County region. Proportions of hantavirus positive rodents were compared for years with normal weather events and years where an El Nino event occurred between 1994 and 2009 in San Diego County. Annual precipitation was also evaluated to establish if weather effects played a significant role in proportion of hantavirus positive rodents as well. When comparing percent of rodents positive for hantavirus in the year following an El Nino event as compared to all other years sampled, rodents in the year following El Nino years were almost three times more likely to test positive for hantavirus that rodents during normal weather events (OR 2.96, 95% CI 1.98-4.41) El Nino years did not experience a statistically significant greater rainfall than normal weather years with the mean rainfall for normal year 8.1 inches (SD 5.0 inches) and the mean rainfall for El Nino years 12.1 inches (SD 6.4 inches) and a p-value of 0.35. The excess rains in a region like San Diego County likely allow for a great increase in plant growth and therefore rodent resources, allowing for an increase in the hantavirus vector. The increase in hantavirus positive rodents in any given region presents a public health concern as hantavirus has no vaccine or virus specific treatment and has a high case fatality rate. A better understanding of whether or not El Nino events correlate with increases in hantavirus positive rodents in San Diego County can allow for increased surveillance and public alerts to prevent human cases.
Connectivity of Large Wireless Networks Under A General Connection Model

This paper studies networks where all nodes are distributed on a unit square <i>A</i>=<sup>Δ</sup> [- [1/2], [1/2]]<sup>2</sup> following a Poisson distribution with known density ρ and a pair of nodes separated by an Euclidean distance <i>x</i> are directly connected with probability <i>gr</i><sub>ρ</sub>(<i>x</i>)=<sup>Δ</sup><i>g</i>(<i>x</i>/<i>r</i><sub>ρ</sub>), independent of the event that any other pair of nodes are directly connected. Here, <i>g</i>:[0,∞)→ [0,1] satisfies the conditions of rotational invariance, nonincreasing monotonicity, integral boundedness, and <i>g</i>(<i>x</i>)=<i>o</i>(1/(<i>x</i><sup>2</sup>log<sup>2</sup><i>x</i>)) ; further, <i>r</i><sub>ρ</sub>=√{(logρ+<i>b</i>)/(<i>C</i>ρ)} where <i>C</i>=∫<sub>ℜ</sub><sup>2</sup><i>g</i>(||<i>x</i>||)<i>dx</i> and <i>b</i> is a constant. Denote the aforementioned network by <i>G</i>(<i>X</i><sub>ρ</sub>,<i>gr</i><sub>ρ</sub>,<i>A</i>). We show that as ρ→ ∞, 1) the distribution of the number of isolated nodes in <i>G</i>(<i>X</i><sub>ρ</sub>,<i>gr</i><sub>ρ</sub>,<i>A</i>) converges to a Poisson distribution with mean <i>e</i><sup>-</sup><i>b</i> ; 2) asymptotically almost surely (a.a.s.) there is no component in <i>G</i>(<i>X</i><sub>ρ</sub>,<i>gr</i><sub>ρ</sub>,<i>A</i>) of fixed and finite order <i>k</i> >; 1; c) a.a. s. the number of components with an unbounded order is one. Therefore, as ρ→ ∞, the network a.a. s. contains a unique unbounded component and isolated nodes only; a sufficient and necessary condition for <i>G</i>(<i>X</i><sub>ρ</sub>,<i>gr</i><sub>ρ</sub>,<i>A</i>) to be a.a. s. connected is that there is no isolated node in the network, which occurs when <i>b</i>→ ∞ as ρ→ ∞. These results expand recent results obtained for connectivity of random geometric graphs from the unit disk model and the fewer results from the log-normal model to the more general and more practical random connection model.
Strategies for Searching Video Content with Text Queries or Video Examples Features, Semantic Detectors, Fusion, Efficient Search and Reranking

The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search. However, metadata is often lacking for user-generated videos, thus these videos are unsearchable by current search engines. Therefore, content-based video retrieval (CBVR) tackles this metadata-scarcity problem by directly analyzing the visual and audio streams of each video. CBVR encompasses multiple research topics, including low-level feature design, feature fusion, semantic detector training and video search/reranking. We present novel strategies in these topics to enhance CBVR in both accuracy and speed under different query inputs, including pure textual queries and query by video examples. Our proposed strategies have been incorporated into our submission for the TRECVID 2014 Multimedia Event Detection evaluation, where our system outperformed other submissions in both text queries and video example queries, thus demonstrating the effectiveness of our proposed approaches.
The quasi 2 day wave activities during 2007 austral summer period as revealed by Whole Atmosphere Community Climate Model

The quasi 2 day wave (QTDW) observed during 2007 austral summer period is well reproduced in an reanalysis produced by the data assimilation version of the Whole Atmosphere Community Climate Model (WACCM + Data Assimilation Research Testbed) developed at National Center for Atmospheric Research (NCAR). It is found that the QTDW peaked 3 times from January to February but with different zonal wave numbers. Diagnostic analysis shows that the mean flow instabilities, refractive index, and critical layers of QTDWs are fundamental for their propagation and amplification, and thus, the temporal variations of the background wind are responsible for the different wave number structures at different times. The westward propagating wave number 2 mode (W2) grew and maximized in the first half of January, when the mean flow instabilities related to the summer easterly jet were enclosed by the critical layers of the westward propagating wave number 3 (W3) and wave number 4 (W4) modes. This prevented W3 and W4 from approaching and extracting energy from the unstable region. The W2 decayed rapidly thereafter due to the recession of critical layer and thus the lack of additional amplification by the mean flow instability. The W3 peaked in late January, when the instabilities were still encircled by the critical layer of W4. The attenuation of W3 afterward was also due to the disappearance of critical layer and thus the lack of overreflection. Finally, the W4 peaked in late February when both the instability and critical layer were appropriate.
Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning: Supplementary Materials

According to the main paper, we consider the generative model p(xt, zt, st|x1:t−1, s1:t−1) = p(st)p(zt|st; τ t)p(xt|zt) at time step t, where the dependence on x1:t−1, s1:t−1 is contained in τ t. Upon observing the data xt, both zt and st are inferred. However, exact inference is not available due to the intractability of the marginal likelihood p(xt|s1:t,x1:t−1). To tackle this, we utilize structured variational inference for both the latent variables zt and the Bernoulli change variable st. To this end, we define the joint variational distribution q(zt, st) = q(st|s1:t−1)q(zt|s1:t) as in the main paper. For notational simplicity, we omit the dependence on s1:t−1. Then the updating procedure for q(st) and q(zt|st) is obtained by maximizing the ELBO L(q): q(zt, st) = arg max q(zt,st)∈Q L(q),
A Simple and Approximately Optimal Mechanism for a Buyer with Complements: Abstract

We consider a revenue-maximizing seller with m heterogeneous items and a single buyer whose valuation v for the items may exhibit both substitutes (i.e., for some S, T, v(S ∪ T) < v(S) + v(T)) and complements (i.e., for some S, T, v(S ∪ T) > v(S) + v(T)). We show that the mechanism first proposed by Babaioff et al. [2014] -- the better of selling the items separately and bundling them together -- guarantees a Θ(d) fraction of the optimal revenue, where $d$ is a measure on the degree of complementarity. Note that this is the first approximately optimal mechanism for a buyer whose valuation exhibits any kind of complementarity. It extends the work of Rubinstein and Weinberg [2015], which proved that the same simple mechanisms achieve a constant factor approximation when buyer valuations are subadditive, the most general class of complement-free valuations. Our proof is enabled by the recent duality framework developed in Cai et al. [2016], which we use to obtain a bound on the optimal revenue in this setting. Our main technical contributions are specialized to handle the intricacies of settings with complements, and include an algorithm for partitioning edges in a hypergraph. Even nailing down the right model and notion of "degree of complementarity" to obtain meaningful results is of interest, as the natural extensions of previous definitions provably fail.
Scour Reduction by Collars around Offshore Monopiles

Excessive scour is a threat to the stability of monopile foundations, e.g. in offshore windparks. Particularly in shallow waters with strong tidal currents such as the North Sea, protection against excessive scouring is required. Commonly applied scour protection consists of loose rock. Little is known about alternative scour protection measures. One of the alternatives is to use a collar installed around the base of the monopile at seabed level. Indicative laboratory experiments were conducted to investigate whether collars reduce pile scouring under combined current and waves. This paper summarizes the experimental set-up, monitoring techniques, test program and results of the conducted experiments. The analysis focused on the equilibrium scour depth and the rate of scour development. The results showed that collars prevented pile scour under currents. Under extreme storm conditions scour occurred, but with a significant time delay, at a lower rate and a reduced equilibrium scour depth.