Miracle-dz/UniFAR-FTUs huggingface dataset:
Human Attention in Image Captioning: Dataset and Analysis

In this work, we present a novel dataset consisting of eye movements and verbal descriptions recorded synchronously over images. Using this data, we study the differences in human attention during free-viewing and image captioning tasks. We look into the relationship between human atten- tion and language constructs during perception and sen- tence articulation. We also analyse attention deployment mechanisms in the top-down soft attention approach that is argued to mimic human attention in captioning tasks, and investigate whether visual saliency can help image caption- ing. Our study reveals that (1) human attention behaviour differs in free-viewing and image description tasks. Hu- mans tend to fixate on a greater variety of regions under the latter task, (2) there is a strong relationship between de- scribed objects and attended objects (97% of the described objects are being attended), (3) a convolutional neural net- work as feature encoder accounts for human-attended re- gions during image captioning to a great extent (around 78%), (4) soft-attention mechanism differs from human at- tention, both spatially and temporally, and there is low correlation between caption scores and attention consis- tency scores. These indicate a large gap between humans and machines in regards to top-down attention, and (5) by integrating the soft attention model with image saliency, we can significantly improve the model’s performance on Flickr30k and MSCOCO benchmarks. The dataset can be found at: https://github.com/SenHe/ Human-Attention-in-Image-Captioning.
Leveraging Random Assignment in Multiple Imputation of Missing Covariates in Causal Studies

Baseline covariates in randomized experiments are often used in the estimation of treatment effects, for example, when estimating treatment effects within covariate-defined subgroups. In practice, however, covariate values may be missing for some data subjects. To handle missing values, analysts can use multiple imputation to create completed datasets, from which they can estimate the treatment effects. We investigate the performance of multiple imputation routines that utilize randomized treatment assignment, that is, make use of the fact that the true covariate distributions are the same across treatment arms. We do so for both ignorable and non-ignorable missing data, using simulation studies to compare the quality of inferences when we respect or disregard randomization. We consider this question for imputation routines estimated with covariates only, and imputation routines estimated conditional on the outcome variable. In either case, accounting for randomization does not noticeably improve inferences when sample sizes are in hundreds of units.
Research Opportunities in Sociotechnical Interventions for Health Disparity Reduction

The implicit and explicit biases built into our computing systems are becoming increasingly clear -- they impact everything from targeting of advertisements to how we are identified as people. These biases disproportionately affect marginalized groups -- people who are excluded from mainstream social, economic, cultural, or political life -- more acutely. While these biases can affect all aspects of our lives, from leisure to criminal justice to personal finances, they are all the more critical in the context of health and healthcare due to their significant personal and societal implications. In this interdisciplinary workshop, we explored how to design and build health systems for diverse populations through the following disciplinary lenses. The Computing Community Consortium (CCC) sponsored a two-day workshop titled Sociotechnical Interventions for Health Disparity Reduction in collaboration with the leadership of the Society for Behavioral Medicine's (SBM) 39th Annual Meeting on Monday, April 9 and Tuesday, April 10, 2018 in New Orleans, Louisiana. The workshop's goal was to bring together leading researchers in computing, health informatics, behavioral medicine, and health disparities to develop an integrative research agenda focused on sociotechnical interventions to reduce health disparities and improve the health of marginalized populations.
RESDN: A Novel Metric and Method for Energy Efficient Routing in Software Defined Networks

Software-defined networking (SDN) paradigm, with the flexible and logically centralized control, enables dynamically minimizing the network energy consumption by redirecting paths of packets. However, the links and switches are designed to accommodate maximum traffic volume and their power consumption is not traffic proportional. Moreover, there exists a trade-off between energy efficiency and network performance that need to be considered together. Addressing these issues, we propose an energy efficiency metric named Ratio for Energy Saving in SDN (RESDN) that quantifies energy efficiency based on link utility intervals. We provide integer programming formulation and method for maximizing the RESDN of the network. To the best of our knowledge, RESDN approach is novel as it measures how links are profitably utilized in terms of the amount of energy they consume with respect to their utility. We analyze our approach considering various metrics of interest and different types of SDN enabled switches. Experiments show that maximizing the RESDN value improves energy efficiency while maintaining acceptable network performance. In comparison to state-of-the-art utility-based heuristics, RESDN method achieves up to 30% better ratio for energy saving, 14.7 watts per switch power saving, 38% link saving, 2 hops decrease in average path length, and 5% improved traffic proportionality.
Measurement and modeling of electron-cloud-induced betatron tune shifts at the Cornell Electron-Positron Storage Ring test accelerator

We report on extensive measurements at the Cornell Electron-positron Storage Ring of electron-cloud-induced betatron tune shifts for trains of positron bunches at 2.1 and 5.3 GeV with bunch populations ranging between 0.64x10^10 and 9.6x10^10. Measurements using a witness bunch with variable distance from the end of the train and variable bunch population provide information on cloud decay and cloud pinching during the bunch passage. We employ Monte Carlo simulations of the reflection and absorption of synchrotron radiation photons to determine the pattern of absorption sites around the circumference of the storage ring. The Geant4 simulation toolkit is used to model the interactions of the photons with the beampipe wall and determine the production energy and location distributions of the photoelectrons which seed the electron cloud. An electron cloud buildup model based on fitted ring-averaged secondary-yield properties of the vacuum chamber predicts tune shifts in good agreement with the measurements.
Adversarially Trained Convolutional Neural Networks for Semantic Segmentation of Ischaemic Stroke Lesion using Multisequence Magnetic Resonance Imaging

Ischaemic stroke is a medical condition caused by occlusion of blood supply to the brain tissue thus forming a lesion. A lesion is zoned into a core associated with irreversible necrosis typically located at the center of the lesion, while reversible hypoxic changes in the outer regions of the lesion are termed as the penumbra. Early estimation of core and penumbra in ischaemic stroke is crucial for timely intervention with thrombolytic therapy to reverse the damage and restore normalcy. Multisequence magnetic resonance imaging (MRI) is commonly employed for clinical diagnosis. However, a sequence singly has not been found to be sufficiently able to differentiate between core and penumbra, while a combination of sequences is required to determine the extent of the damage. The challenge, however, is that with an increase in the number of sequences, it cognitively taxes the clinician to discover symptomatic biomarkers in these images. In this paper, we present a data-driven fully automated method for estimation of core and penumbra in ischaemic lesions using diffusion-weighted imaging (DWI) and perfusion-weighted imaging (PWI) sequence maps of MRI. The method employs recent developments in convolutional neural networks (CNN) for semantic segmentation in medical images. In the absence of availability of a large amount of labeled data, the CNN is trained using an adversarial approach employing cross-entropy as a segmentation loss along with losses aggregated from three discriminators of which two employ relativistic visual Turing test. This method is experimentally validated on the ISLES-2015 dataset through three-fold cross-validation to obtain with an average Dice score of 0.82 and 0.73 for segmentation of penumbra and core respectively.
Detection of White Dwarf companions to Blue Straggler Stars from UVIT observations of M67

Abstract We investigate the old open cluster M67 using ultraviolet photometric data of Ultra-Violet Imaging Telescope in multi-filter far-UV bands. M67, well known for the presence of several blue straggler stars (BSS), has been put to detailed tests to understand their formation pathways. Currently, there are three accepted formation channels: mass transfer due to Roche-lobe overflow in binary systems, stellar mergers either due to dynamical collisions or through coalescence of close binaries. So far, there had not been any confirmed detection of a white dwarf (WD) companion to any of the BSSs in this cluster. Here, we present the detection of WD companions to 5 bright BSSs in M67. The multiwavelength spectral energy distributions covering 0.12 -11.5 μm range, were found to require binary spectral fits for 5 BSSs, consisting of a cool (BSS) and a hot companion. The parameters (Luminosity, Temperature, Radius and Mass) of the hot companions suggest them to be WDs with mass in the range 0.2 - 0.35 M⊙ with Teff ~11000 –24000 K.
SF-Net: Structured Feature Network for Continuous Sign Language Recognition

Continuous sign language recognition (SLR) aims to translate a signing sequence into a sentence. It is very challenging as sign language is rich in vocabulary, while many among them contain similar gestures and motions. Moreover, it is weakly supervised as the alignment of signing glosses is not available. In this paper, we propose Structured Feature Network (SF-Net) to address these challenges by effectively learn multiple levels of semantic information in the data. The proposed SF-Net extracts features in a structured manner and gradually encodes information at the frame level, the gloss level and the sentence level into the feature representation. The proposed SF-Net can be trained end-to-end without the help of other models or pre-training. We tested the proposed SF-Net on two large scale public SLR datasets collected from different continuous SLR scenarios. Results show that the proposed SF-Net clearly outperforms previous sequence level supervision based methods in terms of both accuracy and adaptability.
Machine-learning based three-qubit gate for realization of a Toffoli gate with cQED-based transmon systems

We use machine learning techniques to design a 50 ns three-qubit flux-tunable controlled-controlled-phase gate with fidelity of>99.99% for nearest-neighbor coupled transmons in circuit quantum electrodynamics architectures. We explain our gate design procedure where we enforce realistic constraints, and analyze the new gate's robustness under decoherence, distortion, and random noise. Our controlled-controlled-phase gate in combination with two single-qubit gates realizes a Toffoli gate which is widely used in quantum circuits, logic synthesis, quantum error correction, and quantum games.
VLBI imaging of black holes via second moment regularization

The imaging fidelity of the Event Horizon Telescope (EHT) is currently determined by its sparse baseline coverage. In particular, EHT coverage is dominated by long baselines, and is highly sensitive to atmospheric conditions and loss of sites between experiments. The limited short/mid-range baselines especially affect the imaging process, hindering the recovery of more extended features in the image. We present an algorithmic contingency for the absence of well-constrained short baselines in the imaging of compact sources, such as the supermassive black holes observed with the EHT. This technique enforces a specific second moment on the reconstructed image in the form of a size constraint, which corresponds to the curvature of the measured visibility function at zero baseline. The method enables the recovery of information lost in gaps of the baseline coverage on short baselines and enables corrections of any systematic amplitude offsets for the stations giving short-baseline measurements present in the observation. The regularization can use historical source size measurements to constrain the second moment of the reconstructed image to match the observed size. We additionally show that a characteristic size can be derived from available short-baseline measurements, extrapolated from other wavelengths, or estimated without complementary size constraints with parameter searches. We demonstrate the capabilities of this method for both static and movie reconstructions of variable sources.
Performance of variable and function selection methods for estimating the nonlinear health effects of correlated chemical mixtures: A simulation study

Statistical methods for identifying harmful chemicals in a correlated mixture often assume linearity in exposure‐response relationships. Nonmonotonic relationships are increasingly recognized (eg, for endocrine‐disrupting chemicals); however, the impact of nonmonotonicity on exposure selection has not been evaluated. In a simulation study, we assessed the performance of Bayesian kernel machine regression (BKMR), Bayesian additive regression trees (BART), Bayesian structured additive regression with spike‐slab priors (BSTARSS), generalized additive models with double penalty (GAMDP) and thin plate shrinkage smoothers (GAMTS), multivariate adaptive regression splines (MARS), and lasso penalized regression. We simulated realistic exposure data based on pregnancy exposure to 17 phthalates and phenols in the US National Health and Nutrition Examination Survey using a multivariate copula. We simulated data sets of size N = 250 and compared methods across 32 scenarios, varying by model size and sparsity, signal‐to‐noise ratio, correlation structure, and exposure‐response relationship shapes. We compared methods in terms of their sensitivity, specificity, and estimation accuracy. In most scenarios, BKMR, BSTARSS, GAMDP, and GAMTS achieved moderate to high sensitivity (0.52‐0.98) and specificity (0.21‐0.99). BART and MARS achieved high specificity (≥0.90), but low sensitivity in low signal‐to‐noise ratio scenarios (0.20‐0.51). Lasso was highly sensitive (0.71‐0.99), except for quadratic relationships (≤0.27). Penalized regression methods that assume linearity, such as lasso, may not be suitable for studies of environmental chemicals hypothesized to have nonmonotonic relationships with outcomes. Instead, BKMR, BSTARSS, GAMDP, and GAMTS are attractive methods for flexibly estimating the shapes of exposure‐response relationships and selecting among correlated exposures.
Neural Open Relation Extraction via an Overlap-aware Sequence Tagging Scheme

Solving the Open relation extraction (ORE) task with supervised neural networks, especially the neural sequence learning (NSL) models, is an extraordinarily promising way. However, there are three main challenges: (1) The lack of labeled training corpus; (2) Only one label is assigned to each word, resulting in being difficult to extract multiple, overlapping relations; (3) The confusion about the selection of various neural architectures for the ORE. In this paper, to overcome these challenges, we design a novel tagging scheme to assist in building a large-scale, high-quality training dataset automatically. The scheme can improve the performance of models by assigning multiple, overlapping labels for each word and helping models to learn pre-identifying arguments segment-level information. In addition, we pick out a winning model empirically from various alternative neural structures. The model achieves state-of-the-art performance on four kinds of test sets. The experimental results show that the scheme is effective.
Flow Topology During Multiplexed Particle Manipulation Using a Stokes Trap

Trapping and manipulation of small particles underlies many scientific and technological applications. Recently, the precise manipulation of multiple small particles was demonstrated using a Stokes trap that relies only on fluid flow without the need for optical or electric fields. Active flow control generates complex flow topologies around suspended particles during the trapping process, yet the relationship between the control algorithm and flow structure is not well understood. In this work, we characterize the flow topology during active control of particle trajectories using a Stokes trap. Our results show that optimal control of two particles unexpectedly relies on flow patterns with zero or one stagnation points, as opposed to positioning two particles using two distinct stagnation points. We characterize the sensitivity of the system with respect to the parameters in the control objective function, thereby providing a systematic understanding of the trapping process. Overall, these results will be useful in guiding applications involving the controlled manipulation of multiple colloidal particles and the precise deformation of soft particles in defined flow fields.
Programmable In-Network Security for Context-aware BYOD Policies

Bring Your Own Device (BYOD) has become the new norm in enterprise networks, but BYOD security remains a top concern. Context-aware security, which enforces access control based on dynamic runtime context, holds much promise. Recent work has developed SDN solutions to collect device context for network-wide access control in a central controller. However, the central controller poses a bottleneck that can become an attack target, and processing context changes at remote software has low agility. We present a new paradigm, programmable in-network security (Poise), which is enabled by the emergence of programmable switches. At the heart of Poise is a novel switch primitive, which can be programmed to support a wide range of context-aware policies in hardware. Users of Poise specify concise policies, and Poise compiles them into different instantiations of the security primitive in P4. Compared to centralized SDN defenses, Poise is resilient to control plane saturation attacks, and it dramatically increases defense agility.
Classification is a Strong Baseline for Deep Metric Learning

Deep metric learning aims to learn a function mapping image pixels to embedding feature vectors that model the similarity between images. Two major applications of metric learning are content-based image retrieval and face verification. For the retrieval tasks, the majority of current state-of-the-art (SOTA) approaches are triplet-based non-parametric training. For the face verification tasks, however, recent SOTA approaches have adopted classification-based parametric training. In this paper, we look into the effectiveness of classification based approaches on image retrieval datasets. We evaluate on several standard retrieval datasets such as CAR-196, CUB-200-2011, Stanford Online Product, and In-Shop datasets for image retrieval and clustering, and establish that our classification-based approach is competitive across different feature dimensions and base feature networks. We further provide insights into the performance effects of subsampling classes for scalable classification-based training, and the effects of binarization, enabling efficient storage and computation for practical applications.
CRNet: Image Super-Resolution Using A Convolutional Sparse Coding Inspired Network

Convolutional Sparse Coding (CSC) has been attracting more and more attention in recent years, for making full use of image global correlation to improve performance on various computer vision applications. However, very few studies focus on solving CSC based image Super-Resolution (SR) problem. As a consequence, there is no significant progress in this area over a period of time. In this paper, we exploit the natural connection between CSC and Convolutional Neural Networks (CNN) to address CSC based image SR. Specifically, Convolutional Iterative Soft Thresholding Algorithm (CISTA) is introduced to solve CSC problem and it can be implemented using CNN architectures. Then we develop a novel CSC based SR framework analogy to the traditional SC based SR methods. Two models inspired by this framework are proposed for pre-/post-upsampling SR, respectively. Compared with recent state-of-the-art SR methods, both of our proposed models show superior performance in terms of both quantitative and qualitative measurements.
A Dynamic Analysis of Energy Storage With Renewable and Diesel Generation Using Volterra Equations

Energy storage systems will play a key role in the power system of the 21st century considering the large penetrations of variable renewable energy, growth in transport electrification, and decentralization of heating loads. Therefore, reliable real-time methods to optimize energy storage, demand response, and generation are vital for power system operations. This article presents a concise review of battery energy storage and an example of battery modeling for renewable energy applications and details an adaptive approach to solve this load leveling problem with storage. A dynamic evolutionary model based on the first-kind Volterra integral equation is used in both cases. A direct regularized numerical method is employed to find the least-cost dispatch of the battery in terms of the integral equation solution. Validation on real data shows that the proposed evolutionary Volterra model effectively generalizes a conventional discrete integral model taking into account both the state of health and the availability of generation/storage.
Beyond English-Only Reading Comprehension: Experiments in Zero-shot Multilingual Transfer for Bulgarian

Recently, reading comprehension models achieved near-human performance on large-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely due to the release of pre-trained contextualized representations such as BERT and ELMo, which can be fine-tuned for the target task. Despite those advances and the creation of more challenging datasets, most of the work is still done for English. Here, we study the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and we apply it to Bulgarian multiple-choice reading comprehension. We propose a new dataset containing 2,221 questions from matriculation exams for twelfth grade in various subjects —history, biology, geography and philosophy—, and 412 additional questions from online quizzes in history. While the quiz authors gave no relevant context, we incorporate knowledge from Wikipedia, retrieving documents matching the combination of question + each answer option. Moreover, we experiment with different indexing and pre-training strategies. The evaluation results show accuracy of 42.23%, which is well above the baseline of 24.89%.
Finding singularities in gravitational lensing

The number of strong lens systems is expected to increase significantly in ongoing and upcoming surveys. With an increase in the total number of such systems, we expect to discover many configurations that correspond to unstable caustics. In such cases, the instability can be used to our advantage for constraining the lens model. We have implemented algorithms for detection of different types of singularities in gravitational lensing. We apply our approach on a variety of lens models and then go on to test it with the inferred mass distribution for Abell 697 as an example application. We propose to represent lenses using A3-lines and singular points (A4 and D4) in the image plane. We propose this as a compact representation of complex lens systems that can capture all the details in a single snapshot.