OpenRaiser/Intern-Atlas huggingface dataset:
Informative Visual Storytelling with Cross-modal Rules
Venue: arXiv cs.MM 2019
Year: 2019

Existing methods in the Visual Storytelling field often suffer from the problem of generating general descriptions, while the image contains a lot of meaningful contents remaining unnoticed. The failure of informative story generation can be concluded to the model's incompetence of capturing enough meaningful concepts. The categories of these concepts include entities, attributes, actions, and events, which are in some cases crucial to grounded storytelling. To solve this problem, we propose a method to mine the cross-modal rules to help the model infer these informative concepts given certain visual input. We first build the multimodal transactions by concatenating the CNN activations and the word indices. Then we use the association rule mining algorithm to mine the cross-modal rules, which will be used for the concept inference. With the help of the cross-modal rules, the generated stories are more grounded and informative. Besides, our proposed method holds the advantages of interpretation, expandability, and transferability, indicating potential for wider application. Finally, we leverage these concepts in our encoder-decoder framework with the attention mechanism. We conduct several experiments on the VIsual StoryTelling~(VIST) dataset, the results of which demonstrate the effectiveness of our approach in terms of both automatic metrics and human evaluation. Additional experiments are also conducted showing that our mined cross-modal rules as additional knowledge helps the model gain better performance when trained on a small dataset.
Max-Min Rate of Cell-Free Massive MIMO Uplink with Optimal Uniform Quantization
Venue: arXiv cs.IT 2019
Year: 2019

Cell-free Massive multiple-input multiple-output (MIMO) is considered, where distributed access points (APs) multiply the received signal by the conjugate of the estimated channel, and send back a quantized version of this weighted signal to a central processing unit (CPU). For the first time, we present a performance comparison between the case of perfect fronthaul links, the case when the quantized version of the estimated channel and the quantized signal are available at the CPU, and the case when only the quantized weighted signal is available at the CPU. The Bussgang decomposition is used to model the effect of quantization. The max-min problem is studied, where the minimum rate is maximized with the power and fronthaul capacity constraints. To deal with the non-convex problem, the original problem is decomposed into two sub-problems (referred to as receiver filter design and power allocation). Geometric programming (GP) is exploited to solve the power allocation problem whereas a generalized eigenvalue problem is solved to design the receiver filter. An iterative scheme is developed and the optimality of the proposed algorithm is proved through uplink-downlink duality. A user assignment algorithm is proposed which significantly improves the performance. Numerical results demonstrate the superiority of the proposed schemes.
Detecting Fault Injection Attacks with Runtime Verification
Venue: arXiv cs.CR 2019
Year: 2019

Fault injections are increasingly used to attack/test secure applications. In this paper, we define formal models of runtime monitors that can detect fault injections that result in test inversion attacks and arbitrary jumps in the control flow. Runtime verification monitors offer several advantages. The code implementing a monitor is small compared to the entire application code. Monitors have a formal semantics; and we prove that they effectively detect attacks. Each monitor is a module dedicated to detecting an attack and can be deployed as needed to secure the application. A monitor can run separately from the application or it can be ``weaved'' inside the application. Our monitors have been validated by detecting simulated attacks on a program that verifies a user PIN.
Graphyti: A Semi-External Memory Graph Library for FlashGraph
Venue: arXiv cs.DC 2019
Year: 2019

Graph datasets exceed the in-memory capacity of most standalone machines. Traditionally, graph frameworks have overcome memory limitations through scale-out, distributing computing. Emerging frameworks avoid the network bottleneck of distributed data with Semi-External Memory (SEM) that uses a single multicore node and operates on graphs larger than memory. In SEM, $\mathcal{O}(m)$ data resides on disk and $\mathcal{O}(n)$ data in memory, for a graph with $n$ vertices and $m$ edges. For developers, this adds complexity because they must explicitly encode I/O within applications. We present principles that are critical for application developers to adopt in order to achieve state-of-the-art performance, while minimizing I/O and memory for algorithms in SEM. We present them in Graphyti, an extensible parallel SEM graph library built on FlashGraph and available in Python via pip. In SEM, Graphyti achieves 80% of the performance of in-memory execution and retains the performance of FlashGraph, which outperforms distributed engines, such as PowerGraph and Galois.
Bootstrap Model Ensemble and Rank Loss for Engagement Intensity Regression
Venue: arXiv cs.CV 2019
Year: 2019

This paper presents our approach for the engagement intensity regression task of EmotiW 2019. The task is to predict the engagement intensity value of a student when he or she is watching an online MOOCs video in various conditions. Based on our winner solution last year, we mainly explore head features and body features with a bootstrap strategy and two novel loss functions in this paper. We maintain the framework of multi-instance learning with long short-term memory (LSTM) network, and make three contributions. First, besides of the gaze and head pose features, we explore facial landmark features in our framework. Second, inspired by the fact that engagement intensity can be ranked in values, we design a rank loss as a regularization which enforces a distance margin between the features of distant category pairs and adjacent category pairs. Third, we use the classical bootstrap aggregation method to perform model ensemble which randomly samples a certain training data by several times and then averages the model predictions. We evaluate the performance of our method and discuss the influence of each part on the validation dataset. Our methods finally win 3rd place with MSE of 0.0626 on the testing set.
Fair Byzantine Agreements for Blockchains
Venue: arXiv cs.DC 2019
Year: 2019

Byzantine general problem is the core problem of the consensus algorithm, and many protocols are proposed recently to improve the decentralization level, the performance and the security of the blockchain. There are two challenging issues when the blockchain is operating in practice. First, the outcomes of the consensus algorithm are usually related to the incentive model, so whether each participant's value has an equal probability of being chosen becomes essential. However, the issues of fairness are not captured in the traditional security definition of Byzantine agreement. Second, the blockchain should be resistant to network failures, such as cloud services shut down or malicious attack, while remains the high performance most of the time. This paper has two main contributions. First, we propose a novel notion called fair validity for Byzantine agreement. Intuitively, fair validity lower-bounds the expected numbers that honest nodes' values being decided if the protocol is executed many times. However, we also show that any Byzantine agreement could not achieve fair validity in an asynchronous network, so we focus on synchronous protocols. This leads to our second contribution: we propose a fair, responsive and partition-resilient Byzantine agreement protocol tolerating up to 1/3 corruptions. Fairness means that our protocol achieves fair validity. Responsiveness means that the termination time only depends on the actual network delay instead of depending on any pre-determined time bound. Partition-resilience means that the safety still holds even if the network is partitioned, and the termination will hold if the partition is resolved.
Joint Neural Collaborative Filtering for Recommender Systems
Venue: arXiv cs.IR 2019
Year: 2019

We propose a J-NCF method for recommender systems. The J-NCF model applies a joint neural network that couples deep feature learning and deep interaction modeling with a rating matrix. Deep feature learning extracts feature representations of users and items with a deep learning architecture based on a user-item rating matrix. Deep interaction modeling captures non-linear user-item interactions with a deep neural network using the feature representations generated by the deep feature learning process as input. J-NCF enables the deep feature learning and deep interaction modeling processes to optimize each other through joint training, which leads to improved recommendation performance. In addition, we design a new loss function for optimization, which takes both implicit and explicit feedback, point-wise and pair-wise loss into account. Experiments on several real-word datasets show significant improvements of J-NCF over state-of-the-art methods, with improvements of up to 8.24% on the MovieLens 100K dataset, 10.81% on the MovieLens 1M dataset, and 10.21% on the Amazon Movies dataset in terms of HR@10. NDCG@10 improvements are 12.42%, 14.24% and 15.06%, respectively. We also conduct experiments to evaluate the scalability and sensitivity of J-NCF. Our experiments show that the J-NCF model has a competitive recommendation performance with inactive users and different degrees of data sparsity when compared to state-of-the-art baselines.
A Multi-Stage Clustering Framework for Automotive Radar Data
Venue: arXiv cs.LG 2019
Year: 2019

Radar sensors provide a unique method for executing environmental perception tasks towards autonomous driving. Especially their capability to perform well in adverse weather conditions often makes them superior to other sensors such as cameras or lidar. Nevertheless, the high sparsity and low dimensionality of the commonly used detection data level is a major challenge for subsequent signal processing. Therefore, the data points are often merged in order to form larger entities from which more information can be gathered. The merging process is often implemented in form of a clustering algorithm. This article describes a novel approach for first filtering out static background data before applying a twostage clustering approach. The two-stage clustering follows the same paradigm as the idea for data association itself: First, clustering what is ought to belong together in a low dimensional parameter space, then, extracting additional features from the newly created clusters in order to perform a final clustering step. Parameters are optimized for filtering and both clustering steps. All techniques are assessed both individually and as a whole in order to demonstrate their effectiveness. Final results indicate clear benefits of the first two methods and also the cluster merging process under specific circumstances.
Inapproximability Results for Scheduling with Interval and Resource Restrictions
Venue: arXiv cs.CC 2019
Year: 2019

In the restricted assignment problem, the input consists of a set of machines and a set of jobs each with a processing time and a subset of eligible machines. The goal is to find an assignment of the jobs to the machines minimizing the makespan, that is, the maximum summed up processing time any machine receives. Herein, jobs should only be assigned to those machines on which they are eligible. It is well-known that there is no polynomial time approximation algorithm with an approximation guarantee of less than 1.5 for the restricted assignment problem unless P=NP. In this work, we show hardness results for variants of the restricted assignment problem with particular types of restrictions. In the case of interval restrictions the machines can be totally ordered such that jobs are eligible on consecutive machines. We resolve the open question of whether the problem admits a polynomial time approximation scheme (PTAS) in the negative (unless P=NP). There are several special cases of this problem known to admit a PTAS. Furthermore, we consider a variant with resource restriction where each machine has capacities and each job demands for a fixed number of resources. A job is eligible on a machine if its demand is at most the capacity of the machine for each resource. For one resource, this problem is known to admit a PTAS, for two, the case of interval restrictions is contained, and in general, the problem is closely related to unrelated scheduling with a low rank processing time matrix. We show that there is no polynomial time approximation algorithm with a rate smaller than 48/47 or 1.5 for scheduling with resource restrictions with 2 or 4 resources, respectively, unless P=NP. All our results can be extended to the so called Santa Claus variants of the problems where the goal is to maximize the minimal processing time any machine receives.
IoT Platforms for 5G Network and Practical Considerations: A Survey
Venue: arXiv cs.DC 2019
Year: 2019

The fifth generation (5G) mobile network will enable the Internet of Things (IoT) to take a large leap into the age of future computing. As a result of extended connectivity, high speed, reduced latency services being provided by 5G, IoT has experienced and will continue to undergo a remarkable transition in every field of daily life. Furthermore, fog computing will revolutionize the IoT platforms by decentralizing the operations by the cloud and ensuring sustainability with big data, mobility and reduced processing lag. 5G is ubiquitous, reliable, scalable and economic in nature. The features will not only globalize IoT in a broader spectrum, but also make common people interact smartly and efficiently with the environment in real time. In this study, a combined survey is presented on different IoT applications coupled with cloud platforms. Moreover, the capabilities of IoT in the influence of 5G are explored as well as how the IoT platform and services will adopt through 5G are envisaged. Additionally, some open issues triggered by 5G have been introduced to harness the maximum benefit out of this network. Finally, a platform is proposed to implement in the telepresence project based on the investigation and findings.
Metamorphic IOTA
Venue: arXiv cs.CR 2019
Year: 2019

IOTA opened recently a new line of research in distributed ledgers area by targeting algorithms that ensure a high throughput for the transactions generated in IoT systems. Transactions are continuously appended to an acyclic structure called tangle and each new transaction selects as parents two existing transactions (called tips) that it approves. G-IOTA, a very recent improvement of IOTA, targets to protect tips left behind offering hence a good confidence level. However, this improvement had a cost: the use of an additional tip selection mechanism which may be critical in IoT systems since it needs additional energy consumption. In this paper we propose a new metamorphic algorithm for tip selection that offers the best guaranties of both IOTA and G-IOTA. Our contribution is two fold. First, we propose a parameterized algorithm, E-IOTA, for tip selection which targets to reduce the number of random walks executed in previous versions (IOTA and G-IOTA) while maintaining the same security guaranties as IOTA and the same confidence level and fairness with respect to tips selection as G-IOTA. Then we propose a formal analysis of the security guaranties offered by E-IOTA against various attacks mentioned in the original IOTA proposal (e.g. large weight attack, parasite chain attack and splitting attack). Interestingly, to the best of our knowledge this is the first formal analysis of the security guaranties of IOTA and its derivatives.
$\unicode{8523}$ means Parallel: Multiplicative Linear Logic Proofs as Concurrent Functional Programs
Venue: arXiv cs.LO 2019
Year: 2019

Along the lines of the Abramsky ``Proofs-as-Processes'' program, we present an interpretation of multiplicative linear logic as typing system for concurrent functional programming. In particular, we study a linear multiple-conclusion natural deduction system and show it is isomorphic to a simple and natural extension of $λ$-calculus with parallelism and communication primitives, called $λ_{\unicode{8523}}$. We shall prove that $λ_{\unicode{8523}}$ satisfies all the desirable properties for a typed programming language: subject reduction, progress, strong normalization and confluence.
StakeDag: Stake-based Consensus For Scalable Trustless Systems
Venue: arXiv cs.DC 2019
Year: 2019

Trustless systems, such as those blockchain enpowered, provide trust in the system regardless of the trust of its participants, who may be honest or malicious. Proof-of-stake (PoS) protocols and DAG-based approaches have emerged as a better alternative than the proof of work (PoW) for consensus. This paper introduces a new model, so-called \emph{\stakedag}, which aims for PoS consensus in a DAG-based trustless system. We address a general model of trustless system in which participants are distinguished by their stake or trust: users and validators. Users are normal participants with a no assumed trust and validators are high profile participants with an established trust. We then propose a new family of stake-based consensus protocols $\mathfrak{S}$, operating on the DAG as in the Lachesis protocol~\cite{lachesis01}. Specifically, we propose a stake-based protocol $S_ϕ$ that leverages participants' stake as validating weights to achieve more secure distributed systems with practical Byzantine fault tolerance (pBFT) in leaderless asynchronous Directed Acyclic Graph (DAG). We then present a general model of staking for asynchronous DAG-based distributed systems.
General non-linear Bellman equations
Venue: arXiv cs.LG 2019
Year: 2019

We consider a general class of non-linear Bellman equations. These open up a design space of algorithms that have interesting properties, which has two potential advantages. First, we can perhaps better model natural phenomena. For instance, hyperbolic discounting has been proposed as a mathematical model that matches human and animal data well, and can therefore be used to explain preference orderings. We present a different mathematical model that matches the same data, but that makes very different predictions under other circumstances. Second, the larger design space can perhaps lead to algorithms that perform better, similar to how discount factors are often used in practice even when the true objective is undiscounted. We show that many of the resulting Bellman operators still converge to a fixed point, and therefore that the resulting algorithms are reasonable and inherit many beneficial properties of their linear counterparts.
TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications
Venue: arXiv cs.LG 2019
Year: 2019

Ball trajectory data are one of the most fundamental and useful information in the evaluation of players' performance and analysis of game strategies. Although vision-based object tracking techniques have been developed to analyze sport competition videos, it is still challenging to recognize and position a high-speed and tiny ball accurately. In this paper, we develop a deep learning network, called TrackNet, to track the tennis ball from broadcast videos in which the ball images are small, blurry, and sometimes with afterimage tracks or even invisible. The proposed heatmap-based deep learning network is trained to not only recognize the ball image from a single frame but also learn flying patterns from consecutive frames. TrackNet takes images with a size of $640\times360$ to generate a detection heatmap from either a single frame or several consecutive frames to position the ball and can achieve high precision even on public domain videos. The network is evaluated on the video of the men's singles final at the 2017 Summer Universiade, which is available on YouTube. The precision, recall, and F1-measure of TrackNet reach $99.7\%$, $97.3\%$, and $98.5\%$, respectively. To prevent overfitting, 9 additional videos are partially labeled together with a subset from the previous dataset to implement 10-fold cross-validation, and the precision, recall, and F1-measure are $95.3\%$, $75.7\%$, and $84.3\%$, respectively. A conventional image processing algorithm is also implemented to compare with TrackNet. Our experiments indicate that TrackNet outperforms conventional method by a big margin and achieves exceptional ball tracking performance. The dataset and demo video are available at https://nol.cs.nctu.edu.tw/ndo3je6av9/.
Multirate PWM balance method for the efficient field-circuit coupled simulation of power converters
Venue: arXiv cs.CE 2019
Year: 2019

The field-circuit coupled simulation of switch-mode power converters with conventional time discretization is computationally expensive since very small time steps are needed to appropriately account for steep transients occurring inside the converter, not only for the degrees of freedom (DOFs) in the circuit, but also for the large number of DOFs in the field model part. An efficient simulation technique for converters with idealized switches is obtained using multirate partial differential equations, which allow for a natural separation into components of different time scales. This paper introduces a set of new PWM eigenfunctions which decouple the systems of equations and thus yield an efficient simulation of the field-circuit coupled problem. The resulting method is called the multirate PWM balance method.
Correlation via synthesis: end-to-end nodule image generation and radiogenomic map learning based on generative adversarial network
Venue: arXiv cs.CV 2019
Year: 2019

Radiogenomic map linking image features and gene expression profiles is useful for noninvasively identifying molecular properties of a particular type of disease. Conventionally, such map is produced in three separate steps: 1) gene-clustering to "metagenes", 2) image feature extraction, and 3) statistical correlation between metagenes and image features. Each step is independently performed and relies on arbitrary measurements. In this work, we investigate the potential of an end-to-end method fusing gene data with image features to generate synthetic image and learn radiogenomic map simultaneously. To achieve this goal, we develop a generative adversarial network (GAN) conditioned on both background images and gene expression profiles, synthesizing the corresponding image. Image and gene features are fused at different scales to ensure the realism and quality of the synthesized image. We tested our method on non-small cell lung cancer (NSCLC) dataset. Results demonstrate that the proposed method produces realistic synthetic images, and provides a promising way to find gene-image relationship in a holistic end-to-end manner.
Assembly Planning by Subassembly Decomposition Using Blocking Reduction
Venue: arXiv cs.RO 2019
Year: 2019

The sequence in which a complex product is assembled directly impacts the ease and efficiency of the assembly process, whether executed by a human or a robot. A sequence that gives the assembler the greatest freedom of movement is therefore desirable. Our main contribution is an expression of obstruction relationships between parts as a disassembly interference graph (DIG). We validate this heuristic by developing a disassembly sequence planner that partitions assemblies in a way that prioritizes access to parts, resulting in plans that are comparable in efficiency to two state-of-the-art assembly methods in terms of total plan length. Using DIG, our method generates successive subassembly decompositions, yielding a tree structure that makes parallization opportunities apparent. Our planner generates viable disassembly plans by minimizing our part blockage measure, and thereby demonstrates that this measure is a valuable addition to the Assembly Sequence Planning toolkit.