nils-herrmann/clean_scirepeval huggingface dataset:
(Meta-) Evaluation of Machine Translation

This paper evaluates the translation quality of machine translation systems for 8 language pairs: translating French, German, Spanish, and Czech to English and back. We carried out an extensive human evaluation which allowed us not only to rank the different MT systems, but also to perform higher-level analysis of the evaluation process. We measured timing and intraand inter-annotator agreement for three types of subjective evaluation. We measured the correlation of automatic evaluation metrics with human judgments. This meta-evaluation reveals surprising facts about the most commonly used methodologies.
Predictive Tool for Software Team Performance

When supervising software engineering team projects, having all team members contribute actively to the project is often a challenge. Most often than not, there will be teams having some members with limited or no contribution. Thus one of the key roles of a team leader and academic supervisor are to monitor who is contributing and who is falling behind. Assessing the progress information of each team member becomes vital. This is to introduce strategies that encourages ensure every member is contributing effectively and efficiently in a timely manner. This paper proposes a rubric solution that assesses the progress information of each team member and provides a formative performance feedback on how each member is contributing to the project. This will enable each team member to reflect on his or her performance, and hopefully self-regulate and put in the necessary contribution. At the same time, this will assist the team leader and academic supervisor in monitoring the team member.
Online learning the consensus of multiple correspondences between sets

When several subjects solve the assignment problem of two sets, differences on the correspondences computed by these subjects may occur. These differences appear due to several factors. For example, one of the subjects may give more importance to some of the elements’ attributes than another subject. Another factor could be that the assignment problem is computed through a suboptimal algorithm and different nonoptimal correspondences can appear. In this paper, we present a consensus methodology to deduct the consensus of several correspondences between two sets. Moreover, we also present an online learning algorithm to deduct some weights that gauge the impact of each initial correspondence on the consensus. In the experimental section, we show the evolution of these parameters together with the evolution of the consensus accuracy. We observe that there is a clear dependence of the learned weights with respect to the quality of the initial correspondences. Moreover, we also observe that in the first iterations of the learning algorithm, the consensus accuracy drastically increases and then stabilises.
Sentiment Analysis for Social Media Images

In this proposal, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higher level challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them, and formulate sentiment prediction problem in two scenarios: supervised and unsupervised. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods. In the future, we are going to incorporating more information on the social network and explore sentiment on signed social network.
Thoth: Comprehensive Policy Compliance in Data Retrieval Systems

Data retrieval systems process data from many sources, each subject to its own data use policy. Ensuring compliance with these policies despite bugs, misconfiguration, or operator error in a large, complex, and fast evolving system is a major challenge. Thoth provides an efficient, kernel-level compliance layer for data use policies. Declarative policies are attached to the systems’ input and output files, key-value tuples, and network connections, and specify the data’s integrity and confidentiality requirements. Thoth tracks the flow of data through the system, and enforces policy regardless of bugs, misconfigurations, compromises in application code, or actions by unprivileged operators. Thoth requires minimal changes to an existing system and has modest overhead, as we show using a prototype Thoth-enabled data retrieval system based on the popular Apache Lucene.
GestureGAN for Hand Gesture-to-Gesture Translation in the Wild

Hand gesture-to-gesture translation in the wild is a challenging task since hand gestures can have arbitrary poses, sizes, locations and self-occlusions. Therefore, this task requires a high-level understanding of the mapping between the input source gesture and the output target gesture. To tackle this problem, we propose a novel hand Gesture Generative Adversarial Network (GestureGAN). GestureGAN consists of a single generator G and a discriminator D, which takes as input a conditional hand image and a target hand skeleton image. GestureGAN utilizes the hand skeleton information explicitly, and learns the gesture-to-gesture mapping through two novel losses, the color loss and the cycle-consistency loss. The proposed color loss handles the issue of "channel pollution" while back-propagating the gradients. In addition, we present the Frechet ResNet Distance (FRD) to evaluate the quality of generated images. Extensive experiments on two widely used benchmark datasets demonstrate that the proposed GestureGAN achieves state-of-the-art performance on the unconstrained hand gesture-to-gesture translation task. Meanwhile, the generated images are in high-quality and are photo-realistic, allowing them to be used as data augmentation to improve the performance of a hand gesture classifier. Our model and code are available at https://github.com/Ha0Tang/GestureGAN.
BRAD 1.0: Book reviews in Arabic dataset

The availability of rich datasets is a pre-requisite for proposing robust sentiment analysis systems. A variety of such datasets exists in English language. However, it is rare or nonexistent for the Arabic language except for a recent LABR dataset, which consists of a little bit over 63,000 book reviews extracted from. Goodreads. com. We introduce BRAD 1.0, the largest Book Reviews in Arabic Dataset for sentiment analysis and machine language applications. BRAD comprises of almost 510,600 book records. Each record corresponds for a single review and has the review in Arabic language and the reviewer's rating on a scale of 1 to 5 stars. In this paper, we present and describe the properties of BRAD. Further, we provide two versions of BRAD: the complete unbalanced dataset and the balanced version of BRAD. Finally, we implement four sentiment analysis classifiers based on this dataset and report our findings. When training and testing the classifiers on BRAD as opposed to LABR, an improvement rate growth of 46% is reported. The highest accuracy attained is 91%. Our core contribution is to make this benchmark-dataset available and accessible to the research community on Arabic language.
Transfer of mitochondria via tunneling nanotubes rescues apoptotic PC12 cells

Tunneling nanotubes (TNTs) are F-actin-based membrane tubes that form between cells in culture and in tissues. They mediate intercellular communication ranging from electrical signalling to the transfer of organelles. Here, we studied the role of TNTs in the interaction between apoptotic and healthy cells. We found that pheochromocytoma (PC) 12 cells treated with ultraviolet light (UV) were rescued when cocultured with untreated PC12 cells. UV-treated cells formed a different type of TNT with untreated PC12 cells, which was characterized by continuous microtubule localized inside these TNTs. The dynamic behaviour of mCherry-tagged end-binding protein 3 and the accumulation of detyrosinated tubulin in these TNTs indicate that they are regulated structures. In addition, these TNTs show different biophysical properties, for example, increased diameter allowing dye entry, prolonged lifetime and decreased membrane fluidity. Further studies demonstrated that microtubule-containing TNTs were formed by stressed cells, which had lost cytochrome c but did not enter into the execution phase of apoptosis characterized by caspase-3 activation. Moreover, mitochondria colocalized with microtubules in TNTs and transited along these structures from healthy to stressed cells. Importantly, impaired formation of TNTs and untreated cells carrying defective mitochondria were unable to rescue UV-treated cells in the coculture. We conclude that TNT-mediated transfer of functional mitochondria reverse stressed cells in the early stages of apoptosis. This provides new insights into the survival mechanisms of damaged cells in a multicellular context.
A mathematical model of the finding of usability problems

For 11 studies, we find that the detection of usability problems as a function of number of users tested or heuristic evaluators employed is well modeled as a Poisson process. The model can be used to plan the amount of evaluation required to achieve desired levels of thoroughness or benefits. Results of early tests can provide estimates of the number of problems left to be found and the number of additional evaluations needed to find a given fraction. With quantitative evaluation costs and detection values, the model can estimate the numbers of evaluations at which optimal cost/benefit ratios are obtained and at which marginal utility vanishes. For a “medium” example, we estimate that 16 evaluations would be worth their cost, with maximum benefit/cost ratio at four.
Stability of Causal Inference

We consider the sensitivity of causal identi cation to small perturbations in the input. A long line of work culminating in papers by Shpitser and Pearl (2006) and Huang and Valtorta (2008) led to a complete procedure for the causal identi cation problem. In our main result in this paper, we show that the identi cation function computed by these procedures is in some cases extremely unstable numerically. Speci cally, the “condition number” of causal identi cation can be of the order of Ω(exp(n0.49)) on an identi able semi-Markovian model with n visible nodes. That is, in order to give an output accurate to d bits, the empirical probabilities of the observable events need to be obtained to accuracy d +Ω(n0.49) bits.
Multi-scale Patch Aggregation (MPA) for Simultaneous Detection and Segmentation

Aiming at simultaneous detection and segmentation (SD-S), we propose a proposal-free framework, which detect and segment object instances via mid-level patches. We design a unified trainable network on patches, which is followed by a fast and effective patch aggregation algorithm to infer object instances. Our method benefits from end-to-end training. Without object proposal generation, computation time can also be reduced. In experiments, our method yields results 62.1% and 61.8% in terms of mAPr on VOC2012 segmentation val and VOC2012 SDS val, which are state-of-the-art at the time of submission. We also report results on Microsoft COCO test-std/test-dev dataset in this paper.
Stochastic Backpropagation and Approximate Inference in Deep Generative Models

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent an approximate posterior distribution and uses this for optimisation of a variational lower bound. We develop stochastic backpropagation – rules for gradient backpropagation through stochastic variables – and derive an algorithm that allows for joint optimisation of the parameters of both the generative and recognition models. We demonstrate on several real-world data sets that by using stochastic backpropagation and variational inference, we obtain models that are able to generate realistic samples of data, allow for accurate imputations of missing data, and provide a useful tool for high-dimensional data visualisation.
Adaptive Nonlinear System Identification with Echo State Networks

Echo state networks (ESN) are a novel approach to recurrent neural network training. An ESN consists of a large, fixed, recurrent "reservoir" network, from which the desired output is obtained by training suitable output connection weights. Determination of optimal output weights becomes a linear, uniquely solvable task of MSE minimization. This article reviews the basic ideas and describes an online adaptation scheme based on the RLS algorithm known from adaptive linear systems. As an example, a 10-th order NARMA system is adaptively identified. The known benefits of the RLS algorithms carryover from linear systems to nonlinear ones; specifically, the convergence rate and misadjustment can be determined at design time.
Correlation clustering in general weighted graphs

We consider the following general correlation-clustering problem [N. Bansal, A. Blum, S. Chawla, Correlation clustering,  given a graph with real nonnegative edge weights and a 〈+〉/〈−〉 edge labelling, partition the vertices into clusters to minimize the total weight of cut 〈+〉 edges and uncut 〈−〉 edges. Thus, 〈+〉 edges with large weights (representing strong correlations between endpoints) encourage those endpoints to belong to a common cluster while 〈−〉 edges with large weights encourage the endpoints to belong to different clusters. In contrast to most clustering problems, correlation clustering specifies neither the desired number of clusters nor a distance threshold for clustering; both of these parameters are effectively chosen to be the best possible by the problem definition. Correlation clustering was introduced by Bansal et al. [Correlation clustering, , motivated by both document clustering and agnostic learning. They proved NP-hardness and gave constant-factor approximation algorithms for the special case in which the graph is complete (full information) and every edge has the same weight. We give an O(log n)-approximation algorithm for the general case based on a linear-programming rounding and the “region-growing” technique. We also prove that this linear program has a gap of (log n), and therefore our approximation is tight under this approach. We also give an O(r3)-approximation algorithm for Kr,r -minor-free graphs. On the other hand, we show that the problem is equivalent to minimum multicut, and therefore APX-hard and difficult to approximate better than (log n).
HYDRA: hybrid design for remote attestation (using a formally verified microkernel)

Remote Attestation (RA) allows a trusted entity (verifier) to securely measure internal state of a remote untrusted hardware platform (prover). RA can be used to establish a static or dynamic root of trust in embedded and cyber-physical systems. It can also be used as a building block for other security services and primitives, such as software updates and patches, verifiable deletion and memory resetting. There are three major types of RA designs: hardware-based, software-based, and hybrid, each with its own set of benefits and drawbacks.
 This paper presents the first hybrid RA design - called HYDRA - that builds upon formally verified software components that ensure memory isolation and protection, as well as enforce access control to memory and other resources. HYDRA obtains these properties by using the formally verified seL4 microkernel. (Until now, this was only attainable with purely hardware-based designs.) Using seL4 imposes fewer hardware requirements on the underlying microprocessor. Also, building upon a formally verified software component increases confidence in security of the overall design of HYDRA and its implementation. We instantiate HYDRA on two commodity hardware platforms and assess the performance and overhead of performing RA on such platforms via experimentation; we show that HYDRA can attest 10MB of memory in less than 250msec when using a Speck-based cryptographic checksum.
A new structure of universal motor using soft magnetic composites

The universal or ac commutator motor, widely used in hand tools and domestic appliances generally uses a two-pole stator with a concentrated winding and an armature with interlocked coils elements. The copper volume and the axial length of the end windings of such conventional structures are then usually very important. In this paper, the authors present a new universal motor structure based on an efficient use of the isotropic magnetic properties of the soft magnetic composites and on the concentrated winding technique. The stator core presents a claw-pole structure and the armature has a concentrated winding with several coils wound around the same tooth. With this new ac commutator motor structure, a reduction of the total volume by a ratio equal to 200% is obtained when compared to a classical universal motor structure with nearly identical performance.
Revisiting Random Binning Features: Fast Convergence and Strong Parallelizability

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features method gains much popularity due to its simplicity and direct reduction of nonlinear problem to a linear one. Different random feature functions have since been proposed to approximate a variety of kernel functions. Among them the Random Binning (RB) feature, proposed in the first random-feature paper [21], has drawn much less attention than the Random Fourier (RF) feature proposed also in . In this work, we observe that the RB features, with right choice of optimization solver, could be orders-of-magnitude more efficient than other random features and kernel approximation methods under the same requirement of accuracy. We thus propose the first analysis of RB from the perspective of optimization, which by interpreting RB as a Randomized Block Coordinate Descent in the infinite-dimensional space, gives a faster convergence rate compared to that of other random features. In particular, we show that by drawing R random grids with at least κ number of non-empty bins per grid in expectation, RB method achieves a convergence rate of O(1/κ R)), which not only sharpens its O(1/√R) rate from Monte Carlo analysis, but also shows a κ times speedup over other random features under the same analysis framework. In addition, we demonstrate another advantage of RB in the L1-regularized setting, where unlike other random features, a RB-based Coordinate Descent solver can be parallelized with guaranteed speedup proportional to κ. Our extensive experiments demonstrate the superior performance of the RB features over other random features and kernel approximation methods.
Rumors Detection in Sina Weibo Based on Text and User Characteristics

M414 such as Twitter and Sina Weibo have brought convenience to people in recent years, but there are a large number of internet rumors bring serious harm to us. Because of the fuzziness and concealment of rumors, the automatic identification of microblog rumors is very difficult, which brings great challenges to microblog's supervision. In this paper, for the text information of rumors, we construct a new emotional classification method, and use this method to analyze the emotional orientation of comments, and then calculate the proportion of negative emotional comments. Combined with user information at the same time, we analyze retweet-list and comment-list, and then judge whether there are authoritative users who participated in the spread of rumors. Moreover, we extract features from the information of user and calculate the user's reputation value. Finally, we use the training classifier to detect microblog rumors. Experimental results illustrate the efficacy and efficiency of the proposed new features in microblog rumors detection.
Reduction in gravitational torques of an industrial robot equipped with 2 DOF passive counterbalance mechanisms

In most 6 DOF robot arms, considerable amounts of gravitational torques due to the robot's own weight are applied to pitch joints of the robot, which causes most arms to use high capacity motors and speed reducers. A spring-based counterbalance mechanism can compensate for this gravitational torque, thus leading to a significant reduction in the effect of gravity. However, a simple installation of counterbalance mechanisms at each pitch joint does not work properly because the gravitational torque at each joint is dependent also on the other joints. To achieve multi-DOF counterbalancing, we propose a parallelogram linkage combined with dual counterbalance mechanisms, each being composed of a slider-crank mechanism and springs. Simulations and experimental results showed that the counterbalance robot arm based on the proposed counterbalance mechanisms effectively reduced the torques required to support the robot mass, thus allowing the prospective use of much smaller motors and speed reducers than traditional industrial robots.
The Collateral Damage of Internet Censorship by DNS Injection

Some ISPs and governments (most notably the Great Firewall of China) use DNS injection to block access to “unwanted” websites. The censorship tools inspect DNS queries near the ISP’s boundary routers for sensitive domain keywords and injecting forged DNS responses, blocking the users from accessing censored sites, such as twitter.com and facebook. com. Unfortunately this causes large scale collateral damage, affecting communication beyond the censored networks when outside DNS traffic traverses censored links. In this paper, we analyze the causes of the collateral damages comprehensively and measure the Internet to identify the injecting activities and their effect. We find 39 ASes in China injecting forged replies even for transit DNS traffic, and 26% of 43,000 measured open resolvers outside China, distributed in 109 countries, may suffer some collateral damage. Different from previous work, we find that most collateral damage arises from resolvers querying TLD name servers who’s transit passes through China rather than effects due to root servers (F, I, J) located in China.
ManyEyes: a Site for Visualization at Internet Scale

We describe the design and deployment of Many Eyes, a public Web site where users may upload data, create interactive visualizations, and carry on discussions. The goal of the site is to support collaboration around visualizations at a large scale by fostering a social style of data analysis in which visualizations not only serve as a discovery tool for individuals but also as a medium to spur discussion among users. To support this goal, the site includes novel mechanisms for end-user creation of visualizations and asynchronous collaboration around those visualizations. In addition to describing these technologies, we provide a preliminary report on the activity of our users.