Reviewerly/exHarmony huggingface dataset:
Efficient Task/Motion Planning for a Dual-arm Robot from Language Instructions and Cooking Images

When generating robot motions based on instructions such as cooking recipes, ambiguity of the instructions and lack of necessary information are problematic for the robot. To solve this problem, we propose an efficient motion planning approach for a dual-arm robot by constructing a graph repre-senting a motion sequence based on a recipe consisting of verbal instructions and cooking images. A functional unit is generated based on the linguistic instructions in the recipe. Since most recipes lack the necessary information for executing the motion, we first consider extracting the information about the cooking motion like cutting from the food images of the recipe and supplementing it. In addition, to supplement the actions that humans perform unconsciously, we generate functional units for actions not explicitly mentioned in the recipe based on the current situation of the cooking process, and then connect them to the functional units generated from the recipe. Moreover, during the connection we consider the motion of the robot's arms in parallel for an efficient execution of the recipe, similar to those of a human. Through experiments, we demonstrate that for a given recipe, the proposed method can be used to generate a cooking sequence with the supplementary information needed, and executed by a dual-arm robot. The results show that the proposed method is effective and can simplify robot teaching in cooking tasks.
Object-Relation Reasoning Graph for Action Recognition

Action recognition is a challenging task since the attributes of objects as well as their relationships change constantly in the video. Existing methods mainly use object-level graphs or scene graphs to represent the dynamics of objects and relationships, but ignore modeling the fine-grained relationship transitions directly. In this paper, we propose an Object-Relation Reasoning Graph (OR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> G) for reasoning about action in videos. By combining an object-level graph (OG) and a relation-level graph (RG), the proposed OR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> G catches the attribute transitions of objects and reasons about the relationship transitions between objects simultaneously. In addition, a graph aggregating module (GAM) is investigated by applying the multi-head edge-to-node message passing operation. GAM feeds back the information from the relation node to the object node and enhances the coupling between the object-level graph and the relation-level graph. Experiments in video action recognition demonstrate the effectiveness of our approach when compared with the state-of-the-art methods.
Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalization

Out-Of-Distribution generalization (OOD) is all about learning invariance against environmental changes. If the context (In this paper, the word “context” denotes any class-agnostic attributes such as color, texture and background. The formal definition can be found in Appendix, A.2.) in every class is evenly distributed, OOD would be trivial because the context can be easily removed due to an underlying principle: class is invariant to context. However, collecting such a balanced dataset is impractical. Learning on imbalanced data makes the model bias to context and thus hurts OOD. Therefore, the key to OOD is context balance. We argue that the widely adopted assumption in prior work—the context bias can be directly annotated or estimated from biased class prediction—renders the context incomplete or even incorrect. In contrast, we point out the ever-overlooked other side of the above principle: context is also invariant to class, which motivates us to consider the classes (which are already labeled) as the varying environments (The word “environments” [2] denotes the subsets of training data built by some criteria. In this paper, we take a class as an environment—our key idea.) to resolve context bias (without context labels). We implement this idea by minimizing the contrastive loss of intra-class sample similarity while assuring this similarity to be invariant across all classes. On benchmarks with various context biases and domain gaps, we show that a simple re-weighting based classifier equipped with our context estimation achieves state-of-the-art performance. We provide the theoretical justifications in Appendix and codes on Github: https://github.com/simpleshinobu/IRMCon .
COMPASS: Contrastive Multimodal Pretraining for Autonomous Systems

Learning representations that generalize across tasks and domains is challenging yet necessary for autonomous systems. Although task-driven approaches are appealing, de-signing models specific to each application can be difficult in the face of limited data, especially when dealing with highly variable multimodal input spaces arising from different tasks in different environments. We introduce the first general-purpose pretraining pipeline, COntrastive Multimodal Pretraining for AutonomouS Systems (COMPASS), to overcome the limitations of task-specific models and existing pretraining approaches. COMPASS constructs a multimodal graph by considering the essential information for autonomous systems and the proper-ties of different modalities. Through this graph, multimodal signals are connected and mapped into two factorized spatio-temporal latent spaces: a “motion pattern space” and a “current state space.” By learning from multimodal correspondences in each latent space, COMPASS creates state representations that models necessary information such as temporal dynamics, geometry, and semantics. We pretrain COMPASS on a large-scale multimodal simulation dataset TartanAir [1] and evaluate it on drone navigation, vehicle racing, and visual odometry tasks. The experiments indicate that COMPASS can tackle all three scenarios and can also generalize to unseen environments and real-world data. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Our code implementation can be found at https://github.com/microsoft/COMPASS
Few-Shot Class-Incremental Learning for 3D Point Cloud Objects

Few-shot class-incremental learning (FSCIL) aims to incrementally fine-tune a model (trained on base classes) for a novel set of classes using a few examples without forgetting the previous training. Recent efforts address this problem primarily on 2D images. However, due to the advancement of camera technology, 3D point cloud data has become more available than ever, which warrants considering FSCIL on 3D data. This paper addresses FSCIL in the 3D domain. In addition to well-known issues of catastrophic forgetting of past knowledge and overfitting of few-shot data, 3D FSCIL can bring newer challenges. For example, base classes may contain many synthetic instances in a realistic scenario. In contrast, only a few real-scanned samples (from RGBD sensors) of novel classes are available in incremental steps. Due to the data variation from synthetic to real, FSCIL endures additional challenges, degrading performance in later incremental steps. We attempt to solve this problem using Microshapes (orthogonal basis vectors) by describing any 3D objects using a pre-defined set of rules. It supports incremental training with few-shot examples minimizing synthetic to real data variation. We propose new test protocols for 3D FSCIL using popular synthetic datasets (ModelNet and ShapeNet) and 3D real-scanned datasets (ScanObjectNN and CO3D). By comparing state-of-the-art methods, we establish the effectiveness of our approach in the 3D domain. Code is available at: https://github.com/townim-faisal/FSCIL-3D .
A Method for Measuring the Inclination of Forgings Based on an Improved Optimization Algorithm for Fitting Ellipses

During a piercing or punching process, the center axis of a forging stock is required to be parallel to the direction of movement of the upsetting rod or piercing needle because any small tilt of the stock will cause uneven upsetting or uneven punching wall thickness. In order to detect the inclination of a stock during forging, a method for measuring the inclination of forgings based on an improved optimization algorithm for fitting ellipses is proposed in the work. Specifically, a laser rangefinder is used to measure the distances to the surface of the workpiece, which are converted into Section profile coordinates through coordinate transformation. Next, the randomly divided area and random sampling are presented for screening data points, making it possible to effectively eliminate the influence of abnormal values in the measurement points. Then, taking the point-to-ellipse orthogonal distance as the geometric model, we propose an improved particle swarm optimization (PSO) algorithm to solve the target ellipse parameters. Subsequently, we compare the fitting effect with four other contemporary advanced methods to verify the superiority of the algorithm in ellipse fitting. Finally, the three-dimensional linear equation of the center axis of the forging stock is obtained according to the center point of the long axis of the ellipse in the vertical direction, and the inclination angle of the forging stock is calculated according to the linear equation. The experimental measurement reveals that the error that occurred using the method is less than 0.05°.
Pushing the Envelope of Gradient Boosting Forests via Globally-Optimized Oblique Trees

Ensemble methods based on decision trees, such as Random Forests or boosted forests, have long been established as some of the most powerful, off-the-shelf machine learning models, and have been widely used in computer vision and other areas. In recent years, a specific form of boosting, gradient boosting (GB), has gained prominence. This is partly because of highly optimized implementations such as XGBoost or LightGBM, which incorporate many clever modifications and heuristics. However, one gaping hole remains unexplored in GB: the construction of individual trees. To date, all successful GB versions use axis-aligned trees trained in a suboptimal way via greedy recursive partitioning. We address this gap by using a more powerful type of trees (having hyperplane splits) and an algorithm that can optimize, globally over all the tree parameters, the objective function that GB dictates. We show, in several benchmarks of image and other data types, that GB forests of these stronger, well-optimized trees consistently exceed the test accuracy of axis-aligned forests from XGBoost, Light-GBM and other strong baselines. Further, this happens using many fewer trees and sometimes even fewer parameters overall.
Challenges and Trends in Federated Learning for Well-being and Healthcare

Currently, research in Artificial Intelligence, both in Machine Learning and Deep Learning, paves the way for promising innovations in several areas. In healthcare, especially, where large amounts of quantitative and qualitative data are transferred to support studies and early diagnosis and monitoring of any diseases, potential security and privacy issues cannot be underestimated. Federated learning is an approach where privacy issues related to sensitive data management can be significantly reduced, due to the possibility to train algorithms without exchanging data. The main idea behind this approach is that learning models can be trained in a distributed way, where multiple devices or servers with decentralized data samples can provide their contributions without having to exchange their local data. Recent studies provided evidence that prototypes trained by adopting Federated Learning strategies are able to achieve reliable performance, thus by generating robust models without sharing data and, consequently, limiting the impact on security and privacy. This work propose a literature overview of Federated Learning approaches and systems, focusing on its application for healthcare. The main challenges, implications, issues and potentials of this approach in the healthcare are outlined.
An Experimental Study of a Fuzzy Adaptive Emperor Penguin Optimizer for Global Optimization Problem

Emperor Penguin Optimizer (EPO) is a recently developed population-based meta-heuristic algorithm that simulates the huddling behavior of emperor penguins. Mixed results have been observed on the performance of EPO in solving general optimization problems. Within the EPO, two parameters need to be tuned (namely f and l) to ensure a good balance between exploration (i.e., roaming unknown locations) and exploitation (i.e., manipulating the current known best). Since the search contour varies depending on the optimization problem, the tuning of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> is problem-dependent, and there is no one-size-fits-all approach. To alleviate these problems, an adaptive mechanism can be introduced in EPO. This paper proposes a fuzzy adaptive variant of EPO, namely Fuzzy Adaptive Emperor Penguin Optimizer (FAEPO), to solve this problem. As the name suggests, FAEPO can adaptively tune the parameters <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> throughout the search based on three measures (i.e., quality, success rate, and diversity of the current search) via fuzzy decisions. A test suite of twelve optimization benchmark test functions and three global optimization problems (Team Formation Optimization-TFO, Low Autocorrelation Binary Sequence-LABS, and Modified Condition/Decision Coverage-MC/DC test case generation) were solved using the proposed algorithm. The respective solution results of the benchmark meta-heuristic algorithms were compared. The experimental results demonstrate that FAEPO significantly improved the performance of its predecessor (EPO) and gives superior performance against the competing meta-heuristic algorithms including an improved variant of EPO (IEPO).
OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images

Enhancing the robustness of vision algorithms in real-world scenarios is challenging. One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors. We introduce OOD-CV , a benchmark dataset that includes out-of-distribution examples of 10 object categories in terms of pose, shape, texture, context and the weather conditions, and enables benchmarking models for image classification, object detection, and 3D pose estimation. In addition to this novel dataset, we contribute extensive experiments using popular baseline methods, which reveal that: 1) Some nuisance factors have a much stronger negative effect on the performance compared to others, also depending on the vision task. 2) Current approaches to enhance robustness have only marginal effects, and can even reduce robustness. 3) We do not observe significant differences between convolutional and transformer architectures. We believe our dataset provides a rich testbed to study robustness and will help push forward research in this area.
Triangulating Rebound Attack on AES-like Hashing

The rebound attack was introduced by Mendel et al. at FSE 2009 to fulfill a heavy middle round of a differential path for free, utilizing the degree of freedom from states. The inbound phase was extended to 2 rounds by the Super-Sbox technique invented by Lamberger et al. at ASIACRYPT 2009 and Gilbert and Peyrin at FSE 2010. In ASIACRYPT 2010, Sasaki et al. further reduced the requirement of memory by introducing the non-full-active Super-Sbox. In this paper, we further develop this line of research by introducing Super-Inbound, which is able to connect multiple 1-round or 2-round (non-full-active) Super-Sbox inbound phases by utilizing fully the degrees of freedom from both states and key, yet without the use of large memory. This essentially extends the inbound phase by up to 3 rounds. We applied this technique to find classic or quantum collisions on several AES-like hash functions, and improved the attacked round number by 1 to 5 in targets including AES-128 and SKINNY hashing modes, Saturnin-Hash, and Grøstl-512. To demonstrate the correctness of our attacks, the semi-free-start collision on 6-round AES-128-MMO/MP with estimated time complexity $$2^{24}$$ in classical setting was implemented and an example pair was found instantly on a standard PC.
On Leveraging FemtoClouds for Federated Learning

The massive amount of data generated by mobile/IoT devices worldwide has been both a motivation and a key enabler for Machine Learning (ML)-based applications. Cloud and Edge platforms have been traditionally leveraged as underlying systems where data for ML applications is processed using “Centralized ML.” Federated Learning (FL) was introduced as an alternative solution that provides comparable accuracy, to centralized-ML, while maintaining user privacy. FL allows training data on its source device and only model updates can leave the device for learning aggregation at the Cloud/Edge. However, current FL solutions have limitations that originate from the training process and current system implementations. Hence, in this paper, we propose leveraging FemtoClouds (i.e. ensembles of co-located mobile/IoT devices) as a complementary system to existing FL approaches. We discuss and assess the benefits that FemtoClouds introduce for both the FL training algorithms, and end devices. These benefits include improvements in training accuracy, convergence speed, training time, and energy efficiency. We finally discuss challenges and future research directions that need to be addressed to realize the benefits of Femto Clouds-based FL.
An Experimental Analysis of Optimal Hybrid Word Embedding Methods for Text Classification Using a Movie Review Dataset

HIGHLIGHTS Proposed Hybrid Word Embedding (HWE) models for Efficient Text classification. Data Sparsity issue is reduced using WordNet repository along with proposed model. Optimal model is derived based on the Performance evaluation on the model.
Quantifying non-Markovian Memory in a Superconducting Quantum Computer

We measure and quantify non-Markovian effects in IBM's Quantum Experience. Specifically, we analyze the temporal correlations in a sequence of gates by characterizing the performance of a gate conditioned on the gate that preceded it. With this method, we estimate (i) the size of fluctuations in the performance of a gate, i.e., errors due to non-Markovianity; (ii) the length of the memory; and (iii) the total size of the memory. Our results strongly indicate the presence of non-trivial non-Markovian effects in almost all gates in the universal set. However, based on our findings, we discuss the potential for cleaner computation by adequately accounting the non-Markovian nature of the machine.
Semi-supervised Learning of Optical Flow by Flow Supervisor

A training pipeline for optical flow CNNs consists of a pretraining stage on a synthetic dataset followed by a fine tuning stage on a target dataset. However, obtaining ground truth flows from a target video requires a tremendous effort. This paper proposes a practical fine tuning method to adapt a pretrained model to a target dataset without ground truth flows, which has not been explored extensively. Specifically, we propose a flow supervisor for self-supervision, which consists of parameter separation and a student output connection. This design is aimed at stable convergence and better accuracy over conventional self-supervision methods which are unstable on the fine tuning task. Experimental results show the effectiveness of our method compared to different self-supervision methods for semi-supervised learning. In addition, we achieve meaningful improvements over state-of-the-art optical flow models on Sintel and KITTI benchmarks by exploiting additional unlabeled datasets. Code is available at https://github.com/iwbn/flow-supervisor .
Geochemometrics: Petrology of Quaternary Volcanism in the Central Mexican Volcanic Belt

Geochemometrics is a geochemistry systematized branch resulting from the combination of geochemical principles with statistical, mathematical, and computational tools to explain the nature and processes of Earth systems. In this work, several geochemometric techniques, as well as traditional mineralogical and geochemical tools, have been used contributing to petrological model for the Quaternary volcanism in the central Mexican Volcanic Belt (MVB). A wide database of 571 records of major and trace elements and isotopic Sr–Nd ratios has been prepared. Compiled information covers bibliographic sources published during the 1965–2020 period, as well as new petrographic and chemical data. Mineralogical and geochemical criteria have been used to divide the central MVB database in five petrologic groups: (a) G1: basic rocks, (b) G2: high-Mg intermediate rocks, (c) G3: low-Mg intermediate rocks, (d) G4: acid rocks, and (e) G5: disequilibrium rocks. Each group has been evaluated in terms of rock classification, spatial distribution, approximation of intensive properties (pressure and temperature), magmatic process inference, and tectonic setting. Petrogenesis of extensional-related G1 and G2 rocks has been linked to upper mantle processes at the Moho level. In contrast, the geochemometric evaluation has confirmed that the subduction-related Mexican continental crust has been involved in the origin and evolution of G3–G5 rocks.
RVSL: Robust Vehicle Similarity Learning in Real Hazy Scenes Based on Semi-supervised Learning

Recently, vehicle similarity learning, also called re-identification (ReID), has attracted significant attention in computer vision. Several algorithms have been developed and obtained considerable success. However, most existing methods have unpleasant performance in the hazy scenario due to poor visibility. Though some strategies are possible to resolve this problem, they still have room to be improved due to the limited performance in real-world scenarios and the lack of real-world clear ground truth. Thus, to resolve this problem, inspired by CycleGAN, we construct a training paradigm called RVSL which integrates ReID and domain transformation techniques. The network is trained on semi-supervised fashion and does not require to employ the ID labels and the corresponding clear ground truths to learn hazy vehicle ReID mission in the real-world haze scenes. To further constrain the unsupervised learning process effectively, several losses are developed. Experimental results on synthetic and real-world datasets indicate that the proposed method can achieve state-of-the-art performance on hazy vehicle ReID problems. It is worth mentioning that although the proposed method is trained without real-world label information, it can achieve competitive performance compared to existing supervised methods trained on complete label information.