Reviewerly/exHarmony huggingface dataset:
UFO: Unified Feature Optimization

This paper proposes a novel Unified Feature Optimization (UFO) paradigm for training and deploying deep models under real-world and large-scale scenarios, which requires a collection of multiple AI functions. UFO aims to benefit each single task with a large-scale pretraining on all tasks. Compared with existing foundation models, UFO has two points of emphasis, i.e., relatively smaller model size and NO adaptation cost: 1) UFO squeezes a wide range of tasks into a moderate-sized unified model in a multi-task learning manner and further trims the model size when transferred to down-stream tasks. 2) UFO does not emphasize transfer to novel tasks. Instead, it aims to make the trimmed model dedicated for one or more already-seen task. To this end, it directly selects partial modules in the unified model, requiring completely NO adaptation cost. With these two characteristics, UFO provides great convenience for flexible deployment, while maintaining the benefits of large-scale pretraining. A key merit of UFO is that the trimming process not only reduces the model size and inference consumption, but also even improves the accuracy on certain tasks. Specifically, UFO considers the multi-task training and brings a two-fold impact on the unified model: some closely-related tasks have mutual benefits, while some tasks have conflicts against each other. UFO manages to reduce the conflicts and preserve the mutual benefits through a novel Network Architecture Search (NAS) method. Experiments on a wide range of deep representation learning tasks (i.e., face recognition, person re-identification, vehicle re-identification and product retrieval) show that the model trimmed from UFO achieves higher accuracy than its single-task-trained counterpart and yet has smaller model size, validating the concept of UFO. Besides, UFO also supported the release of 17 billion parameters computer vision (CV) foundation model which is the largest CV model in the industry. Code: https://github.com/PaddlePaddle/VIMER/tree/main/UFO .
FedRelay: Federated Relay Learning for 6G Mobile Edge Intelligence

Federated learning (FL) is a promising training paradigm to achieve ubiquitous intelligence for future 6G communication systems. However, it is challenging to apply FL in 6G-enabled edge system since decentralized training consumes considerable energy and mobile devices are mostly battery-powered and resource-constrained. The intensive computation and communication cost of local updates accumulated by hundreds of global rounds bring about the energy bottleneck, which is exacerbated when the data is non identically and independently distributed (non-IID). To address this issue, we propose FedRelay, a generic multi-flow relay learning framework in which local updates are performed relay-by-relay in the training flow via model propagation. We also present a decentralized relay selection protocol that takes advantage of the diversity of cooperative communication networks. Following that, we investigate a FedRelay optimization problem to simultaneously minimize the energy consumption of local updates and alleviate the global non-IIDness. Technically, an approximation algorithm is proposed to jointly optimize computation frequency and transmission power, thus reducing the local training overhead. We further regulate the training topology of each flow by proposing a greedy relay policy that encourages effective information exchange among devices. Experiment results show that, compared to state-of-the-art federated learning algorithms, our learning framework can save up to 5 times the total energy required to achieve a reasonable global test accuracy.
Clustering Plotted Data by Image Segmentation

Clustering is a popular approach to detecting patterns in unlabeled data. Existing clustering methods typically treat samples in a dataset as points in a metric space and compute distances to group together similar points. In this paper, we present a different way of clustering points in 2-dimensional space, inspired by how humans cluster data: by training neural networks to perform instance segmentation on plotted data. Our approach, Visual Clustering, has several advantages over traditional clustering algorithms: it is much faster than most existing clustering algorithms (making it suitable for very large datasets), it agrees strongly with human intuition for clusters, and it is by default hyperparameter free (although additional steps with hyperparameters can be introduced for more control of the algorithm). We describe the method and compare it to ten other clustering methods on synthetic data to illustrate its advantages and disadvantages. We then demonstrate how our approach can be extended to higher-dimensional data and illustrate its performance on real-world data. Our implementation of Visual Clustering is publicly available as a python package that can be installed and used on any dataset in a few lines of code <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://hithub.com/tareknaous/visual-clustering. A demo on synthetic datasets is provided <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> https://huggingface.co/spaces/CVPR/visual-clustering.
Architecture and Methodology of Unit Testing Embedding Pair-Wise Mode for Small Team

In this paper, the new organization for unit testing embedding pair-wise mode is proposed with the core thought focused on the cooperation of programmer and tester by "cross-testing".The typical content of unit testing for the new organizing mode should have three aspects, including self-checking, cross-testing and independent-testing.For cross-testing, executing "pair-wise" mode, mainly tackles data testing, function testing and state testing, which function testing must be done by details and state testing must be considered for completeness.With the specializing of independent-testing, it should be taken as more rigid testing without arbitrariness.Consequently, strategy and measure are addressed for data testing focusing on boundary testing and function/state testing.And organizing method of procedure and key points of tackling unit testing are investigated for the new organizing mode.In order to assess the validity of our study and approach, a series of actual examples are demonstrated for GUI software.The result indicates that the execution of unit testing for the new organizing mode is effective and applicable.
Fast Detection of Moving Traffic Participants in LiDAR Point Clouds by using Particles augmented with Free Space Information

To navigate safely, it is essential for a robot to detect all kinds of moving objects that could possibly interfere with the own trajectory. For common object classes, like cars, regular pedestrians, and trucks, there are large scale datasets as well as corresponding machine learning techniques, which provide remarkable results in commonly available detection benchmarks. A big challenge that remains, are less frequent classes, which are not part of a dataset in a sufficient number and variation. Dynamic occupancy grids are a promising approach for detection of moving objects in point clouds since they impose only a few assumptions about the objects' appearance and shape. Typically, they use particle filters to detect motion of occupancy in the grid. Existing approaches, however, often generate false positives at long obstacles because particles move along them. Therefore we propose a highly efficient approach, which performs the classification in a more structured and conservative way by making extensive use of available free space information. As a result, much less false positives are generated while the number of false negatives remains low. Our approach can be used to complement CNN-based object detections in order to detect both, frequent and uncommon object classes reliably. By using polar data structures that match the polar measurement principle, we are able to process even large point clouds of modern LiDARs with 128 lasers efficiently.
Toward Interactive Music Generation: A Position Paper

Music generation using deep learning has received considerable attention in recent years.Researchers have developed various generative models capable of cloning musical conventions, comprehending the musical corpora, and generating new samples based on the learning outcome.Although the samples generated by these models are persuasive, they often lack musical structure and creativity.Moreover, a vanilla end-to-end approach, which deals with all levels of music representation at once, does not offer human-level control and interaction during the learning process, leading to constrained results.Indeed, music creation is a recurrent process that follows some principles by a musician, where various musical features are reused or adapted.On the other hand, a musical piece adheres to a musical style, breaking down into precise concepts of timbre style, performance style, composition style, and the coherency between these aspects.Here, we study and analyze the current advances in music generation using deep learning models through different criteria.We discuss the shortcomings and limitations of these models regarding interactivity and adaptability.Finally, we draw the potential future research direction addressing multi-agent systems and reinforcement learning algorithms to alleviate these shortcomings and limitations.
Supporting Multi-User Co-located Training for Industrial Procedures through Immersive Virtual Reality (VR) and a Large-scale Display

Virtual Reality (VR) has enormous potential to support training scenarios in the most diverse areas of application. In industry scenarios, it can allow operators to acquire new skills and higher levels of confidence before moving to real-world activities. Trainees must immerse themselves within the task space, which means using a Head-Mounted Display (HMD) to experience a virtual representation of the environment. Nevertheless, such activities are mostly isolated, removing the ability to perform training sessions in a collaborative manner with a trainer and other trainees that are learning how to perform identical tasks. This work presents a multi-user colocated training prototype based on VR, wireless controllers/pointer and a large-scale display. By using a shared immersive display, while a trainee is immersed in the VR environment, the trainer can visualize the same content and explain what to do, as well as interact, highlight areas of interest or correct possible mistakes. This enables other's trainees to watch and learn from each other training sessions, better preparing themselves. As use-case, we took inspiration from a real-life collaborative scenario thanks to an on-going research project with partners from the industry sector.
Object Detection Algorithms: A Comparison

Object detection, whose main task is to detect objects in a picture to determine the type, location, and scene to which they belong, has become one of the most central problems in computer vision. Object detection is currently broadly divided into two implementations, one-stage detector will get the prediction results directly from the image, two-stage detector is also known as Region-based, because they will regionally generate the image and then submit it to the convolutional neural network. These two approaches have their own advantages and disadvantages, and this paper will study their characteristics and summarize them with examples. This paper summarizes a total of five outstanding works on these two approaches, including YOLO, ConerNet, R-CNN, SPPNet, and Auto-FPN. The summary of these five efforts may help others better understand the field.
AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networks

We propose a new algorithm for training deep neural networks (DNNs) with binary weights. In particular, we first cast the problem of training binary neural networks (BiNNs) as a bilevel optimization instance and subsequently construct flexible relaxations of this bilevel program. The resulting training method shares its algorithmic simplicity with several existing approaches to train BiNNs, in particular with the straight-through gradient estimator successfully employed in BinaryConnect and subsequent methods. Infact, our proposed method can be interpreted as an adaptive variant of the original straight-through estimator that conditionally (but not always) acts like a linear mapping in the backward pass of error propagation. Experimental results demonstrate that our new algorithm offers favorable performance compared to existing approaches. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> This work was partially supported by theWallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
Research on fire rescue path optimization of unmanned equipment based on improved Slime mould Algorithm

An improved slime Mould Algorithm is proposed to solve the problem of optimal path planning for unmanned equipment in fire rescue. By introducing mutation mechanism and dynamic weight coefficient, the problems of slow convergence and low optimization precision of standard SMA algorithm are solved, and the reinforcement learning method is introduced to traverse the whole situation and search for the next optimal solution, in each iteration, slime mould algorithm is used for a local optimization to improve the quality of the solution. The simulation results show that the improved SMA algorithm has high precision and fast convergence speed, which is better than the genetic algorithm and another heuristic under the same conditions At the same time, the improved SMA algorithm is applied to the optimization of fire rescue route, which can effectively plan the rescue route and improve the rescue efficiency.
Machine Learning Resource Optimization Enabled by Cross Layer Monitoring

In this paper, we introduce a novel architecture and its open-source implementation that exploits the monitoring data from heterogeneous resources and uses them to train machine learning models, which can be used for dynamic resource management optimization. The existence of such a solution is extremely important for Service Providers (SP) as it can lead to the optimal use of their physical and virtual infrastructures avoiding potential waste of resources due to overdesign while at the same time it can ensure that the required Quality of Service (QoS) levels are met. The proposed solution is validated in two real-life services showing very good accuracy in predicting the required resources in both cases for a large number of operational scenarios.
Master of All: Simultaneous Generalization of Urban-Scene Segmentation to All Adverse Weather Conditions

Computer vision systems for autonomous navigation must generalize well in adverse weather and illumination conditions expected in the real world. However, semantic segmentation of images captured in such conditions remains a challenging task for current state-of-the-art (SOTA) methods trained on broad daylight images, due to the associated distribution shift. On the other hand, domain adaptation techniques developed for the purpose rely on the availability of the source data, (un)labeled target data and/or its auxiliary information (e.g., GPS). Even then, they typically adapt to a single(specific) target domain(s). To remedy this, we propose a novel, fully test time, adaptation technique, named Master of ALL (MALL), for simultaneous generalization to multiple target domains. MALL learns to generalize on unseen adverse weather images from multiple target domains directly at the inference time. More specifically, given a pre-trained model and its parameters, MALL enforces edge consistency prior at the inference stage and updates the model based on (a) a single test sample at a time (MALL-sample), or (b) continuously for the whole test domain (MALL-domain). Not only the target data, MALL also does not need access to the source data and thus, can be used with any pre-trained model. Using a simple model pre-trained on daylight images, MALL outperforms specially designed adverse weather semantic segmentation methods, both in domain generalization and test-time adaptation settings. Our experiments on foggy, snow, night, cloudy, overcast, and rainy conditions demonstrate the target domain-agnostic effectiveness of our approach. We further show that MALL can improve the performance of a model on an adverse weather condition, even when the model is already pre-trained for the specific condition.
RCA: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning

Under shared autonomy, wheelchair users expect vehicles to provide safe and comfortable rides while following users' high-level navigation plans. To find such a path, vehicles negotiate with different terrains and assess their traversal difficulty. Most prior works model surroundings either through geometric representations or semantic classifications, which do not reflect perceived motion intensity and ride comfort in downstream navigation tasks. We propose to model ride comfort explicitly in traversability analysis using proprioceptive sensing. We develop a self-supervised learning framework to predict traversability costmap from first-person-view images by leveraging vehicle states as training signals. Our approach estimates how the vehicle would “feel” if traversing over based on terrain appearances. We then show our navigation system provides human-preferred ride comfort through robot experiments together with a human evaluation study. The project could be found at https://sites.google.com/view/rca-navigation.
Sign language detection and conversion to text using CNN and OpenCV

It becomes very difficult to converse with deaf and mute people, so we need a language structure to understand what they are trying to say. Language is a barrier in between normal people and mute people. Our aim is to create an interface that convert sign language to the text by which everyone can understand the sign language of the peoples who are unable to speak. For this purpose, we will train a model to recognize hand gestures. The model can be trained using KNN, support vector machine, Logistic regression and CNN. But CNN is more accurate and effective in case of image recognition, that’s why we have train our model using CNN algorithm. After training the model on the American sign language dataset we can predict the text with good accuracy, but the model fails when we use OpenCV to recognize hand gestures. To further improve the model, we have created our own dataset to train the model using the dataset of images captured and then predict the text by taking the input from webcam. Also integrate the inputs to black and white after background subtraction. This will recognize hand gestures more accurately.
Deep Learning based Underwater Acoustic OFDM Receiver with Joint Channel Estimation and Signal Detection

This paper presents a deep learning (DL) based underwater acoustic (UWA) orthogonal frequency division multiplexing (OFDM) receiver, which exploits both DL and expert knowledge of block-based signal processing methods in wireless communication to improve system performance and interpretability. The proposed receiver jointly employs skip connection (SC) convolutional neural network (CNN) for channel estimation and bi-directional long short-term memory (BiLSTM) network for signal detection, abbreviated as SCBNet. Specifically, the channel estimation subnet is designed with SC CNN to utilize the thought of image super-resolution to reconstruct the entire channel frequency response of all subcarriers. The signal detection subnet is designed with BiLSTM to extract the correlations of received sequential data for signal detection. The proposed SCBNet was evaluated by experimental data, and the results have demonstrated that the SCBNet has the lowest BER performance compared with traditional linear algorithms, DL-based black-box receiver, and ComNet receiver.
Federated Meta Embedding Concept Stock Recommendation

Mining relevant stocks given a trending topic/concept in capital markets is an application with significant economic and societal impacts. Previous concept stock recommendation system mines concept stocks only from public social media like financial news. On stock forums, investors discuss emerging concepts and stocks by using forum comments, which are unneglectable resources to capture trending concept stocks accurately and timely. However, the comment data from a single forum is insufficient to build a high-quality recommendation system. The forums are data silos protected by privacy regulations, and their comments are still underutilized. In this paper, we propose a federated concept stock recommendation baseline and an optimized method that both leverage the private forum comments and public social media without compromising privacy regulations. Our baseline, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , Federated Meta Embedding (FedME), is built upon the federated learning framework and learns a concept-stock embedding jointly from private and public data. Our optimized method, Federated Graph Meta Embedding (FedGME), improves FedME by using a graph to combine two sources of embeddings and additional human experts' concept-stock knowledge. Empirically, the experiments on two concept stock datasets show that FedME and FedGME substantially improve the performance of recommendation. Our methods provide practical guidance on privacy-preserving FinTech applications.
Multi-Target Detection of Table Tennis Video Based on CNN and Yolov3

In order to explore the effect of convolutional neural network (CNN) on the detection of athletes and balls in table tennis, and to solve the problems of low accuracy and weak generalization ability of table tennis athletes training data, this paper analyzed videos of table tennis athletes.For the detection of multiple targets in the videos, including athletes and balls, we used Yolov3 as a deep learning framework, and CNN as an automatic detection method when processing images.We trained and test the video data to improve the stability and accuracy of target detection, through modifying its model on the basis of the Yolov3 model.Finally, we detect the movement trajectories of athletes and balls in table tennis videos stably, and the accuracy is above 0.8.
Autonomous learning of multiple curricula with non-stationary interdependencies

Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing artificial agents to acquire a wide repertoire of goals and motor skills without the necessity of specific assignments. Leveraging intrinsic motivations, different works have developed systems that can autonomously allocate training time amongst different goals to maximise their overall competence. However, only few works in the field of intrinsically motivated open-ended learning focus on scenarios where goals have interdependent relations, and even fewer tackle scenarios involving non-stationary interdependencies. Building on previous works, we propose a new hierarchical architecture (H-GRAIL) that selects its own goals on the basis of intrinsic motivations and treats curriculum learning of interdependent tasks as a Markov Decision Process. Moreover, we provide H-GRAIL with a novel mechanism that allows the system to self-regulate its exploratory behaviour and cope with the non-stationarity of the dependencies between goals. The system is tested in a simulated and real robotic environment with different experimental scenarios involving interdependent tasks.