OpenRaiser/Intern-Atlas huggingface dataset:
Speaker-Adapted End-to-End Visual Speech Recognition for Continuous Spanish
Venue: arXiv cs.CV 2023
Year: 2023

Different studies have shown the importance of visual cues throughout the speech perception process. In fact, the development of audiovisual approaches has led to advances in the field of speech technologies. However, although noticeable results have recently been achieved, visual speech recognition remains an open research problem. It is a task in which, by dispensing with the auditory sense, challenges such as visual ambiguities and the complexity of modeling silence must be faced. Nonetheless, some of these challenges can be alleviated when the problem is approached from a speaker-dependent perspective. Thus, this paper studies, using the Spanish LIP-RTVE database, how the estimation of specialized end-to-end systems for a specific person could affect the quality of speech recognition. First, different adaptation strategies based on the fine-tuning technique were proposed. Then, a pre-trained CTC/Attention architecture was used as a baseline throughout our experiments. Our findings showed that a two-step fine-tuning process, where the VSR system is first adapted to the task domain, provided significant improvements when the speaker adaptation was addressed. Furthermore, results comparable to the current state of the art were reached even when only a limited amount of data was available.
Convolutional Neural Networks for Neuroimaging in Parkinson's Disease: Is Preprocessing Needed?
Venue: arXiv cs.CV 2023
Year: 2023

Spatial and intensity normalization are nowadays a prerequisite for neuroimaging analysis. Influenced by voxel-wise and other univariate comparisons, where these corrections are key, they are commonly applied to any type of analysis and imaging modalities. Nuclear imaging modalities such as PET-FDG or FP-CIT SPECT, a common modality used in Parkinson's Disease diagnosis, are especially dependent on intensity normalization. However, these steps are computationally expensive and furthermore, they may introduce deformations in the images, altering the information contained in them. Convolutional Neural Networks (CNNs), for their part, introduce position invariance to pattern recognition, and have been proven to classify objects regardless of their orientation, size, angle, etc. Therefore, a question arises: how well can CNNs account for spatial and intensity differences when analysing nuclear brain imaging? Are spatial and intensity normalization still needed? To answer this question, we have trained four different CNN models based on well-established architectures, using or not different spatial and intensity normalization preprocessing. The results show that a sufficiently complex model such as our three-dimensional version of the ALEXNET can effectively account for spatial differences, achieving a diagnosis accuracy of 94.1% with an area under the ROC curve of 0.984. The visualization of the differences via saliency maps shows that these models are correctly finding patterns that match those found in the literature, without the need of applying any complex spatial normalization procedure. However, the intensity normalization -- and its type -- is revealed as very influential in the results and accuracy of the trained model, and therefore must be well accounted.
Crowd management, crime detection, work monitoring using aiml
Venue: arXiv cs.CV 2023
Year: 2023

This research endeavors to harness the potential of existing Closed-Circuit Television (CCTV) networks for a comprehensive approach to crowd management, crime prevention, and workplace monitoring through the integration of Artificial Intelligence (AI) and Machine Learning (ML) technologies. The primary objective is to develop and implement advanced algorithms capable of real-time analysis of video feeds, enabling the identification and assessment of crowd dynamics, early detection of potential criminal activities, and continuous monitoring of workplace environments. By leveraging AI/ML, the project aims to optimize surveillance capabilities, thereby enhancing public safety measures and improving organizational productivity. This initiative underscores the transformative impact that intelligent video analytics can have on existing infrastructure, mitigating the need for extensive system overhauls while significantly advancing security and operational efficiency.
Empirical Validation of the Impedance-Based RIS Channel Model in an Indoor Scattering Environment
Venue: arXiv cs.IT 2023
Year: 2023

Ensuring the precision of channel modeling plays a pivotal role in the development of wireless communication systems, and this requirement remains a persistent challenge within the realm of networks supported by Reconfigurable Intelligent Surfaces (RIS). Achieving a comprehensive and reliable understanding of channel behavior in RIS-aided networks is an ongoing and complex issue that demands further exploration. In this paper, we empirically validate a recently-proposed impedance-based RIS channel model that accounts for the mutual coupling at the antenna array and precisely models the presence of scattering objects within the environment as a discrete array of loaded dipoles. To this end, we exploit real-life channel measurements collected in an office environment to demonstrate the validity of such a model and its applicability in a practical scenario. Finally, we provide numerical results demonstrating that designing the RIS configuration based upon such model leads to superior performance as compared to reference schemes.
Polyhedral Object Recognition by Indexing
Venue: arXiv cs.CV 2023
Year: 2023

In computer vision, the indexing problem is the problem of recognizing a few objects in a large database of objects while avoiding the help of the classical image-feature-to-object-feature matching paradigm. In this paper we address the problem of recognizing 3-D polyhedral objects from 2-D images by indexing. Both the objects to be recognized and the images are represented by weighted graphs. The indexing problem is therefore the problem of determining whether a graph extracted from the image is present or absent in a database of model graphs. We introduce a novel method for performing this graph indexing process which is based both on polynomial characterization of binary and weighted graphs and on hashing. We describe in detail this polynomial characterization and then we show how it can be used in the context of polyhedral object recognition. Next we describe a practical recognition-by-indexing system that includes the organization of the database, the representation of polyhedral objects in terms of 2-D characteristic views, the representation of this views in terms of weighted graphs, and the associated image processing. Finally, some experimental results allow the evaluation of the system performance.
MathGloss: Building mathematical glossaries from text
Venue: arXiv cs.CL 2023
Year: 2023

MathGloss is a project to create a knowledge graph (KG) for undergraduate mathematics from text, automatically, using modern natural language processing (NLP) tools and resources already available on the web. MathGloss is a linked database of undergraduate concepts in mathematics. So far, it combines five resources: (i) Wikidata, a collaboratively edited, multilingual knowledge graph hosted by the Wikimedia Foundation, (ii) terms covered in mathematics courses at the University of Chicago, (iii) the syllabus of the French undergraduate mathematics curriculum which includes hyperlinks to the automated theorem prover Lean 4, (iv) MuLiMa, a multilingual dictionary of mathematics curated by mathematicians, and (v) the nLab, a wiki for category theory also curated by mathematicians. MathGloss's goal is to bring together resources for learning mathematics and to allow every mathematician to tailor their learning to their own preferences. Moreover, by organizing different resources for learning undergraduate mathematics alongside those for learning formal mathematics, we hope to make it easier for mathematicians and formal tools (theorem provers, computer algebra systems, etc) experts to "understand" each other and break down some of the barriers to formal math.
Informative Path Planning with Limited Adaptivity
Venue: arXiv cs.DS 2023
Year: 2023

We consider the informative path planning ($\mathtt{IPP}$) problem in which a robot interacts with an uncertain environment and gathers information by visiting locations. The goal is to minimize its expected travel cost to cover a given submodular function. Adaptive solutions, where the robot incorporates all available information to select the next location to visit, achieve the best objective. However, such a solution is resource-intensive as it entails recomputing after every visited location. A more practical approach is to design solutions with a small number of adaptive "rounds", where the robot recomputes only once at the start of each round. In this paper, we design an algorithm for $\mathtt{IPP}$ parameterized by the number $k$ of adaptive rounds, and prove a smooth trade-off between $k$ and the solution quality (relative to fully adaptive solutions). We validate our theoretical results by experiments on a real road network, where we observe that a few rounds of adaptivity suffice to obtain solutions of cost almost as good as fully-adaptive ones.
Short Voting Codes For Practical Code Voting
Venue: arXiv cs.CR 2023
Year: 2023

To preserve voter secrecy on untrusted voter devices we propose to use short voting codes. This ensures voting codes remain practical even if the voter is able to select multiple voting choices. We embed the mechanism in a protocol that avoids complex cryptography in both the setup and the voting phase and relies only on standard cryptographic primitives. Trusting the setup, and one out of multiple server components, the protocol provides vote secrecy, cast-as-intended, recorded-as-cast, tallied-as-recorded, eligibility and universal verifiability.
Development of a Legal Document AI-Chatbot
Venue: arXiv cs.AI 2023
Year: 2023

With the exponential growth of digital data and the increasing complexity of legal documentation, there is a pressing need for efficient and intelligent tools to streamline the handling of legal documents.With the recent developments in the AI field, especially in chatbots, it cannot be ignored as a very compelling solution to this problem.An insight into the process of creating a Legal Documentation AI Chatbot with as many relevant features as possible within the given time frame is presented.The development of each component of the chatbot is presented in detail.Each component's workings and functionality has been discussed.Starting from the build of the Android app and the Langchain query processing code till the integration of both through a Flask backend and REST API methods.
SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction
Venue: arXiv cs.CV 2023
Year: 2023

3D occupancy prediction is an important task for the robustness of vision-centric autonomous driving, which aims to predict whether each point is occupied in the surrounding 3D space. Existing methods usually require 3D occupancy labels to produce meaningful results. However, it is very laborious to annotate the occupancy status of each voxel. In this paper, we propose SelfOcc to explore a self-supervised way to learn 3D occupancy using only video sequences. We first transform the images into the 3D space (e.g., bird's eye view) to obtain 3D representation of the scene. We directly impose constraints on the 3D representations by treating them as signed distance fields. We can then render 2D images of previous and future frames as self-supervision signals to learn the 3D representations. We propose an MVS-embedded strategy to directly optimize the SDF-induced weights with multiple depth proposals. Our SelfOcc outperforms the previous best method SceneRF by 58.7% using a single frame as input on SemanticKITTI and is the first self-supervised work that produces reasonable 3D occupancy for surround cameras on nuScenes. SelfOcc produces high-quality depth and achieves state-of-the-art results on novel depth synthesis, monocular depth estimation, and surround-view depth estimation on the SemanticKITTI, KITTI-2015, and nuScenes, respectively. Code: https://github.com/huang-yh/SelfOcc.
Frequency Analysis with Multiple Kernels and Complete Dictionary
Venue: arXiv cs.IT 2023
Year: 2023

In signal analysis, among the effort of seeking for efficient representations of a signal into the basic ones of meaningful frequencies, to extract principal frequency components, consecutively one after another or $n$ at one time, is a fundamental strategy. For this goal, we define the concept of mean-frequency and develop the related frequency decomposition with the complete Szegö kernel dictionary, the latter consisting of the multiple kernels, being defined as the parameter-derivatives of the Szegö kernels. Several major energy matching pursuit type sparse representations, including greedy algorithm (GA), orthogonal greedy algorithm (OGA), adaptive Fourier decomposition (AFD), pre-orthogonal adaptive Fourier decomposition (POAFD), $n$-Best approximation and unwinding Blaschke expansion, are analyzed and compared. Of which an order in re-construction efficiency between the mentioned algorithms is given based on detailed study of their respective remainders. The study spells out the natural connections between the multiple kernels and the related Laguerre system, and in particular shows that both, like the Fourier series, extract out the $O(n^{-σ})$ order convergence rate from the functions in the Hardy-Sobolev space of order $σ>0.$ Existence of the $n$-Best approximation with the complete Szegö dictionary is proved and the related algorithm aspects are discussed. The included experiments form a significant integration part of the study, for they not only illustrate the theoretical results, but also provide cross comparison between various ways of combination between the matching pursuit algorithms and the dictionaries in use. Experiments show that the complete dictionary remarkably improves approximation efficiency.
MatGD: Materials Graph Digitizer
Venue: arXiv cs.CV 2023
Year: 2023

We have developed MatGD (Material Graph Digitizer), which is a tool for digitizing a data line from scientific graphs. The algorithm behind the tool consists of four steps: (1) identifying graphs within subfigures, (2) separating axes and data sections, (3) discerning the data lines by eliminating irrelevant graph objects and matching with the legend, and (4) data extraction and saving. From the 62,534 papers in the areas of batteries, catalysis, and MOFs, 501,045 figures were mined. Remarkably, our tool showcased performance with over 99% accuracy in legend marker and text detection. Moreover, its capability for data line separation stood at 66%, which is much higher compared to other existing figure mining tools. We believe that this tool will be integral to collecting both past and future data from publications, and these data can be used to train various machine learning models that can enhance material predictions and new materials discovery.
Improved vectorization of OpenCV algorithms for RISC-V CPUs
Venue: arXiv cs.DC 2023
Year: 2023

The development of an open and free RISC-V architecture is of great interest for a wide range of areas, including high-performance computing and numerical simulation in mathematics, physics, chemistry and other problem domains. In this paper, we discuss the possibilities of accelerating computations on available RISC-V processors by improving the vectorization of several computer vision and machine learning algorithms in the widely used OpenCV library. It is shown that improved vectorization speeds up computations on existing prototypes of RISC-V devices by tens of percent.
High-Power and Safe RF Wireless Charging: Cautious Deployment and Operation
Venue: arXiv cs.NI 2023
Year: 2023

The wired charging and the need for battery replacements are critical barriers to unlimited, scalable, and sustainable mobile connectivity, motivating the interest in radio frequency (RF) wireless power transfer (WPT) technology. However, the inherently low end-to-end power transfer efficiency (PTE) and health/safety-related apprehensions about the technology are critical obstacles. Indeed, RF-WPT implementation and operation require efficient and cautious strategies and protocols, especially when targeting high-power charging, which constitutes the scope of this work. Herein, we overview the main factors affecting the end-to-end PTE of RF-WPT systems and their multiplicative effect and interdependencies. Moreover, we discuss key electromagnetic field (EMF) exposure metrics, safety limits, and approaches for efficient and EMF-aware deployment and operation. Quantitatively, we show that near-field RF charging may significantly reduce EMF exposure, and thus must be promoted. We also present our vision of a cyber-physical system for efficient and safe wireless charging, specify key components and their interrelation, and illustrate numerically the PTE attained by two modern low-power multi-antenna architectures in a simple setup. Throughout the paper, we highlight the need for high end-to-end PTE architectures and charging protocols transparently complying with EMF exposure regulations and outline relevant challenges and research directions. This work expands the vision and understanding of modern RF-WPT technology and constitutes a step towards making the technology attractive for worldwide commercial exploitation.
Targeted Activation Penalties Help CNNs Ignore Spurious Signals
Venue: arXiv cs.CV 2023
Year: 2023

Neural networks (NNs) can learn to rely on spurious signals in the training data, leading to poor generalisation. Recent methods tackle this problem by training NNs with additional ground-truth annotations of such signals. These methods may, however, let spurious signals re-emerge in deep convolutional NNs (CNNs). We propose Targeted Activation Penalty (TAP), a new method tackling the same problem by penalising activations to control the re-emergence of spurious signals in deep CNNs, while also lowering training times and memory usage. In addition, ground-truth annotations can be expensive to obtain. We show that TAP still works well with annotations generated by pre-trained models as effective substitutes of ground-truth annotations. We demonstrate the power of TAP against two state-of-the-art baselines on the MNIST benchmark and on two clinical image datasets, using four different CNN architectures.
EWasteNet: A Two-Stream Data Efficient Image Transformer Approach for E-Waste Classification
Venue: arXiv cs.CV 2023
Year: 2023

Improper disposal of e-waste poses global environmental and health risks, raising serious concerns. The accurate classification of e-waste images is critical for efficient management and recycling. In this paper, we have presented a comprehensive dataset comprised of eight different classes of images of electronic devices named the E-Waste Vision Dataset. We have also presented EWasteNet, a novel two-stream approach for precise e-waste image classification based on a data-efficient image transformer (DeiT). The first stream of EWasteNet passes through a sobel operator that detects the edges while the second stream is directed through an Atrous Spatial Pyramid Pooling and attention block where multi-scale contextual information is captured. We train both of the streams simultaneously and their features are merged at the decision level. The DeiT is used as the backbone of both streams. Extensive analysis of the e-waste dataset indicates the usefulness of our method, providing 96% accuracy in e-waste classification. The proposed approach demonstrates significant usefulness in addressing the global concern of e-waste management. It facilitates efficient waste management and recycling by accurately classifying e-waste images, reducing health and safety hazards associated with improper disposal.
Tool Wear Segmentation in Blanking Processes with Fully Convolutional Networks based Digital Image Processing
Venue: arXiv cs.CV 2023
Year: 2023

The extend of tool wear significantly affects blanking processes and has a decisive impact on product quality and productivity. For this reason, numerous scientists have addressed their research to wear monitoring systems in order to identify or even predict critical wear at an early stage. Existing approaches are mainly based on indirect monitoring using time series, which are used to detect critical wear states via thresholds or machine learning models. Nevertheless, differentiation between types of wear phenomena affecting the tool during blanking as well as quantification of worn surfaces is still limited in practice. While time series data provides partial insights into wear occurrence and evolution, direct monitoring techniques utilizing image data offer a more comprehensive perspective and increased robustness when dealing with varying process parameters. However, acquiring and processing this data in real-time is challenging. In particular, high dynamics combined with increasing strokes rates as well as the high dimensionality of image data have so far prevented the development of direct image-based monitoring systems. For this reason, this paper demonstrates how high-resolution images of tools at 600 spm can be captured and subsequently processed using semantic segmentation deep learning algorithms, more precisely Fully Convolutional Networks (FCN). 125,000 images of the tool are taken from successive strokes, and microscope images are captured to investigate the worn surfaces. Based on findings from the microscope images, selected images are labeled pixel by pixel according to their wear condition and used to train a FCN (U-Net).
Towards new challenges of modern Pentest
Venue: arXiv cs.CR 2023
Year: 2023

With the increasing number of internet-based resources and applications, the amount of attacks faced by companies has increased significantly in the past years. Likewise, the techniques to test security and emulate attacks need to be constantly improved and, as a consequence, help to mitigate attacks. Among these techniques, penetration test (Pentest) provides methods to assess the security posture of assets, using different tools and methodologies applied in specific scenarios. Therefore, this study aims to present current methodologies, tools, and potential challenges applied to Pentest from an updated systematic literature review. As a result, this work provides a new perspective on the scenarios where penetration tests are performed. Also, it presents new challenges such as automation of techniques, management of costs associated with offensive security, and the difficulty in hiring qualified professionals to perform Pentest.