OpenRaiser/Intern-Atlas huggingface dataset:
Concept Drift Challenge in Multimedia Anomaly Detection: A Case Study with Facial Datasets
Venue: arXiv cs.CV 2022
Year: 2022

Anomaly detection in multimedia datasets is a widely studied area. Yet, the concept drift challenge in data has been ignored or poorly handled by the majority of the anomaly detection frameworks. The state-of-the-art approaches assume that the data distribution at training and deployment time will be the same. However, due to various real-life environmental factors, the data may encounter drift in its distribution or can drift from one class to another in the late future. Thus, a one-time trained model might not perform adequately. In this paper, we systematically investigate the effect of concept drift on various detection models and propose a modified Adaptive Gaussian Mixture Model (AGMM) based framework for anomaly detection in multimedia data. In contrast to the baseline AGMM, the proposed extension of AGMM remembers the past for a longer period in order to handle the drift better. Extensive experimental analysis shows that the proposed model better handles the drift in data as compared with the baseline AGMM. Further, to facilitate research and comparison with the proposed framework, we contribute three multimedia datasets constituting faces as samples. The face samples of individuals correspond to the age difference of more than ten years to incorporate a longer temporal context.
VICTOR: Visual Incompatibility Detection with Transformers and Fashion-specific contrastive pre-training
Venue: arXiv cs.CV 2022
Year: 2022

For fashion outfits to be considered aesthetically pleasing, the garments that constitute them need to be compatible in terms of visual aspects, such as style, category and color. Previous works have defined visual compatibility as a binary classification task with items in a garment being considered as fully compatible or fully incompatible. However, this is not applicable to Outfit Maker applications where users create their own outfits and need to know which specific items may be incompatible with the rest of the outfit. To address this, we propose the Visual InCompatibility TransfORmer (VICTOR) that is optimized for two tasks: 1) overall compatibility as regression and 2) the detection of mismatching items and utilize fashion-specific contrastive language-image pre-training for fine tuning computer vision neural networks on fashion imagery. We build upon the Polyvore outfit benchmark to generate partially mismatching outfits, creating a new dataset termed Polyvore-MISFITs, that is used to train VICTOR. A series of ablation and comparative analyses show that the proposed architecture can compete and even surpass the current state-of-the-art on Polyvore datasets while reducing the instance-wise floating operations by 88%, striking a balance between high performance and efficiency. We release our code at https://github.com/stevejpapad/Visual-InCompatibility-Transformer
Modelling Social Context for Fake News Detection: A Graph Neural Network Based Approach
Venue: arXiv cs.SI 2022
Year: 2022

Detection of fake news is crucial to ensure the authenticity of information and maintain the news ecosystems reliability. Recently, there has been an increase in fake news content due to the recent proliferation of social media and fake content generation techniques such as Deep Fake. The majority of the existing modalities of fake news detection focus on content based approaches. However, most of these techniques fail to deal with ultra realistic synthesized media produced by generative models. Our recent studies find that the propagation characteristics of authentic and fake news are distinguishable, irrespective of their modalities. In this regard, we have investigated the auxiliary information based on social context to detect fake news. This paper has analyzed the social context of fake news detection with a hybrid graph neural network based approach. This hybrid model is based on integrating a graph neural network on the propagation of news and bi directional encoder representations from the transformers model on news content to learn the text features. Thus this proposed approach learns the content as well as the context features and hence able to outperform the baseline models with an f1 score of 0.91 on PolitiFact and 0.93 on the Gossipcop dataset, respectively
A Hybrid Deep Animation Codec for Low-bitrate Video Conferencing
Venue: arXiv cs.MM 2022
Year: 2022

Deep generative models, and particularly facial animation schemes, can be used in video conferencing applications to efficiently compress a video through a sparse set of keypoints, without the need to transmit dense motion vectors. While these schemes bring significant coding gains over conventional video codecs at low bitrates, their performance saturates quickly when the available bandwidth increases. In this paper, we propose a layered, hybrid coding scheme to overcome this limitation. Specifically, we extend a codec based on facial animation by adding an auxiliary stream consisting of a very low bitrate version of the video, obtained through a conventional video codec (e.g., HEVC). The animated and auxiliary videos are combined through a novel fusion module. Our results show consistent average BD-Rate gains in excess of -30% on a large dataset of video conferencing sequences, extending the operational range of bitrates of a facial animation codec alone
Satisfiability Bounds for $ω$-Regular Properties in Bounded-Parameter Markov Decision Processes
Venue: arXiv cs.LO 2022
Year: 2022

We consider the problem of computing minimum and maximum probabilities of satisfying an $ω$-regular property in a bounded-parameter Markov decision process (BMDP). BMDP arise from Markov decision processes (MDP) by allowing for uncertainty on the transition probabilities in the form of intervals where the actual probabilities are unknown. $ω$-regular languages form a large class of properties, expressible as, e.g., Rabin or parity automata, encompassing rich specifications such as linear temporal logic. In a BMDP the probability to satisfy the property depends on the unknown transitions probabilities as well as on the policy. In this paper, we compute the extreme values. This solves the problem specifically suggested by Dutreix and Coogan in CDC 2018, extending their results on interval Markov chains with no adversary. The main idea is to reinterpret their work as analysis of interval MDP and accordingly the BMDP problem as analysis of an $ω$-regular stochastic game, where a solution is provided. This method extends smoothly further to bounded-parameter stochastic games.
On Using Linux Kernel Huge Pages with FLASH, an Astrophysical Simulation Code
Venue: arXiv cs.DC 2022
Year: 2022

We present efforts at improving the performance of FLASH, a multi-scale, multi-physics simulation code principally for astrophysical applications, by using huge pages on Ookami, an HPE Apollo 80 A64FX platform. FLASH is written principally in modern Fortran and makes use of the PARAMESH library to manage a block-structured adaptive mesh. We explored options for enabling the use of huge pages with several compilers, but we were only able to successfully use huge pages when compiling with the Fujitsu compiler. The use of huge pages substantially reduced the number of translation lookaside buffer misses, but overall performance gains were marginal.
Modelling Equity Transaction Networks as Bursty Processes
Venue: arXiv cs.CE 2022
Year: 2022

Trade executions for major stocks come in bursts of activity, which can be partly attributed to the presence of self- and mutual excitations endogenous to the system. In this paper, we study transaction reports for five FTSE 100 stocks. We model the dynamic of transactions between counterparties using both univariate and multivariate Hawkes processes, which we fit to the data using a parametric approach. We find that the frequency of transactions between counterparties increases the likelihood of them to transact in the future, and that univariate and multivariate Hawkes processes show promise as generative models able to reproduce the bursty, hub dominated systems that we observe in the real world. We further show that Hawkes processes perform well when used to model buys and sells through a central clearing counterparty when considered as a bivariate process, but not when these are modelled as individual univariate processes, indicating that mutual excitation between buys and sells is present in these markets.
Nutzungsverhalten und Funktionsanforderungen digitaler Trainingsanwendungen während der Pandemie
Venue: arXiv cs.HC 2022
Year: 2022

Due to contact restrictions, closure of fitness centers and quarantine measures, the SARS-CoV-2 pandemic led to a considerable decline of sporting activities. The first relaxation of these restrictions allowed German citizens to mostly return to their normal training and exercise behavior, yet the long-term impact of the recurring measures (i.e. the "Lockdown", "Lockdown light" as well as the "Corona Emergency Break" in the case of Germany) remain rather under-investigated. Using a survey of (n=108) German sportspersons, we measured a significant decline of sporting activities even within the intermediary phases without major pandemic constraints. To evaluate the capabilities of digital training applications in countering these effects, we additionally recorded the usage of, among others, apps, trackers, videos and conferencing systems and identified the most important as well as missing and/or essential features with regards to their capabilities of facilitating individual sport and training in times without access to facilities or social contacts. Effectively, the usage of smart watches, online videos and conferences increased significantly when compared to before the pandemic; and especially online videos and conferences contributed to higher training frequencies. Data-driven or individual feedback, motivation and collaboration revealed to be the most important or even necessary functions for users of digital training applications to counter the decline of social components of training.
Physical Pooling Functions in Graph Neural Networks for Molecular Property Prediction
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks (GNNs) are emerging in chemical engineering for the end-to-end learning of physicochemical properties based on molecular graphs. A key element of GNNs is the pooling function which combines atom feature vectors into molecular fingerprints. Most previous works use a standard pooling function to predict a variety of properties. However, unsuitable pooling functions can lead to unphysical GNNs that poorly generalize. We compare and select meaningful GNN pooling methods based on physical knowledge about the learned properties. The impact of physical pooling functions is demonstrated with molecular properties calculated from quantum mechanical computations. We also compare our results to the recent set2set pooling approach. We recommend using sum pooling for the prediction of properties that depend on molecular size and compare pooling functions for properties that are molecular size-independent. Overall, we show that the use of physical pooling functions significantly enhances generalization.
Multiscale nonlocal beam theory: An application of distributed-order fractional operators
Venue: arXiv cs.CE 2022
Year: 2022

This study presents a comprehensive theoretical framework to simulate the response of multiscale nonlocal elastic beams. By employing distributed-order (DO) fractional operators with a fourth-order tensor as the strength-function, the framework can accurately capture anisotropic behavior of 2D heterogeneous beams with nonlocal effects localized across multiple scales. Building upon this general continuum theory and on the multiscale character of DO operators, a one-dimensional (1D) multiscale nonlocal Timoshenko model is also presented. This approach enables a significant model-order reduction without compromising the heterogeneous nonlocal description of the material, hence leading to an efficient and accurate multiscale nonlocal modeling approach. Both 1D and 2D approaches are applied to simulate the mechanical responses of nonlocal beams. The direct comparison of numerical simulations produced by either the DO or an integer-order fully-resolved model (used as ground truth) clearly illustrates the ability of the DO formulation to capture the effect of the microstructure on the macroscopic response. The assessment of the computational cost also indicates the superior efficiency of the proposed approach.
Voronoi Diagrams of Arbitrary Order on the Sphere
Venue: arXiv cs.CG 2022
Year: 2022

For a given set of points $U$ on a sphere $S$, the order $k$ spherical Voronoi diagram $SV_k(U)$ decomposes the surface of $S$ into regions whose points have the same $k$ nearest points of $U$. Hyeon-Suk Na, Chung-Nim Lee, and Otfried Cheong (Comput. Geom., 2002) applied inversions to construct $SV_1(U)$. We generalize their construction for spherical Voronoi diagrams from order $1$ to any order $k$. We use that construction to prove formulas for the numbers of vertices, edges, and faces in $SV_k(U)$. These formulas were not known before. We obtain several more properties for $SV_k(U)$, and we also show that $SV_k(U)$ has a small orientable cycle double cover.
Precision-based attacks and interval refining: how to break, then fix, differential privacy on finite computers
Venue: arXiv cs.CR 2022
Year: 2022

Despite being raised as a problem over ten years ago, the imprecision of floating point arithmetic continues to cause privacy failures in the implementations of differentially private noise mechanisms. In this paper, we highlight a new class of vulnerabilities, which we call \emph{precision-based attacks}, and which affect several open source libraries. To address this vulnerability and implement differentially private mechanisms on floating-point space in a safe way, we propose a novel technique, called \emph{interval refining}. This technique has minimal error, provable privacy, and broad applicability. We use interval refining to design and implement a variant of the Laplace mechanism that is equivalent to sampling from the Laplace distribution and rounding to a float. We report on the performance of this approach, and discuss how interval refining can be used to implement other mechanisms safely, including the Gaussian mechanism and the exponential mechanism.
Decentralized Computation Offloading With Cooperative UAVs: Multi-Agent Deep Reinforcement Learning Perspective
Venue: arXiv cs.IT 2022
Year: 2022

Limited computing resources of internet-of-things (IoT) nodes incur prohibitive latency in processing input data. This triggers new research opportunities toward task offloading systems where edge servers handle intensive computations of IoT devices. Deploying the computing servers at existing base stations may not be sufficient to support IoT nodes operating in a harsh environment. This requests mobile edge servers to be mounted on unmanned aerial vehicles (UAVs) that provide on-demand mobile edge computing (MEC) services. Time-varying offloading demands and mobility of UAVs need a joint design of the optimization variables for all time instances. Therefore, an online decision mechanism is essential for UAV-aided MEC networks. This article presents an overview of recent deep reinforcement learning (DRL) approaches where decisions about UAVs and IoT nodes are taken in an online manner. Specifically, joint optimization over task offloading, resource allocation, and UAV mobility is addressed from the DRL perspective. For the decentralized implementation, a multi-agent DRL method is proposed where multiple intelligent UAVs cooperatively determine their computations and communication policies without central coordination. Numerical results demonstrate that the proposed decentralized learning strategy is superior to existing DRL solutions. The proposed framework sheds light on the viability of the decentralized DRL techniques in designing self-organizing IoT networks.
Certified Everlasting Functional Encryption
Venue: arXiv cs.CR 2022
Year: 2022

Computational security in cryptography has a risk that computational assumptions underlying the security are broken in the future. One solution is to construct information-theoretically-secure protocols, but many cryptographic primitives are known to be impossible (or unlikely) to have information-theoretical security even in the quantum world. A nice compromise (intrinsic to quantum) is certified everlasting security, which roughly means the following. A receiver with possession of quantum encrypted data can issue a certificate that shows that the receiver has deleted the encrypted data. If the certificate is valid, the security is guaranteed even if the receiver becomes computationally unbounded. Although several cryptographic primitives, such as commitments and zero-knowledge, have been made certified everlasting secure, there are many other important primitives that are not known to be certified everlasting secure. In this paper, we introduce certified everlasting FE. In this primitive, the receiver with the ciphertext of a message m and the functional decryption key of a function f can obtain f(m) and nothing else. The security holds even if the adversary becomes computationally unbounded after issuing a valid certificate. We, first, construct certified everlasting FE for P/poly circuits where only a single key query is allowed for the adversary. We, then, extend it to q-bounded one for NC1 circuits where q-bounded means that q key queries are allowed for the adversary with an a priori bounded polynomial q. For the construction of certified everlasting FE, we introduce and construct certified everlasting versions of secret-key encryption, public-key encryption, receiver non-committing encryption, and a garbling scheme, which are of independent interest.
Why Accuracy Is Not Enough: The Need for Consistency in Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Object detectors are vital to many modern computer vision applications. However, even state-of-the-art object detectors are not perfect. On two images that look similar to human eyes, the same detector can make different predictions because of small image distortions like camera sensor noise and lighting changes. This problem is called inconsistency. Existing accuracy metrics do not properly account for inconsistency, and similar work in this area only targets improvements on artificial image distortions. Therefore, we propose a method to use non-artificial video frames to measure object detection consistency over time, across frames. Using this method, we show that the consistency of modern object detectors ranges from 83.2% to 97.1% on different video datasets from the Multiple Object Tracking Challenge. We conclude by showing that applying image distortion corrections like .WEBP Image Compression and Unsharp Masking can improve consistency by as much as 5.1%, with no loss in accuracy.
Frobenius structures in star-autonomous categories
Venue: arXiv cs.LO 2022
Year: 2022

It is known that the quantale of sup-preserving maps from a complete lattice to itself is a Frobenius quantale if and only if the lattice is completely distributive. Since completely distributive lattices are the nuclear objects in the autonomous category of complete lattices and sup-preserving maps, we study the above statement in a categorical setting. We introduce the notion of Frobenius structure in an arbitrary autonomous category, generalizing that of Frobenius quantale. We prove that the monoid of endomorphisms of a nuclear object has a Frobenius structure. If the environment category is star-autonomous and has epi-mono factorizations, a variant of this theorem allows to develop an abstract phase semantics and to generalise the previous statement. Conversely, we argue that, in a star-autonomous category where the monoidal unit is a dualizing object, if the monoid of endomorphisms of an object has a Frobenius structure and the monoidal unit embeds into this object as a retract, then the object is nuclear.
Persona-Knowledge Dialogue Multi-Context Retrieval and Enhanced Decoding Methods
Venue: arXiv cs.CL 2022
Year: 2022

Persona and Knowledge dual context open-domain chat is a novel dialogue generation task introduced recently. While Persona and Knowledge is each interesting context of open-domain dialogue, the combination of both has not been well studied. We tackle Persona-Knowledge identification and response generation tasks in this paper. We design an informed data augmentation strategy that is compatible with neural Q&A retrieval models. With the augmented data, we perform permutative Persona-Knowledge evaluation and successive Persona search fine-tuning. Furthermore, we perform dialogue generation with various decoding techniques and illustrate crucial elements. We achieve SOTA across official metrics with 93.99% Grounding accuracy average and 23.62 SacreBLEU score.
Very large-scale neighborhood search for drone routing with energy replenishment
Venue: arXiv cs.DM 2022
Year: 2022

The Drone Routing Problem with Energy replenishment (DRP-E) belongs to a general class of routing problems with intermediate stops and synchronization constraints. In DRP-E, the drone has to visit a set of nodes and routinely requires battery swaps from a (potentially) mobile replenishment station. Contrary to widespread restrictions in the drone routing literature, several destinations may be visited in between two consecutive battery swaps. In this paper, we propose a nontrivial very large-scale neighbourhood for DRP-E, which synergetically leverages two large-sized polynomially solvable DRP-E SubProblems (SP1 and SP2). The number of feasible solutions in the resulting neighborhood is a multiple of those in SP1 and SP2, and, thus, exponential in the input size of the problem, whereas the computational time to search it remains polynomial. The proposed polynomial two-stage dynamic programming algorithm VLSN to search this neighborhood can be flexibly adjusted to the desired trade-off between accuracy and computational time. For instance, the search procedure can be converted into an exact algorithm of competitive runtime for DRP-E. In computational tests, the developed solution methods outperform current state-of-the art heuristics for DRP-E by a significant margin. A case study based on a search for missing persons demonstrates that VLSN easily accommodates additional practice relevant features and outperforms the state-of-the-art solution in disaster relief by 20%.