Reviewerly/exHarmony huggingface dataset:
Context-Guided Super-Class Inference for Zero-Shot Detection

Zero-shot object detection (ZSD) is a newly proposed re-search problem, which aims to simultaneously locate and recognize objects of previously unseen classes. Existing algorithms usually formulate it as a simple combination of a typical detection framework and zero-shot classifier, by learning a visual-semantic mapping from the visual features of bounding box proposals to semantic embeddings of class labels. In this paper, we propose a novel ZSD approach that leverages the context information surrounding objects in the image, following the principle that objects tend to be found in certain contexts. It also incorporates the semantic relations between seen and unseen classes to help recognize located instances. Comprehensive experiments on PASCAL VOC and MS COCO datasets show that context and class hierarchy truly improve the performance of detection.
Container Based On-Premises Cloud Security Framework

Securing a cloud orchestration framework is the need of the hour, as more attacks on cloud infrastructure are reported in recent times. Orchestration of cloud with the help of container is one of the design strategies which is being employed lately. Containers are lighter than Virtual Machines, significantly reduces start-up and processing time and incurs less storage overhead as well. Our aim in this research work is to explore container based on-premise cloud orchestration, analyse its security scenario, study current research efforts, and propose a secure framework for container based cloud orchestration to minimize the vulnerability if any. To achieve this goal, sample deployment set of container images for risk assessment is identified, vulnerability or risk analysis is performed on the set of images with the help of image scanning tool. Based on the risk analysis, parameters are defined. With these parameters, the best image suitable for completing client request is selected from the pool of images and that image is deployed in our secure container based cloud infrastructure.
Towards Making Unsupervised Graph Hashing Robust

Unsupervised hashing without supervision easily deteriorates in the case of grossly corrupted data. Motivated by robust optimization, this paper proposes a dual-graph regularized robust hashing (DGRH) based on both manifold smoothness and robust estimators in a more intuitive manner. Orthogonal to existing robust hashing methods, DGRH directly removes the outliers of datasets with M-estimator to exert robustness. In specific, it intends to recover low-rank representation from corrupted data via l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> loss while preserving neighborhood relationships among samples with dual-graph regularization. Although DGRH seems a simple extension of robust PCA on graphs with hashing trick, it is easy to implement yet effective. Theory analysis is provided to support our claim. Experiments of image retrieval on three popular benchmark datasets show the efficacy of DGRH as compared to several well-behaved representative counterparts.
Exploring the Performance Bound of Cambricon Accelerator in End-to-End Inference Scenario

Deep learning algorithms have become pervasive in a broad range of industrial application scenarios. DianNao/Cambricon family is a set of energy-efficient hardware accelerators for machine learning, especially for deep learning, covering from edge embedded devices to cloud data centers. However, in the real application scenario, the complicated software stack and the extra overhead (memory copy) hinder the full exploitation of the accelerator performance. In this paper, we try to explore the performance bound of Cambricon accelerator MLU100 in end-to-end deep learning inference scenarios (from data/model load to inference results store). We leverage the offline model to bypass the general deep learning framework, use the multiple threads programming to fully exploit the parallelism of the multi-core accelerator and apply specific data structure to decrease the memory copy overhead. The evaluation results show that, for RetNet-50 on CIFAR-10 dataset, our optimization methods are 32.09$$\times $$ faster than the baseline of the optimized batch size (64), and achieve $$85\%$$ of the performance upper-bound on the Cambricon MLU100 board.
An SSI Based System for Incentivized and SelfDetermined Customer-to-Business Data Sharing in a Local Economy Context

With ongoing developments in the field of smart cities and digitalization in general, data is becoming a driving factor and value stream for new and existing economies alike. However, there exists an increasing centralization and monopolization of data holders and service providers, especially in the form of the big US-based technology companies in the western world and central technology providers with close ties to the government in the Asian regions. Self Sovereign Identity (SSI) provides the technical building blocks to create decentralized data-driven systems, which bring data autonomy back to the users. In this paper, we propose a system in which the combination of SSI and a token economy based incentivisation strategies makes it possible to unlock the potential value of data-pools without compromising the data autonomy of the users.
Gold Seeker: Information Gain From Policy Distributions for Goal-Oriented Vision-and-Langauge Reasoning

As Computer Vision moves from passive analysis of pixels to active analysis of semantics, the breadth of information algorithms need to reason over has expanded significantly. One of the key challenges in this vein is the ability to identify the information required to make a decision, and select an action that will recover it. We propose a reinforcement-learning approach that maintains a distribution over its internal information, thus explicitly representing the ambiguity in what it knows, and needs to know, towards achieving its goal. Potential actions are then generated according to this distribution. For each potential action a distribution of the expected outcomes is calculated, and the value of the potential information gain assessed. The action taken is that which maximizes the potential information gain. We demonstrate this approach applied to two vision-and-language problems that have attracted significant recent interest, visual dialog and visual query generation. In both cases the method actively selects actions that will best reduce its internal uncertainty, and outperforms its competitors in achieving the goal of the challenge.
C3Net: Demoiréing Network Attentive in Channel, Color and Concatenation

Attentive neural networks for image restoration are in the spotlight because they got remarkable results both qualitatively and quantitatively. Networks attentive in RGB channels were effective in fields such as single image super-resolution and RAW to RGB mapping. In addition, networks attentive in positions of pixels were used in image denoising. However, networks attentive in positions of pixels, so called spatial attention or pixel attention algorithm, were not as effective in image restoration because the number of pixels in patches of an image is so many that the weights by sigmoid function are insignificant. Also, networks attentive in positions of pixels were mainly used in high-level vision such as image classification and image captioning where there is no need to restore an image itself. In this paper, we propose a demoiréing network attentive in channel, color, and concatenation, named C3Net. The proposed algorithm uses residual blocks attentive in RGB channels to take advantage of channel attention algorithm. In addition, we introduce a L1 color loss for demoiréing to solve moiré patterns caused by color-striped patterns. Also, we transferred multi-scale information by concatenation, not multiplying with the insignificant weights by sigmoid function. As a result, our proposed C3Net showed state-of-the-art results in the benchmark dataset on NTIRE 2020 demoiréing challenge.
Information Extraction from Document Images via FCA based Template Detection and Knowledge Graph Rule Induction

We view information extraction from document images as a complex problem that requires a combination of 1) state of the art deep learning vision models for detection of entities and primitive relations, 2) symbolic background knowledge that expresses prior information of spatial and semantic relationships, using the entities and primitive relations from the neural detectors, and 3) learning of symbolic extraction rules using one, or few examples of annotated document images. Several challenges arise in ensuring that this neuro-symbolic software stack works together seamlessly. These include vision-based challenges to ensure that the documents are "seen" at the appropriate level of detail to detect entities; symbolic representation challenges in identifying primitive relations between the entities identified by the vision system; learning-based challenges of identifying the appropriate level of symbolic abstraction for the retrieval rules, the need to identify background knowledge that is relevant to the documents being analyzed, and learning general symbolic rules in data-deficient domains. In this paper, we describe how we meet some of these challenges in the design of our document-reading platform. In particular, we focus on use cases with multiple templates which additionally involves finding structurally similar images in large heterogeneous document image collections. An adaptive lattice based template allocation module was utilized for evaluating document similarity based on both textual content and document structure. A knowledge graph is used for capturing document structure and a relational rule learning system is employed on the knowledge graph for generating extraction rules. Experiments on a publicly shared data-set <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> of 1400 trade finance documents demonstrates the viability of the proposed system.
Benchmarking Deep Neural Network Architectures for Machining Tool Anomaly Detection

With the democratization of cyber-physical systems, edge computing, and large-scale data infrastructure, the volume of operational data available is continuously increasing. One of the significant challenges in current industrial research is defining a robust and scalable approach for machine health monitoring and anomaly detection. The methods that exist for such purposes rely extensively on feature engineering and are strongly dependent on the expertise of the operator, hence limiting their generalization. Deep learning techniques, on the other hand, are known to automate feature engineering and allow complex abstractions to be learned, making them particularly suitable for machine health monitoring. This paper presents a benchmarking of deep neural network architectures for the identification of machining tool anomalies on a lathe machine. The features are generated using indirect metrics such as sensor data and process variables from the machine controller, but without direct metrics like surface roughness or finished part quality. The ability of different architectures to identify incipient anomalies in tool quality is compared. A detailed treatment of various subsets of features is provided along with their relative importance to identify the minimum required parameters for accurately identifying the tool anomaly for each architecture considered. Finally, a recommendation is provided based on the results obtained on the type of architecture that is appropriate for the identification of machining tool anomalies. It is expected that the embeddings of the training data learned by the chosen network can be used for other learning tasks, such as transfer learning to another machine or anomaly type. The methodology described in this paper lends itself well to continuous monitoring through the use of scalable robust models and appropriate units of analysis for each model.
Homomorphic Encryption-Based LSB Substitution for High Capacity Data Hiding in the Encrypted Domain

During the last few decades, multimedia security over the cloud has become a major issue. Public-key homomorphism is an efficient approach for data hiding in encrypted images (DHEI). An original image is encrypted using a public key and sent across a network and then processed to embed a secret message directly in the encrypted domain. During the decoding step, a private key is used to obtain a marked reconstructed image, where the secret message can be extracted. In this paper, we propose an efficient method of DHEI based on the Paillier cryptosystem. Using its homomorphic properties, pixel blocks and bits of the message are multiplied in the encrypted domain resulting in an addition in the clear domain. By applying a pre-processing step on the original image before encryption, this addition becomes a least significant bits (LSB) substitution. Experimental results show that using our proposed scheme, we obtain a high payload value (1 bpp) without expanding a lot the original image size contrary to current state-of-the-art methods. Indeed, whatever the key size, the expansion rate is always equal to 2. In addition, the original image and the marked reconstructed image are very similar.
User Conditional Hashtag Recommendation for Micro-Videos

When a user tend to publish a micro-video, hashtag recommendation aims to suggest hashtags that can reflect the theme or contents of the micro-video, and meet the user tagging preference as well. In this paper, we show how user profile and historical hashtags combined with micro-video representations can be used to perform hashtag recommendation. Specifically, a User-guided Hierarchical Multi-head Attention Network (UHMAN) is proposed to attend both image-level and video-level representations of micro-videos with user side information. We evaluate the proposed model on the dataset collected from micro-video sharing platform Musical.ly. The experimental results demonstrate the effectiveness of the proposed method.
Category-Level Adversarial Self-Ensembling for Domain Adaptation

Domain adaptation aims at learning from a source data distribution a well-performing model on a different target data distribution. Recently, the self-ensembling-based methods have been proved to be effective for unsupervised domain adaptation. However, they still have two shortcomings: 1) no explicitly constraint about the distributions between the source and target domains; 2) the Euclidean distance fails to measure the similarity between two distributions with no overlap. To solve those shortcomings, we propose a novel Category-level Adversarial Self-ensembling (CAS) model for domain adaptation, which contains two types of consistency constraints. The first one is how to constrain the descriptions for source and target domains to be aligned. Therefore, we adopt a minimax game with a discrepancy loss between the category information generated by two classifiers. As the self-ensembling consists of two sub-networks: student and teacher networks, the second one is the consistency between those two networks for the target samples. Aiming to overcome the disadvantage of the Euclidean metric, we employ the Wasserstein distance to measure the difference between two probabilistic distributions. Experiments on several benchmarks demonstrate that our proposed CAS is superior to existing methods.
Swarm intelligence algorithms’ solutions to the travelling salesman’s problem

Abstract This paper presents research findings on the application of swarm intelligence techniques in computational intelligence to solve the travelling salesman’s problem. The travelling salesman’s problem finds real-life application in post office mail delivery, school bus routing, delivery of food to home-bound people etc. After a number of experimental procedures, the study concludes that all the comparative algorithms are very efficient in providing solutions to the benchmark travelling salesman’s problems considered, though the Discrete Cuckoo Search and the African Buffalo Optimization have a slight edge in performance over the other comparative algorithms. In all, the study agrees with earlier studies in reaching the conclusion that swarm-based optimization techniques are not only effective but also are very efficient in providing solutions to the travelling salesman’s problems.
Benefit Analysis of Remote Oceanic Meteorology Information Operational (ROMIO) Demonstration

Convective weather is a significant factor causing airspace restrictions to commercial air traﬃc. Frequent weather updates are critical for flights over oceans and regions with limited coverage of ground-based radars since convective weather events (e.g., thunderstorms) can evolve relatively fast. The Federal Aviation Administration (FAA) Weather Technology in the Cockpit (WTIC) program has sponsored the Remote Oceanic Meteorology Information Operational (ROMIO) demonstration to evaluate the feasibility of uplinking convective weather products into the flight deck of oceanic and remote flights. For this purpose, two satellite-based weather products developed by the National Center for Atmospheric Research (NCAR) were used, namely the Cloud Top Height (CTH) and Convection Diagnosis Oceanic (CDO) products. In this paper, we studied the operational benefits of using the ROMIO application by analyzing over 18,000 historical flight trajectories crossing the Inter-Tropical Convergence Zone (ITCZ). We used flight azimuth analysis to identify two types of weather deviations (i.e., macroscopic and microscopic) for avoiding inclement weather. Using the ROMIO demonstration display enhances the situational awareness and helps pilots to identify adverse weather earlier com-pared to airborne radars. We quantified the potential benefits of the ROMIO-aided strategic decision making in terms of travel distance, travel time, fuel consumption, and greenhouse gas emission savings. Also, we extracted several features of weather deviation maneuvers including maximum lateral deviations from flight plans, deviation angles, deviation times, and Closest Point of Approaches (CPA) to CDO and CTH contours. We considered a pilot survey to select flights that employed the ROMIO application for convective weather avoidance and compared the distribution of deviation angles and lateral deviations in the Pre-ROMIO and Post-ROMIO periods.
Query Rewriting for Voice Shopping Null Queries

Voice shopping using natural language introduces new challenges related to customer queries, like handling mispronounced, misexpressed, and misunderstood queries. Voice null queries, which result in no offers, have negative impact on customers shopping experience. Query rewriting (QR) attempts to automatically replace null queries with alternatives that lead to relevant results. We present a new approach for pre-retrieval QR of voice shopping null queries. Our proposed QR framework first generates alternative queries using a search index-based approach that targets different potential failures in voice queries. Then, a machine-learning component ranks these alternatives, and the original query is amended by the selected alternative. We provide an experimental evaluation of our approach based on data logs of a commercial voice assistant and an e-commerce website, demonstrating that it outperforms several baselines by more than $22%$. Our evaluation also highlights an interesting phenomenon, showing that web shopping null queries are considerably different, and apparently easier to fix, than voice queries. This further substantiates the use of specialized mechanisms for the voice domain. We believe that our proposed framework, mapping tail queries to head queries, is of independent interest since it can be extended and applied to other domains.
Interactive Training And Architecture For Deep Object Selection

Interactive object cutout tools are the cornerstone of the image editing workflow. Algorithms that can reduce the number of interactions are clearly valuable. Recent deep-learning based interactive segmentation algorithms are capable of rough binary selections with a handful of clicks, yet, they tend to plateau once this rough selection has been reached. In this work, we interpret this plateau as an inability of the algorithm to precisely leverage each user interaction.We introduce a novel interactive architecture and a training scheme that are both tailored to better exploit the user input at higher numbers of clicks. Comprehensive experiments support our approach, and our network achieves state of the art performance.
Fooling Gaussian PTFs via local hyperconcentration

We give a pseudorandom generator that fools degree-d polynomial threshold functions over n-dimensional Gaussian space with seed length d O(logd) · logn. All previous generators had a seed length with at least a 2 d dependence on d.
BUILDING THE HISTORY OF THE FUTURE: A TOOL FOR CULTURE-CENTRED DESIGN FOR THE SPECULATIVE FUTURE

Abstract In this paper, the authors propose a culture-centred tool called Speculative Ethnography to support the practice of design for the speculative future. Most explorations of tools and techniques of design for the speculative future focus on materialising the speculation. Instead, Speculative Ethnography provides a guideline to help designers define their speculation through the scientific imagination of the plausible future. It ensures the effectiveness of design for the speculative future and supports the interaction of culture and design for the speculative future.
Font-ProtoNet: Prototypical Network based Font Identification of Document Images in Low Data Regime

While optical character recognition has attracted considerable interest from researchers in recent times, automating font identification in printed/scanned documents is still not a well explored problem. With the increasing variety of fonts in the open community, identifying the different fonts used in a given document image can often provide important visual cues for document understanding. Font identification is a challenging task owing to smaller inter-class variations and limited availability of labeled image data for a large variety of font images. In the absence of the original true type format (ttf) files, even synthetic data generation is not possible. To this end, we propose to utilize recent few-shot learning techniques like prototypical networks for font identification in scanned / printed document images using character images from different fonts as input for scarce data scenarios and call the proposed method Font-ProtoNet. This approach uses an initial set of classes to learn an embedding and centroid representations (as class prototypes), that are used to classify novel samples based on euclidean distance. We demonstrate that Font-ProtoNet gives encouraging results by training prototypical networks in few-shot learning settings on a synthetic dataset of 200 font classes and using the trained network to identify fonts on a synthetic dataset of 100 novel font classes. We have also tested our approach on the real-world Adobe Visual Font Recognition (Adobe VFR) dataset and obtained 59.86% and 71.01% word-level accuracy of font identification using 1-shot and 5-shot i.e., 1 and 5 character images per font class, respectively.
Ontology mediated information extraction in financial domain with Mastro System-T

Information extraction (IE) refers to the task of turning text documents into a structured form, in order to make the information contained therein automatically processable. Ontology Mediated Information Extraction (OMIE) is a new paradigm for IE that seeks to exploit the semantic knowledge expressed in ontologies to improve query answering over unstructured data (properly raw text). In this paper we present Mastro System-T, an OMIE tool born from a joint collaboration between the University of Rome "La Sapienza" and IBM Research Almaden and its first application in a financial domain, namely to facilitate the access to and the sharing of data extracted from the EDGAR system.