pwc-archive/papers-with-abstracts huggingface dataset:
Improving Diversity and Reducing Redundancy in Paragraph Captions
Authors: Kanani, Chandresh S., Sriparna Saha, and Pushpak Bhattacharyya
Date: 2020-07-19 00:00:00
Proceeding: International Joint Conference on Neural Networks (IJCNN) 2020 7
Tasks: Decoder, Dense Captioning, Diversity, Image Captioning, Image Paragraph Captioning

The purpose of an image paragraph captioning
model is to produce detailed descriptions of the source images.
Generally, paragraph captioning models use encoder-decoder
based architectures similar to the standard image captioning
models. The encoder is a CNN based model, and the decoder is a
LSTM or GRU. The standard image captioning models produce
unsatisfactory results for the paragraph captioning task due to
the lack of diversity in the generated outputs [9]. The paragraphs
generated from standard image captioning models lack in language diversity and contain redundant information. In this work,
we have proposed an approach with language discriminator for
increasing the diversity in language, and dissimilarity score using
word mover’s distance [4] for reducing redundant information.
Using this approach with a state-of-the-art model at testing time,
we have improved the METEOR score from 13.63 to 19.01 for
the Visual Genome dataset
Hierarchical Scene Graph Encoder-Decoder for Image Paragraph Captioning
Authors: Yang, Xu, Chongyang Gao, Hanwang Zhang, and Jianfei Cai
Date: 2020-10-12 00:00:00
Proceeding: ACM International Conference on Multimedia 2020 10
Tasks: Decoder, Image Paragraph Captioning, Sentence

When we humans tell a long paragraph about an image, we usually
first implicitly compose a mental “script” and then comply with it
to generate the paragraph. Inspired by this, we render the modern
encoder-decoder based image paragraph captioning model such
ability by proposing Hierarchical Scene Graph Encoder-Decoder
(HSGED) for generating coherent and distinctive paragraphs. In
particular, we use the image scene graph as the “script” to incorporate rich semantic knowledge and, more importantly, the hierarchical constraints into the model. Specifically, we design a sentence
scene graph RNN (SSG-RNN) to generate sub-graph level topics,
which constrain the word scene graph RNN (WSG-RNN) to generate the corresponding sentences. We propose irredundant attention
in SSG-RNN to improve the possibility of abstracting topics from
rarely described sub-graphs and inheriting attention in WSG-RNN
to generate more grounded sentences with the abstracted topics,
both of which give rise to more distinctive paragraphs. An efficient
sentence-level loss is also proposed for encouraging the sequence of
generated sentences to be similar to that of the ground-truth paragraphs. We validate HSGED on Stanford image paragraph dataset
and show that it not only achieves a new state-of-the-art 36.02
CIDEr-D, but also generates more coherent and distinctive paragraphs under various metrics.
Look Deeper See Richer: Depth-aware Image Paragraph Captioning
Authors: Ziwei Wang, Yadan Luo, Yang Li, Zi Huang, Hongzhi Yin
Date: 2018-10-15 00:00:00
Proceeding: ACM International Conference on Multimedia 2018 10
Tasks: Decoder, Image Captioning, Image Paragraph Captioning, Sentence

With the widespread availability of image captioning at a sentence level, how to automatically generate image paragraphs is yet well explored. Describing an image by a full paragraph involves organising sentences orderly, coherently and diversely, inevitably leading higher complexity than by a single sentence. Existing image paragraph captioning methods give a series of sentences to represent the objects and regions of interests, where the descriptions are essentially generated by feeding the image fragments containing objects and regions into conventional image single-sentence captioning models. This strategy is difficult to generate the descriptions that guarantee the stereoscopic hierarchy and non-overlapping objects. In this paper, we propose a Depth-aware Attention Model (\textitDAM ) to generate paragraph captions for images. The depths of image areas are firstly estimated in order to discriminate objects in a range of spatial locations, which can further guide the linguistic decoder to reveal spatial relationships among objects. This model completes the paragraph in a logical and coherent manner. By incorporating the attention mechanism, the learned model swiftly shifts the sentence focus during paragraph generation, whilst avoiding verbose descriptions on a same object. Extensive quantitative experiments and the user study have been conducted on the Visual Genome dataset, which demonstrate the effectiveness and the interpretability of the proposed model.
Dual-CNN: A Convolutional language decoder for paragraph image captioning
Authors: Ruifan Li, Haoyun Liang, Yihui Shi, Fangxiang Feng, Xiaojie Wang
Date: 2020-02-14 00:00:00
Proceeding: Neurocomputing 2020 2
Tasks: Decoder, Diversity, Image Captioning, Image Paragraph Captioning

Abstract The task of paragraph image captioning aims to generate a coherent paragraph describing a given image. However, due to their limited ability to capture long-term dependency, recurrent neural network or long-short term memory based decoders could hardly generate satisfactory textual descriptions with a long paragraph. In addition, the training inefficiency in the sequential decoders is significantly observed. Motivated by the advantage of convolutional neural network (i.e., CNN), in this paper, we propose a Dual-CNN decoder with long-term memory ability and parallel computation, which can produce a semantically coherent paragraph for an image. Our Dual-CNN model is evaluated on the Stanford image-paragraph dataset. Extensive experiments demonstrate that our Dual-CNN achieves comparable results compared with state-of-the-art models. Furthermore, the diversity and coherence of generated paragraphs are analyzed to show the superiority of our approach.
SqueezeLight: Towards Scalable Optical Neural Networks with Multi-Operand Ring Resonators
Authors: Jiaqi Gu, Chenghao Feng, Zheng Zhao, Zhoufeng Ying, Mingjie Liu, Ray T. Chen, David Z. Pan
Date: 2021-02-05 00:00:00
Proceeding: IEEE Design, Automation & Test in Europe Conference & Exhibition (DATE) 2021 2

Optical neural networks (ONNs) have demonstrated promising potentials for next-generation artificial intelligence acceleration with ultra-low latency, high bandwidth, and low energy consumption. However, due to high area cost and lack of efficient sparsity exploitation, previous ONN designs fail to provide scalable and efficient neuromorphic computing, which hinders the practical implementation of photonic neural accelerators. In this work, we propose a novel design methodology to enable a more scalable ONN architecture. We propose a nonlinear optical neuron based on multi-operand ring resonators to achieve neuromorphic computing with a compact footprint, low wavelength usage, learnable neuron balancing, and built-in nonlinearity. The structured sparsity is exploited to support more efficient ONN engines via a fine-grained structured pruning technique. A robustness-aware learning method is adopted to guarantee the variation tolerance of our ONN. Simulation and experimental results show that the proposed ONN achieves one-order-of-magnitude improvement in compactness and efficiency over previous designs with high fidelity and robustness.
MetaVD: A Meta Video Dataset for enhancing human action recognition datasets
Authors: Yuya Yoshikawa, Yutaro Shigeto, Akikazu Takeuchi
Date: 2021-11-01 00:00:00
Proceeding: Computer Vision and Image Understanding 2021 11
Tasks: Action Classification, Action Recognition, Relation, Temporal Action Localization, Video Classification

Numerous practical datasets have been developed to recognize human actions from videos. However, many of them were constructed by collecting videos within a limited domain; thus, a model trained using one of the existing datasets often fails to classify videos in a different domain accurately. A possible solution for this drawback is to enhance the domain of each action label, i.e., to import videos associated with a given action label from the other datasets, and then, to train a model using the enhanced dataset. To realize this solution, we constructed a meta video dataset from the existing datasets for human action recognition, referred to as MetaVD. MetaVD comprises six popular human action recognition datasets, which we integrated by annotating 568,015 relation labels in total. These relation labels reflect equality, similarity, and hierarchy between action labels of the original datasets. We further present simple yet effective dataset enhancement methods using MetaVD, which are useful for training models with higher generalization performance, as established by experiments on human action classification. As a further contribution of MetaVD, we show that its analysis can provide useful insight into the datasets.
Exploiting Temporal Side Information in Massive IoT Connectivity
Authors: QiPeng Wang, Liang Liu, Shuowen Zhang, Francis C. M. Lau
Date: 2022-01-05 00:00:00
Tasks: Action Detection, Activity Detection

This paper considers the joint device activity detection and channel estimation problem in a massive Internet of Things (IoT) connectivity system, where a large number of IoT devices exist but merely a random subset of them become active for short-packet transmission in each coherence block. In particular, we propose to leverage the temporal correlation in device activity, e.g., a device active in the previous coherence block is more likely to be still active in the current coherence block, to improve the detection and estimation performance. However, it is challenging to utilize this temporal correlation as side information (SI), which relies on the knowledge about the exact statistical relation between the estimated activity pattern for the previous coherence block (which may be imperfect with unknown error) and the true activity pattern in the current coherence block. To tackle this challenge, we establish a novel SI-aided multiple measurement vector approximate message passing (MMV-AMP) framework. Specifically, thanks to the state evolution of the MMV-AMP algorithm, the correlation between the activity pattern estimated by the MMV-AMP algorithm in the previous coherence block and the real activity pattern in the current coherence block is quantified explicitly. Based on the well-defined temporal correlation, we further manage to embed this useful SI into the denoiser design under the MMV-AMP framework. Specifically, the SI-based soft-thresholding denoisers with binary thresholds and the SI-based minimum mean-squared error (MMSE) denoisers are characterized for the cases without and with the knowledge of the channel distribution, respectively. Numerical results are given to show the significant gain in device activity detection and channel estimation performance brought by our proposed SI-aided MMV-AMP framework.
Semantic Communications: Principles and Challenges
Authors: Zhijin Qin, Xiaoming Tao, Jianhua Lu, Wen Tong, Geoffrey Ye Li
Date: 2021-12-30 00:00:00
Tasks: Semantic Communication

Semantic communication, regarded as the breakthrough beyond the Shannon paradigm, aims at the successful transmission of semantic information conveyed by the source rather than the accurate reception of each single symbol or bit regardless of its meaning. This article provides an overview on semantic communications. After a brief review of Shannon information theory, we discuss semantic communications with theory, framework, and system design enabled by deep learning. Different from the symbol/bit error rate used for measuring conventional communication systems, performance metrics for semantic communications are also discussed. The article concludes with several open questions in semantic communications.
Deep learning for location based beamforming with NLOS channels
Authors: Luc Le Magoarou, Taha Yassine, Stéphane Paquelet, Matthieu Crussière
Date: 2021-12-29 00:00:00
Tasks: Deep Learning

Massive MIMO systems are highly efficient but critically rely on accurate channel state information (CSI) at the base station in order to determine appropriate precoders. CSI acquisition requires sending pilot symbols which induce an important overhead. In this paper, a method whose objective is to determine an appropriate precoder from the knowledge of the user's location only is proposed. Such a way to determine precoders is known as location based beamforming. It allows to reduce or even eliminate the need for pilot symbols, depending on how the location is obtained. the proposed method learns a direct mapping from location to precoder in a supervised way. It involves a neural network with a specific structure based on random Fourier features allowing to learn functions containing high spatial frequencies. It is assessed empirically and yields promising results on realistic synthetic channels. As opposed to previously proposed methods, it allows to handle both line-of-sight (LOS) and non-line-of-sight (NLOS) channels.
Interference Aware Cooperative Routing for Edge Computing-enabled 5G Networks
Authors: Abdullah Waqas, Hasan Mahmood, Nasir Saeed
Date: 2022-01-05 00:00:00
Tasks: Edge-computing

Recently, there has been growing research on developing interference-aware routing (IAR) protocols for supporting multiple concurrent transmission in next-generation wireless communication systems. The existing IAR protocols do not consider node cooperation while establishing the routes because motivating the nodes to cooperate and modeling that cooperation is not a trivial task. In addition, the information about the cooperative behavior of a node is not directly visible to neighboring nodes. Therefore, in this paper, we develop a new routing method in which the nodes' cooperation information is utilized to improve the performance of edge computing-enabled 5G networks. The proposed metric is a function of created and received interference in the network. The received interference term ensures that the Signal to Interference plus Noise Ratio (SINR) at the route remains above the threshold value, while the created interference term ensures that those nodes are selected to forward the packet that creates low interference for other nodes. The results show that the proposed solution improves ad hoc networks' performance compared to conventional routing protocols in terms of high network throughput and low outage probability.
Towards Maximizing a Perceptual Sweet Spot
Authors: Pedro Izquierdo Lehmann, Rodrigo F. Cadiz, Carlos A. Sing Long
Date: 2022-01-05 00:00:00

The sweet spot can be interpreted as the region where acoustic sources create a spatial auditory illusion. We study the problem of maximizing this sweet spot when reproducing a desired sound wave using an array of loudspeakers. To achieve this, we introduce a theoretical framework for spatial sound perception that can be used to define a sweet spot, and we develop a method that aims to generate a sound wave that directly maximizes the sweet spot defined by a model within this framework. Our method aims to incorporate perceptual principles from the onset and is flexible: while it imposes little to no constraints on the regions of interest, the arrangement of loudspeakers or their radiation pattern, it allows for audio perception models that include state-of-the-art monaural perceptual models. Proof-of-concept experiments show that our method outperforms state-of-the-art methods when comparing them in terms of their localization and coloration properties.
Sub-Structuring Modeling of Large Space Truss Structures for Structure/Control Optimization in Presence of Parametric Uncertainties
Authors: Antonio Finozzi, Francesco Sanfedino, Daniel Alazard
Date: 2022-01-05 00:00:00

Modern and future high precision pointing space missions face increasingly high challenges related to the widespread use of large flexible structures. The development of new modeling tools which are able to account for the multidisciplinary nature of this problem becomes extremely relevant in order to meet both structure and control performance criteria. This paper proposes a novel methodology to analytically model large truss structures in a sub-structuring framework. A three dimensional unit cube element has been designed and validated with a Finite Element commercial software. This model is composed by multiple two-dimensional sub-mechanisms assembled using block-diagram models. This constitutes the building block for constructing complex truss structures by repetitions of the element. The accurate vibration description of the system and its minimal representation, as well as the possibility of accounting for parametric uncertainties in its mechanical parameters, make it an appropriate tool to perform robust Structure/Control co-design. In order to demonstrate the strengths of the proposed approach, a co-design study case is proposed by combining a multidisciplinary optimization approach based on particle swarm algorithm and multiple structured robust Hinf-synthesis. This has been used to optimize the pointing performances of an high pointing antenna, minimizing the perturbations coming from the Solar Array Mechanisms (SADM) of two solar panels, performing active control by means of multiple proof mass actuators, and simultaneously reduce the mass of the truss-structure which connects the antenna to the main spacecraft body.
Energy Optimal Point-to-Point Motion Profile Optimization
Authors: Nick Van Oosterwyck, Foeke Vanbecelaere, Ferre Knaepkens, Michael Monte, Kurt Stockman, Annie Cuyt, Stijn Derammelaere
Date: 2022-01-05 00:00:00
Tasks: Position

Position-controlled systems driving repetitive tasks are of significant importance in industrial machinery. The electric actuators used in these systems are responsible for a large part of the global energy consumption, indicating that major savings can be made in this field. In this context, motion profile optimization is a very cost-effective solution as it allows for more energy-efficient machines without additional hardware investments or adaptions. In particular, mono-actuated mechanisms with position-dependent system properties have received considerable attention in literature. However, the current state-of-the-art methods often use unbounded design parameters to describe the motion profile. This both increases the computational complexity and hampers the search for a global optimum. In this paper, Chebyshev polynomials are used to describe the motion profile. Moreover, the exact bounds on the Chebyshev design parameters are derived. This both seriously reduces the computational complexity and limits the design space, allowing the application of a global optimizer such as the genetic algorithm. Experiments validate the added value of the chosen approach. In this study, it is found that the energy consumption can be reduced by 62.9% compared to a standard reference motion profile.
CAD Based Design Optimization of Four-bar Mechanisms: a coronaventilator case study
Authors: Abdelmajid Ben Yahya, Nick Van Oosterwyck, Ferre Knaepkens, Simon Houwen, Stijn Herregodts, Jan Herregodts, Bart Vanwalleghem, Annie Cuyt, Stijn Derammelaere
Date: 2022-01-05 00:00:00
Tasks: Design Synthesis

Design optimization of mechanisms is a promising research area as it results in more energy-efficient machines without compromising performance. However, machine builders do not actually use the design methods described in the literature as these algorithms require too much theoretical analysis. Moreover, the design synthesis approaches in the literature predominantly utilize heuristic optimizers leading to suboptimal local minima. This research introduces a convenient optimization workflow allowing wide industrial adoption, while guaranteeing to reveal the global optimum. To guarantee that we find the global optimum, a mathematical expression of the constraints describing the feasible region of possible designs is of great importance. Therefore, kinematic analysis of the point-to-point (PTP) planar four-bar mechanism is discussed to obtain the static and dynamic constraints. Within the feasible region, objective value samples are generated through CAD multi-body software. These motion simulations determine the required torque to fulfill the movement for a certain combination of design parameters. Sparse interpolation techniques allow minimizing the required amount of samples and thus CAD simulations. Moreover, this interpolation of simulation results enables the representation of the objective in a mathematical description without in-depth analytical design analysis by the machine designer. Subsequently, the mathematical expression of the objective allows global optimizers to find a global optimal design within the feasible design space. In a case study of a coronaventilator mechanism with three design parameters (DP's), 1870 CAD motion simulations from which only 618 are used to build a model allowed to reduce the RMS torque of the mechanism by 67%.
Model Order Reduction of The Time-Dependent Advection-Diffusion-Reaction Equation with Time-Varying Coefficients: Application to Real-Time Water Quality Monitoring
Authors: Ahmed Elkhashap, Dirk Abel
Date: 2022-01-05 00:00:00

Advection-Diffusion-Reaction (ADR) Partial Differential Equations (PDEs) appear in a wide spectrum of applications such as chemical reactors, concentration flows, and biological systems. A large number of these applications require the solution of ADR equations involving time-varying coefficients, where analytical solutions are usually intractable. Numerical solutions on the other hand require fine discretization and are computationally very demanding. Consequently, the models are normally not suitable for real-time monitoring and control purposes. In this contribution, a reduced order modeling method for a general ADR system with time-varying coefficients is proposed. Optimality of the reduced order model regarding the reduction induced error is achieved by using an H2-norm reduction method. The efficacy of the method is demonstrated using two test cases. Namely, a case for an ADR with arbitrary dynamics varying coefficients and a second case including the modeling of an exemplary water quality distribution path with randomly generated demand. The reduced order models are evaluated against high fidelity simulations using MATLAB's finite element method PDE toolbox. It is shown that the reduction can achieve a significant computational speedup allowing for the usage of the model for real-time applications with sampling times in milliseconds range. Moreover, the constructed ROM is shown to achieve high prediction accuracy with the normalized mean square error below 2.3 % for a real-world water quality simulation test case.
Transmission-Constrained Consensus of Multiagent Networks
Authors: Xiaotian Wang, Housheng Su
Date: 2022-01-05 00:00:00

This paper studies the consensus problem for multiagent systems with transmission constraints. A novel model of multiagent systems is proposed where the information transmissions between agents are disturbed by irregular distortions or interferences (named transmission constraint functions), and this model is universal which can be applied in many cases, such as interval consensus and discarded consensus. In the transmission-constrained consensus problem, we obtain the necessary and sufficient condition that agents can converge to state consensus. Furthermore, a more general case is studied in which the system reaches an equilibrium. Based on some techniques of algebraic topology and stability theory, the existence, uniqueness and stability of the system equilibrium point can be proven, which means the system can reach an asymptotically stable equilibrium. Moreover, the state values of the equilibrium are only decided by the network structure and transmission constraint functions, but not the agents' initial states. Finally, numerical simulations are presented to illustrate the proposed theorems and corollaries.