Reviewerly/exHarmony huggingface dataset:
Analysis and Extensions of Adversarial Training for Video Classification

Adversarial training (AT) is a simple yet effective defense against adversarial attacks to image classification systems, which is based on augmenting the training set with attacks that maximize the loss. However, the effectiveness of AT as a defense for video classification has not been thoroughly studied. Our first contribution is to show that generating optimal attacks for video requires carefully tuning the attack parameters, especially the step size. Notably, we show that the optimal step size varies linearly with the attack budget. Our second contribution is to show that using a smaller (sub-optimal) attack budget at training time leads to a more robust performance at test time. Based on these findings, we propose three defenses against attacks with variable attack budgets. The first one, Adaptive AT, is a technique where the attack budget is drawn from a distribution that is adapted as training iterations proceed. The second, Curriculum AT, is a technique where the attack budget is increased as training iterations proceed. The third, Generative AT, further couples AT with a denoising generative adversarial network to boost robust performance. Experiments on the UCF101 dataset demonstrate that the proposed methods improve adversarial robustness against multiple attack types.
Medusa: Universal Feature Learning via Attentional Multitasking

Recent approaches to multi-task learning (MTL) have focused on modelling connections between tasks at the decoder level. This leads to a tight coupling between tasks, which need retraining if a new task is inserted or removed. We argue that MTL is a stepping stone towards universal feature learning (UFL), which is the ability to learn generic features that can be applied to new tasks without retraining.We propose Medusa to realize this goal, designing task heads with dual attention mechanisms. The shared feature attention masks relevant backbone features for each task, allowing it to learn a generic representation. Meanwhile, a novel Multi-Scale Attention head allows the network to better combine per-task features from different scales when making the final prediction. We show the effectiveness of Medusa in UFL (+13.18% improvement), while maintaining MTL performance and being 25% more efficient than previous approaches.
Unpaired Faces to Cartoons: Improving XGAN

Domain Adaptation is a task that aims to translate an image from a source domain to a desired target domain. Current methods in domain adaptation use adversarial training based on Generative Adversarial Networks (GAN). In the present work, we focus on the task of domain adaptation from real faces to cartoon face images. We start from a baseline architecture called XGAN and introduce some improvements to it. Our proposed model is called W-XDGAN, which uses a form of GAN called Wasserstein-GAN, learns to approximate the Wasserstein Distance, and adds a denoiser to smooth the output cartoons. Whereas the original XGAN paper only presented a qualitative analysis, the advantages of this solution are demonstrated both quantitatively and qualitatively by comparing the results with models such as UNIT and original XGAN. Our code and models are publicly available at https://github.com/IAmigos/avatar-image-generator.
Multi-Attribute Joint Point Cloud Super-Resolution with Adversarial Feature Graph Networks

3D point cloud super-resolution (PCSR) plays an important role in many applications, which can infer a dense geometric shape from a sparse one. However, existing PCSR methods only leverage the geometric properties to predict dense geometric coordinates without considering the importance of correlated attributes in the prediction of complex geometric structures. In this paper, we propose a novel PCSR network by leveraging color attributes to improve the reconstruction quality of dense geometric shape. In the proposed network, we utilize graph convolutions to obtain cross-domain structure representation for point cloud from both geometric coordinates and color attributes, which is constructed by aggregating local points based on the similarity of cross-domain features. Furthermore, we propose a shape-aware loss function to cooperate with network training, which constrains the point cloud generation from both overall and detailed aspects. Extensive experimental results show that our proposed method outperforms the state-of-the-art methods from both objective and subjective quality.
Conditional Sentence Rephrasing without Parallel Training Corpus

This paper aims to rephrase a sentence with a given condition, and the generated sentence should be similar to the origin sentence and satisfy the given condition without parallel training corpus. We propose a conditional sentence VAE (CS-VAE) model to solve the task. CS-VAE is trained as an autoencoder, along with the condition control on the generated sentence with the same semantics. With the experimental demonstration supported, CS-VAE is proven to effectively solve the task with high-quality sentences.
Demusa: Demo for Multimodal Sentiment Analysis

Recently, a lot of Multimodal Sentiment Analysis (MSA) models appeared to understanding opinions in multimedia. To accelerate MSA researches, CMU-MOSI and CMU-MOSEI were released as the open-datasets. However, it is hard to observe the input data elements in detail and analyze the prediction model results with each video clip for qualitative evaluation. For these reasons, this paper suggests DeMuSA, demo for multimodal sentiment analysis to explore raw data instance and compare prediction models by utterance-level.
Integrating Immersive Virtual Environment User Studies into Architectural Design Practice: A Pre-Occupancy User Study of Train Station Waiting Preferences With VREVAL

User-centered studies in Immersive Virtual Environments (IVEs) are able to provide valuable feedback in the form of Pre-Occupancy Evaluations (Pr-OEs). Pr-OEs allow for immersive design reviews of architectural space before construction is complete, thus providing better opportunities for user-centered values to be appraised and addressed by designers. If integrated into the architectural design process, PrOEs can also aid in design development by comparing participant responses to multiple design variations.In this paper we present the further development of VREVAL, a framework for performing user studies in IVEs, and a case study of its application in the study of waiting preferences. An overview of the VREVAL web platform and desktop application explain the modular study components within a BIM workflow. An example study showcases the VREVAL “Placing” module while examining participant waiting preferences in four alternative designs for a train station building in Germany.
Influence of Air Flow Rate and Evaporation Surface on Cooling of Waste Pool Water Produced by Thermal Baths

Thermal baths in Slovakia produce waste pool water, which is most often discharged to the water recipient. Cooling of waste pool water is ensured by means of cooling ponds or canals. However, cooling the water from the pool is not sufficient. Therefore, thermal bath operators face sanctions for environmental pollution. The possibility of ensuring the maximum temperature of waste pool water is by means of a heat recovery system, which will be presented in the article. If it is not possible to apply a heat recovery system, it is necessary to ensure sufficient cooling of the water in another way. Another possibility of increasing the cooling of wastewater is to ensure a sufficient evaporate area which has a great effect on water cooling. Another crucial parameter that affects the cooling of wastewater is the speed of air flow above the water surface. The article is focused on influence of air flow rate and evaporation surface on cooling of waste pool water produced by thermal baths and briefly introduces another wastewater cooling option, which is a heat recovery system. The results presented in the article indicate a significant impact of the evaporation area on the heat flow through the evaporation.
An Iot Based Smart Monitoring System Detecting Patient Falls

Fall detection systems are important instruments for senior citizens whether they are in a hospital, at home, or in a senior living facility. Existing fall detection systems are often passive and require pushbutton to alarm care providers. The delay can cause significant consequences and damages to the patients. Early detection is vital for quick recovery and prevention of post injuries. Recent sensory detections often use pre-settings to predict fall, hence can cause false alarms. Further, they often cannot furnish vital information for treatment. A smart fall detection system can alleviate those concerns and pinpoint what happened and transfers vital data. We propose an active detection system using IoT in patient’s surroundings environment detecting falls, which immediately alarms the care providers and transmits vital information for treatment. The system integrates advanced ICT technologies to monitor, detect, and react to the falls. The system uses ANN to improve fall detections.
Detic-Track: Robust Detection and Tracking of Objects in Video

The automatic detection and tracking of objects in a video is crucial for many video understanding tasks. We propose a novel deep learning based algorithm for object detection and tracking, which is able to detect more than 1,000 object classes and tracks them robustly, even for challenging content. The robustness of the tracking is due to the usage of optical flow information. Additionally, we utilize only the part of the bounding box corresponding to the object shape for the tracking.
Deep Point Cloud Normal Estimation via Triplet Learning (Demonstration)

In this demonstration paper, we show the technical details of our proposed triplet learning-based point cloud normal estimation method. Our network architecture consists of two phases: (a) feature encoding to learn representations of local patches, and (b) normal estimation that takes the learned representations as input to regress normals. We are motivated that local patches on isotropic and anisotropic surfaces respectively have similar and distinct normals, and these separable representations can be learned to facilitate normal estimation. Experiments show that our method preserves sharp features and achieves good normal estimation results especially on computer-aided design (CAD) shapes.
Efficient Topology Coding and Payload Partitioning Techniques for Neural Network Compression (NNC) Standard

A Neural Network Compression (NNC) standard aims to define a set of coding tools for efficient compression and transmission of neural networks. This paper addresses the high-level syntax (HLS) of NNC and proposes three HLS techniques for network topology coding and payload partitioning. Our first technique provides an efficient way to code prune topology information. It removes redundancy in the bitmask and thereby improves coding efficiency by 4–99% over existing approaches. The second technique processes bitmasks in larger chunks instead of one bit at a time. It is shown to reduce computational complexity of NNC encoding by 63% and NNC decoding by 82%. Our third technique makes use of partial data counters to partition an NNC bitstream into uniformly sized units for more efficient data transmission. Even though the smaller partition sizes introduce some overhead, our network simulations show better throughput due to lower packet retransmission rates. To our knowledge, this the first work to address the practical implementation aspects of HLS. The proposed techniques can be seen as key enabling factors for efficient adaptation and economical deployment of the NNC standard in a plurality of next-generation industrial and academic applications.
Specification and Applications of Online Sharing Interface for Geospatial information

In order to support the sharing and interoperability of multi-source and heterogeneous geospatial data in the wen environment, the use of standardized interface is an effective means. This paper analyzes the differences of geospatial data from multiple levels such as data source, metadata model and data interface. The NOAA CLASS data and NASA ECHO data system are adopted to demonstrate the usage of geospatial online sharing. Combined with the current relevant standards of ISO and OGC, this paper puts forward the solution of online sharing from the perspective of standards, and gives an application demonstration in combination with the needs of agricultural applications.
Video Object Segmentation with Online Mask Refinement

This paper proposes a simple and effective video object instance segmentation method without fine-tuning named Mask Refinement Module(MRM). Many papers settle a labeling problem aiming to separate foreground objects, but most of them require training their networks again on target data. In a real scenario, it is not easy to collect dataset on the target environment and to label them as well due to security policies or a cost problem, especially for industry. We solve the problem by reshaping object masks with a video based online-learning method that enables us to adapt various changes frame by frame. In extensive experiments, results show that our approach is highly effective compared to modern methods by up to 13.9% improving of F-measure on large video surveillance dataset such as CDNet (118K images).
Local to Global Transformer for Video Based 3d Human Pose Estimation

Transformer-based architecture has achieved great results in sequence to sequence tasks and vision tasks including 3D human pose estimation. However, transformer based 3D human pose estimation method is not as strong as RNN and CNN in terms of local information acquisition. Additionally, local information plays a major role in obtaining 3D positional relationships. In this paper, we propose a method that combines local human body parts and global skeleton joints using a temporal transformer to finely track the temporal motion of human body parts. First, we encode positional and temporal information, then we use a local to global temporal transformer to obtain local and global information, and finally we obtain the target 3D human pose. To evaluate the effectiveness of our method, we quantitatively and qualitatively evaluated our method on two popular and standard benchmark datasets: Human3.6M and HumanEva-I. Extensive experiments demonstrated that we achieved state-of-the-art performance on Human3.6M with 2D ground truth as input.
OPSE: Online Per-Scene Encoding for Adaptive Http Live Streaming

In live streaming applications, typically a fixed set of bitrateresolution pairs (known as a bitrate ladder) is used during the entire streaming session in order to avoid the additional latency to find scene transitions and optimized bitrateresolution pairs for every video content. However, an optimized bitrate ladder per scene may result in (i) decreased storage or delivery costs or/and (ii) increased Quality of Experience (QoE). This paper introduces an Online Per-Scene Encoding (OPSE) scheme for adaptive HTTP live streaming applications. In this scheme, scene transitions and optimized bitrate-resolution pairs for every scene are predicted using Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features. Experimental results show that, on average, OPSEyields bitrate savings of up to 48.88% in certain scenes to maintain the same VMAF, compared to the reference HTTP Live Streaming (HLS) bitrate ladder without any noticeable additional latency in streaming.
Integer Network for Cross Platform Graph Data Lossless Compression

It has been witnessed that the learned data compression techniques has outperformed conventional ones. However, the non-deterministic floating-point calculation makes the probability prediction inconsistent between sender and receiver, disabling practical applications. We propose to use the integer network to relieve this problem and focus on graph data lossless compression. Firstly, we propose an adaptive fixed-point format, AdaFixedPoint, which can convert a floating-point model, which has graph convolution layers to a fixed-point one with minimal precision loss and enable deterministic graph data lossless compression. Secondly, we propose QbiasFree Compensation and Bin Regularization to quantize the network with fewer bits, relieving the computation cost. Experiments show that our proposed integer network can achieve successful cross-platform graph data compression. And compared with the commonly used 8 bits, our method remarkably decreases the quantized average bit to 5 bits, without a performance drop.
Tachiegan: Generative Adversarial Networks for Tachie Style Transfer

Tachie painting is an emerging digital portrait art form that shows a character in a standing pose. Automatic generation of a Tachie picture from a real photo would facilitate many creation tasks. However, it is non-trivial to represent Tachie’s artistic styles and establish a delicate mapping from the real-world image domain to the Tachie domain. Existing approaches generally suffer from inaccurate style transformation and severe structure distortion when applied to Tachie style transfer. In this paper, we propose the first approach for Tachie stylization of portrait photographs. Based on the unsupervised CycleGAN framework, we design two novel loss functions to emphasize lines and tones in the Tachie style. Furthermore, we design a character-enhanced stylization framework by introducing an auxiliary body mask to better preserve the global body structure. Experiment results demonstrate the robustness and better generation capability of our method in Tachie stylization from photos in a wide range of poses, even trained on a small dataset.
Towards the Web of Embeddings: Integrating multiple knowledge graph embedding spaces with FedCoder

The Semantic Web is distributed yet interoperable: Distributed since resources are created and published by a variety of producers, tailored to their specific needs and knowledge; Interoperable as entities are linked across resources, allowing to use resources from different providers in concord. Complementary to the explicit usage of Semantic Web resources, embedding methods made them applicable to machine learning tasks. Subsequently, embedding models for numerous tasks and structures have been developed, and embedding spaces for various resources have been published. The ecosystem of embedding spaces is distributed but not interoperable: Entity embeddings are not readily comparable across different spaces. To parallel the Web of Data with a Web of Embeddings, we must thus integrate available embedding spaces into a uniform space. Current integration approaches are limited to two spaces and presume that both of them were embedded with the same method — both assumptions are unlikely to hold in the context of a Web of Embeddings. In this paper, we present FedCoder— an approach that integrates multiple embedding spaces via a latent space. We assert that linked entities have a similar representation in the latent space so that entities become comparable across embedding spaces. FedCoder employs an autoencoder to learn this latent space from linked as well as non-linked entities. Our experiments show that FedCoder substantially outperforms state-of-the-art approaches when faced with different embedding models, that it scales better than previous methods in the number of embedding spaces, and that it improves with more graphs being integrated whilst performing comparably with current approaches that assumed joint learning of the embeddings and were, usually, limited to two sources. Our results demonstrate that FedCoder is well adapted to integrate the distributed, diverse, and large ecosystem of embeddings spaces into an interoperable Web of Embeddings.
MOFN: Multi-Offset-Flow-Based Network for Video Restoration and Enhancement

Video restoration and enhancement tasks, including video super-resolution(VSR), are designed to convert low-quality videos into high-quality videos to improve the audience’s visual experience. In recent years, many deep learning methods using optical flow estimation or deformable convolution have been applied to video super-resolution. However, we find that motion estimation based on a single optical flow is difficult to capture enough inter-frame information, and the method using deformable convolution lacks clear motion constraints, which affects its ability to process fast motion. Therefore, we propose a multi-offset-flow-based network (MOFN) to make more effective use of inter-frame information by using optical flow with offset diversity. We proposed an alignment and compensation module that can estimate the optical flow with multiple offsets for neighbouring frames and perform frame alignment. The aligned video frames will be fed into the fusion module, and high-quality video frames will be obtained after fusion and reconstruction. Extensive results show that our proposed model has a good ability to process motion. On several benchmark datasets, our method has achieved favorable performance compared with the most advanced methods.
Postdigital Possibilities and Impossibilities Behind the Screen: Visual Arts Educators in Conversation about Online Learning and Real-world Experiences

Abstract Through an ethnomethodological and dialogical encounter with Australian classrooms in the lived experience of two visual art ( va ) educators, the authors seek to learn how working between online and studio learning approaches shaped teacher perceptions of student learning during the outbreak of covid -19 in 2020 and 2021. The research has two phases. Phase 1 sees the two va educators create learning narratives. These narratives, reported in summary in the article, through both material and digital form became the baseline data. In Phase 2 these themes were reworked as conversational questions. These questions then became the stimulus for a critical reflective online video conversation between the two va educators. The resulting discussion around the borderlines looks beyond specific apps, platforms, or products that the teachers used, their successes and failures and examines the digital, non-digital, material, social relations and pedagogical realities and futures that may or may be possible in the context of the postdigital va secondary classroom. These educators have had little time to assess the shift from a strong and well researched studio-pedagogy to their virtual creative learning futures. The challenges of this shift are revealed through their personal experiences.
Mapping of mafic-ultramafic rocks in SMUC-SGT, India using ASTER &amp; Sentinel-2A satellite images

The Salem Mafic–Ultramafic Complex (SMUC) is a major complex in southern Granulite Terrane (SGT), in the northern part of the Cauvery Suture Zone (CSZ), India, also known as the Chalk Hills of Salem. The present study aims to review the efficiency of satellite image processing and related techniques to map the Mafic-Ultramafic rocks in Salem Mafic-Ultramafic Complex (SMUC). This mapping is done using atmospherically corrected Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) and Sentinel-2A satellite image data. The processes employed were image enhancement techniques like Band Combinations (BC), Principal Component Analysis (PCA), and Minimum Noise Fraction (MNF) to map the various rock types. Band combinations constructed from the output of PCA and MNF in RGB format showed promising results in discriminating various rock units. The Spectral Angle Mapper (SAM) and Support Vector Machine (SVM) classifications techniques were used to classify ASTER and Sentinel-2A satellite images for the study area. The classified lithologies showed strong match with the lithology map published by the Geological Survey of India (GSI), other researches, and field investigation. Particularly, SVM classification of Sentinel-2A MNF + DEM showed 90% overall accuracy when compared with field investigation. The study proved that the processing of VNIR and SWIR bands of ASTER and Sentinel-2A Satellite images coupled with DEM and ground investigation has excellent potential in mapping Mafic-Ultramafic rocks in SMUC SGT.