Reviewerly/exHarmony huggingface dataset:
PDAS

To improve Quality of Experience (QoE) for short video applications, most commercial companies adopt preloading and adaptive streaming technologies concurrently. Though preloading can reduce rebuffering, it may greatly waste bandwidth if the downloaded video chunks are not played. Also, each short video's downloading competes against others, which makes the existing adaptive streaming technologies fail to optimize the QoE for all videos. In this paper, we propose PDAS, a Probability-Driven Adaptive Streaming framework, to minimize the bandwidth waste while guaranteeing QoE simultaneously. We formulate PDAS into an optimization problem, where a probabilistic model is designed to describe the swiping events. Then, the maximum preload size is controlled by the proposed probability-driven max-buffer model, which reduces the bandwidth waste by proactively sleeping. At last, the optimization problem is solved by jointly deciding the preload order and preload bitrate. Extensive experimental results demonstrate that PDAS achieves almost 22.34% gains on QoE and 22.80% reductions on bandwidth usage against the existing methods. As for online evaluation, PDAS ranks first in the ACM MM 2022 Grand Challenge: Short Video Streaming.
Rethinking Open-World Object Detection in Autonomous Driving Scenarios

Existing object detection models have been demonstrated to successfully discriminate and localize the predefined object categories under the seen or similar situations. However, the open-world object detection as required by autonomous driving perception systems refers to recognizing unseen objects under various scenarios. On the one hand, the knowledge gap between seen and unseen object categories poses extreme challenges for models trained with supervision only from the seen object categories. On the other hand, the domain differences across different scenarios also cause an additional urge to take the domain gap into consideration by aligning the sample or label distribution. Aimed at resolving these two challenges simultaneously, we firstly design a pre-training model to formulate the mappings between visual images and semantic embeddings from the extra annotations as guidance to link the seen and unseen object categories through a self-supervised manner. Within this formulation, the domain adaptation is then utilized for extracting the domain-agnostic feature representations and alleviating the misdetection of unseen objects caused by the domain appearance changes. As a result, the more realistic and practical open-world object detection problem is visited and resolved by our novel formulation, which could detect the unseen categories from unseen domains without any bounding box annotations while there is no obvious performance drop in detecting the seen categories. We are the first to formulate a unified model for open-world task and establish a new state-of-the-art performance for this challenge.
An End-to-End Conditional Generative Adversarial Network Based on Depth Map for 3D Craniofacial Reconstruction

Craniofacial reconstruction is fundamental in resolving forensic cases. It is rather challenging due to the complex topology of the craniofacial model and the ambiguous relationship between a skull and the corresponding face. In this paper, we propose a novel approach for 3D craniofacial reconstruction by utilizing Conditional Generative Adversarial Networks (CGAN) based on craniofacial depth map. More specifically, we treat craniofacial reconstruction as a mapping problem from skull to face. We represent 3D cran- iofacial shapes with depth maps, which include most craniofacial features for identification purposes and are easy to generate and apply to neural networks. We designed an end-to-end neural networks model based on CGAN then trained the model with paired craniofacial data to automatically learn the complex nonlinear relationship between skull and face. By introducing body mass index classes(BMIC) into CGAN, we can realize objective reconstruction of 3D facial geometry according to its skull, which is a complicated 3D shape generation task with different topologies. Through comparative experiments, our method shows accuracy and verisimilitude in craniofacial reconstruction results.
Novel chaotic oppositional fruit fly optimization algorithm for feature selection applied on COVID 19 patients’ health prediction

The fast-growing quantity of information hinders the process of machine learning, making it computationally costly and with substandard results. Feature selection is a pre-processing method for obtaining the optimal subset of features in a data set. Optimization algorithms struggle to decrease the dimensionality while retaining accuracy in high-dimensional data set. This article proposes a novel chaotic opposition fruit fly optimization algorithm, an improved variation of the original fruit fly algorithm, advanced and adapted for binary optimization problems. The proposed algorithm is tested on ten unconstrained benchmark functions and evaluated on twenty-one standard datasets taken from the Univesity of California, Irvine repository and Arizona State University. Further, the presented algorithm is assessed on a coronavirus disease dataset, as well. The proposed method is then compared with several well-known feature selection algorithms on the same datasets. The results prove that the presented algorithm predominantly outperform other algorithms in selecting the most relevant features by decreasing the number of utilized features and improving classification accuracy.
Look Less Think More: Rethinking Compositional Action Recognition

Compositional action recognition which aims to identify the unseen combinations of actions and objects has recently attracted wide attention. Conventional methods bring in additional cues (e.g., dynamic motions of objects) to alleviate the inductive bias between the visual appearance of objects and the human action-level labels. Besides, compared with non-compositional settings, previous methods only pursue higher performance in compositional settings, which can not prove their generalization ability. To this end, we firstly rethink the problem and design a more generalized metric (namely Drop Ratio) and a more practical setting to evaluate the compositional generalization of existing action recognition algorithms. Beyond that, we propose a simple yet effective framework, Look Less Think More (LLTM), to reduce the strong association between visual objects and action-level labels (Look Less), and then discover the commonsense relationships between object categories and human actions (Think More). We test the rationality of the proposed Drop Ratio and Practical setting by comparing several popular action recognition methods on SSV2. Besides, the proposed LLTM achieves state-of-the-art performance on SSV2 with different settings.
Differentiable Cross-modal Hashing via Multimodal Transformers

Cross-modal hashing aims at projecting the cross modal content into a common Hamming space for efficient search. Most existing work first encodes the samples with a deep network and then binaries the encoded feature into hashing code. However, the relative location information in the image may be lost when an image is encoded by the convolutional network, which makes it challenging to model the relationship of different modalities. Moreover, it is NP-hard to optimize the model with the discrete sign binary function popularly used in existing solutions. To address these issues, we propose a differentiable cross-modal hashing method that utilizes the multimodal transformer as the backbone to capture the location information in an image when encoding the visual content. In addition, a novel differentiable cross-modal hashing method is proposed to generate the binary code by a selecting mechanism, which could be formulated as a continuous and easily optimized problem. We perform extensive experiments on several cross modal datasets and the results show that the proposed method outperforms many existing solutions.
ABPN

Recently, Micro expression~(ME) has achieved remarkable progress in a wide range of applications, since it's an involuntary facial expression that reflects personal psychological state truly. In the procedure of ME analysis, spotting ME is an essential step, and is non trivial to be detected from a long interval video because of the short duration and low intensity issues. To alleviate this problem, in this paper, we propose a novel Micro- and Macro-Expression~(MaE) Spotting framework based on Apex and Boundary Perception Network~(ABPN), which mainly consists of three parts, i.e., video encoding module ~(VEM), probability evaluation module~(PEM), and expression proposal generation module~(EPGM). Firstly, we adopt Main Directional Mean Optical Flow (MDMO) algorithm and calculate optical flow differences to extract facial motion features in VEM, which can alleviate the impact of head movement and other areas of the face on ME spotting. Then, we extract temporal features with one-dimension convolutional layers and introduce PEM to infer the auxiliary probability that each frame belongs to an apex or boundary frame. With these frame-level auxiliary probabilities, the EPGM further combines the frames from different categories to generate expression proposals for the accurate localization. Besides, we conduct comprehensive experiments on MEGC2022 spotting task, and demonstrate that our proposed method achieves significant improvement with the comparison of state-of-the-art baselines on rm CAS(ME)2 and SAMM-LV datasets. The implemented code is also publicly available at https://github.com/wenhaocold/USTC_ME_Spotting.
Automatic arrival-time picking of P- and S-waves of microseismic events based on object detection and CNN

Automatic, quick and accurate picking of the arrival-times of both the P- and S-waves of microseismic (MS) events is essential to real-time data processing for MS monitoring. This study proposes a two-stage arrival-time picking method that involves extracting the region of interest of the waveform and P- and S-wave arrival-time prediction. The dataset collects 6107 MS signals with low and high signal-to-noise ratios to train and test the proposed method. The YOLO network is employed to extract the MS signal region to narrow the range of arrival-time picking and significantly improve the problem of serious interference caused by excessive negative samples in the MS signal. Then, the convolutional neural network is utilised further to precisely pick the arrival-times of both the P- and S-waves. The results showed that the proposed two-stage method is more accurate than the one-stage method in picking the arrival-time. The effects of different convolutional kernel sizes and fully connected layers on the two-stage model are also discussed. Furthermore, a new multi-index comprehensive evaluation method that considers both the accuracy and the size of the model is proposed to further evaluate the proposed model's practicality. This study presents a new method suitable for both P- and S-wave arrival-time picking to enable MS monitoring, which is valuable for various rock engineering applications.
Review of Development and Application of Digital Image Correlation Method for Study of Stress–Strain State of RC Structures

Reliable assessment and prediction of the technical condition of reinforced concrete structures require accurate data of the stress–strain state of the structure at all stages of loading. The most appropriate technique to obtain such information is digital image correlation. Digital image correlation is a class of contactless methods which includes the following stages: obtaining an image from a studied physical object, saving it in digital form, and further analysis in order to obtain the necessary information about the stress–strain state of the structure. In this research, a detailed analysis of theoretical and experimental findings of digital image correlations was conducted. In the article, the main areas of scientific interest and computational approaches in digital image correlation issues were identified. Moreover, comparative analysis of alternative non-contact techniques, which also could be used for diagnostics of RC structures’ stress–strain state was conducted. The novelty of the study consists of a thorough comparative analysis with the indication of specific features of digital image correlation, which determine its wide application among the other similar methods. On the basis of the conducted literature review, it can be seen that the digital image correlation technique has gone through multi-stage evolution and transformation. Among the most widely studied issues are: image recognition and matching procedures, calibration methods and development of analytical concepts. The digital image correlation technique enables us to study cracking and fracture processes in structural elements, obtaining the full field of deformations and stresses. Further development of image processing methods would provide more precise measuring of stress–strain parameters and reliable assessment of structural behavior.
How digital devices transform literary reading: The impact of e-books, audiobooks and online life on reading habits

Literary reading is under transformation. Digital devices supplement traditional paper books with e-books and audiobooks, and at the same time, ubiquitous digital connection challenges focused reading. Based on a qualitative interview study with adult leisure readers, this article explores how affordances offered by digital technologies influence reading habits. Informants demonstrate how e-books and audiobooks enhance reading experiences, as digital affordances influence the how and the when of literary consumption. Three prominent findings are stated. (1) Readers adapt reading mode to the situation, and experienced readers have developed strategies to maximise the ultimate combination of title, format and reading conditions. (2) Digital reading favours lighter texts. This dimension is more substantial for audiobooks, relating to the wide choice of combining audiobooks with other activities. (3) Being devoted readers motivates people to develop strategies to ensure further reading. These strategies effectively make readers practically and temporally disconnect to immerse in literature.
Object detection in high resolution optical image based on deep learning technique

Deep learning-based object detection in high resolution optical imagery is an active research frontier. Dataset for object detection is fundamentally important to drive deep learning models. Compared to datasets in computer vision, there is a lack of datasets for object detection in the remote sensing field. This work produced a high-quality dataset for object detection based on the high-resolution remote sensing images of the TripleSAT. The spatial resolution of the TripleSAT images is 0.8 ​m and the dataset for object detection includes 3 categories: wind turbine, airplane and oil storage tank. 3583 TripleSAT images are used, in which 7320 instances are manually annotated, to compose a TripleSAT dataset. To test the performance of the TripleSAT dataset, a state-of-the-art deep learning model, the RetinaNet was used with the PyTorch framework. Our results show that the mean average precision of the RetinaNet for wind turbines, airplanes and oil storage tanks are 93.47%, 92.8% and 96.55%, respectively. The results demonstrate the capability of the TripleSAT satellite images in producing datasets for deep learning models. Dataset made from single remote sensing sensor can achieve high accuracy and fast detection. The proposed dataset, including wind turbine, oil storage tank and airplane is meaningful supplement for data on hazard-affected bodies.
Investigating the role of auditory and visual sensory inputs for inducing relaxation during virtual reality stimulation

Stress is a part of everyday life which can be counteracted by evoking the relaxation response via nature scenes presented using immersive virtual reality (VR). The aim of this study was to determine which sensory aspect of immersive VR intervention is responsible for the greatest relaxation response. We compared four conditions: auditory and visual combined (audiovisual), auditory only, visual only, and no artificial sensory input. Physiological changes in heart rate, respiration rate, and blood pressure were recorded, while participants reported their preferred condition and awareness of people, noise, and light in the real-world. Over the duration of the stimulation, participants had the lowest heart rate during the audiovisual and visual only conditions. They had the steadiest decrease in respiration rate and the lowest blood pressure during the audiovisual condition, compared to the other conditions, indicating the greatest relaxation. Moreover, ratings of awareness indicated that participants reported being less aware of their surroundings (i.e., people, noise, light, real environment) during the audiovisual condition versus the other conditions (p < 0.001), with a preference for audiovisual inputs. Overall, the use of audiovisual VR stimulation is more effective at inducing a relaxation response compared to no artificial sensory inputs, or the independent inputs.
Robust watermarking algorithm for medical images based on accelerated‐KAZE discrete cosine transform

With the continuous progress and development in the field of Internet technology, the area of medical image processing has also developed along with it. Specially, digital watermarking technology plays an essential role in the field of medical image processing and greatly improves the security of medical image information. A medical image watermarking algorithm based on an accelerated-KAZE discrete cosine transform (AKAZE-DCT) is proposed to address the poor robustness of medical image watermarking algorithms to geometric attacks, which leads to low security of the information contained in medical images. First, the AKAZE-DCT algorithm is used to extract the feature vector of the medical image and then combined with the perceptual hashing technique to obtain the feature sequence of the medical image; then, the watermarking image is encrypted with logistic chaos dislocation to get the encrypted watermarking image, which ensures the security of the watermarking information; finally, the watermarking is embedded and extracted with the zero-watermarking technique. The experimental results show that the algorithm can effectively extract the watermark under conventional and geometric attacks, reflecting better robustness and invisibility, and has certain practicality in the medical field compared with other algorithms.
Moving beyond cyber security awareness and training to engendering security knowledge sharing

Employees play a critical role in improving workplace cyber security, which builds on widespread security knowledge and expertise. To maximise knowledge levels, organisations run awareness and training course. Yet, they should also encourage and facilitate Security Knowledge Sharing (SKS). To facilitate such sharing, we used a bespoke App which deploys a game to deliver security training and to encourage sharing based on the Transactive Memory System (TMS) theory. An empirical study was conducted within a Saudi Arabian Fortune 100 organisation to test the impact of the app on employee knowledge. The app demonstrated efficacy in enhancing organisational security awareness and knowledge. The results highlight the potential of TMS in improving overall security knowledge in organisations.
Toward Intelligent Fashion Design: A Texture and Shape Disentangled Generative Adversarial Network

Texture and shape in fashion, constituting essential elements of garments, characterize the body and surface of the fabric and outline the silhouette of clothing, respectively. The selection of texture and shape plays a critical role in the design process, as they largely determine the success of a new design for fashion items. In this research, we propose a texture and shape disentangled generative adversarial network (TSD-GAN) to perform “intelligent” design with the transformation of texture and shape in fashion items. Our TSD-GAN aims to learn how to disentangle the features of texture and shape of different fashion items in an unsupervised manner. Specifically, a fashion attribute encoder is developed to decompose the input fashion items into independent representations of texture and shape. Then, to learn the coarse or fine styles hidden in the features of texture and shape, a texture mapping network and a shape mapping network are proposed to disentangle the features into different hierarchical representations. The different hierarchical representations of texture and shape are then fed into a multi-factor-based generator to generate mixed-style fashion items. In addition, a multi-discriminator framework is developed to distinguish the authenticity and texture similarity between the generated images and the real images. Experimental results on different fashion categories demonstrate that our proposed TSD-GAN may be useful for assisting designers to accomplish the design process by transforming the texture and shape of fashion items.
Towards improved multifactorial particle swarm optimization learning of fuzzy cognitive maps: A case study on air quality prediction

Fuzzy cognitive map (FCM) is a very simple, efficient, and powerful soft computing tool for modeling and analysis of a complex system. Due to its simplicity and transparency, FCM has been widely utilized in engineering, environment, medicine, and other complex systems. However, it remains challenging to handle dynamic, non-stationary, and noisy time series, such as air quality monitoring data, which is with typical temporal periodicity, cross-interference, low-quality, and great noise. Concerning the above challenges, we propose an improved multifactorial particle swarm optimization learning algorithm of FCM, termed as IMFPSO-FCM. Within the framework of IMFPSO-FCM, the learning of an FCM is regarded as a multitask optimization problem. Every task represents learning local connections of a node, and a single population is adopted to process these tasks simultaneously. The task selection mechanism is used to automatically select appropriate target tasks, thereby suppressing negative transfer and enabling useful information to be transferred between tasks. In addition, a multi-dimensional adaptive inertia weight strategy and a local search strategy are employed to further improve the performance of the model. The performance of IMFPSO-FCM is validated on several public datasets and real-world air quality monitoring datasets. The experimental results demonstrate the performance of the development method and emphasize its practicality.