Reviewerly/exHarmony huggingface dataset:
3D Winograd Layer with Regular Mask Pruning

The high computational cost and huge amount of parameters of 3D Convolutional Neural Networks(CNN) limit the deployment of 3D CNN-based models to lightweight devices with weak computing ability. Applying Winograd layer that combines the idea of pruning and Winograd algorithms to 3D convolution is a promising solution. However, the irregular and unstructured sparsity of 3D Winograd layer makes it difficult to implement efficient compression. In this paper, we propose Regular Mask Pruning to obtain the regular sparse 3D Winograd layer that can be easily compressed. Furthermore, we present Masked 3D Winograd Layer, which can store the compressed parameters of regular sparse Winograd Layer and reduce the number of multiplications in inference. For C3D-based Winograd model, we obtain a regular sparsity of 38.7% without accuracy loss. By our method, the parameters of Winograd layers and the multiplication operations of 3D convolution can be reduced up to 5.32× and 18× respectively.
Adversarial Machine Learning Using Convolutional Neural Network With Imagenet

Adversarial attacks are types of attacks where adversaries try to deceive the machine learning algorithm by providing deceptive input. Adversarial attacks are focused on providing inaccurate data at its training phase, or introducing maliciously designed data to deceive an already trained model. Adversarial instances are an excellent element of security to focus on because they represent a concrete challenge that AI is currently facing, and they are also a challenging component of security to work on because it requires a significant amount of research effort and time. Systems that are currently in use are very vulnerable to adversarial attacks. It is as simple as vandalizing traffic signs for the self-driving car to make mistakes. Any machine learning model is easy to attack, as we can feed them with malicious and wrong data input with ease. This paper focuses on the Adversarial manipulation of the Machine Learning Algorithm and illustrates how attacks are curated towards an Image-based trained model. A series of experiments based on the ImageNet images and pre-trained MobileNet, Convolutional Neural Network(CNN) model from TensorFlow is used to show how an adversarial attack based on Images can be curated to outwit the machine learning model. This paper will primarily focus on the AI/ML algorithm manipulation with one-pixel, multi-pixel, and all-pixel attacks on the TensorFlow pre-trained model.
CDTNET: Cross-Domain Transformer Based on Attributes for Person Re-Identification

Unsupervised Domain Adaptation (UDA) Person reidentification (ReID) strives towards fine-tuning the model trained on a labelled source-domain dataset to a target-domain dataset, which has grown by leaps and bounds due to the advancement of deep convolution neural network (CNN). However, traditional CNN-based methods mainly focus on learning small discriminative features in local pedestrian region, which fails to exploit the potential of rich structural patterns and suffers from information loss on details caused by convolution operators. To tackle the challenge, this work attempts to exploit the valuable fine-grained attributes based on Transformers. Inspired by this, we propose a Cross-Domain Transformer network CDTnet to enhance the robust feature learning in connection with pedestrian attributes. As far as we are aware, we are among the first attempt to adopt a pure transformer for cross-domain ReID research. All-inclusive experiments conducted on several ReID benchmarks demonstrate that our method can reach a comparable yield with reference to the state-of-the-arts.
Decentralized Federated Learning with Enhanced Privacy Preservation

We present a decentralized federated learning (FL) framework based on blockchain. In traditional federated learning, it is necessary that a third-party centralized server aggregates all the gradients which participant in the upload, but such a trusted third-party may not always exist. We address this issue with the decentralized blockchain and encrypt the neural network model parameters and gradients.
A Unified Video Summarization for Video Anomalies Through Deep Learning

Over the last ten years, integrated video surveillance systems have become increasingly important in protecting public safety. Because a single surveillance camera continuously collects events in a specific field of view at all times of day and night, a system that can create a summary that concisely captures key elements of the incoming frames is required. To be more specific, due to time constraints, the enormous amount of video footage cannot be properly examined for analysis. As a result, it is vital to compile a summary of what happened on the scene and look for anomalous events in the footage. A unified approach for detecting and summarizing anomalous events is proposed. To detect the event and compute the anomaly scores, a 3D deep learning approach is used. Afterward, the scores are utilized to visualize and localize the anomalous regions. Finally, the blob analysis technique is used to extract the anomalous regions. To verify the results, quantitative and qualitative evaluations are provided. Experiments indicate that the proposed summarizing method keeps crucial information while producing competitive results. More qualitative results can be found through our project channel: https://youtu.be/eMPMjiGlCQI
Server-Based Mixed-Reality System For Multiple Devices To Visualize A Large Architectural Model And Simulations

The use of mixed-reality systems in urban architecture allows people without specialized knowledge to intuitively understand the results of urban architectural planning and simulation. However, because the associated three-dimensional (3D) models involve large data volumes and heavy processing loads, they are difficult to process on devices commonly used for mixed-reality display, such as notebook PCs and smartphones. Here, we propose a method in which a server takes over the mixed-reality processing and allows multiple devices to display the 3D model simultaneously from the server. In this study, we verified the real-time performance of mixed-reality display on two devices while performing a daylight simulation with an indoor 3D model of approximately 2.6 million polygons. The results showed that multiple devices shared and displayed the 3D model with real-time performance that was unaffected by the data size up to 2.6 million polygons but was affected by the video-compression performance of the devices.
A Blockchain-Based Autonomous System for Primary Financial Market

The primary financial market has a large size, including both the debt market and equity market.
Towards a Semantics-Based Search Engine for Smart Contract Information

Most blockchains are known for transparency since all on-chain records are open and immutable. However, due to the particularity of the blockchain data structure, users cannot perform semantic searches in the blockchain as in traditional search engines, resulting in inefficient access to blockchain data. To solve this problem, we proposed approaches for semantic search in the blockchain. In particular, we focused on the extraction of semantic information and related data for smart contracts. In this paper, we describe the road map, challenges, and preliminary results of our research.
A Study on the Early Warning Index System of Road Traffic Risks Under Extreme Weather Conditions

Extreme weather frequently occurs in today's society; it is easy to produce natural disasters like snow, water, or ice accumulation in the development of urban construction, which directly affects the regular road traffic operation. In recent years, extreme weather conditions by monitoring data analysis indicate that the use of case history, statistics, or methods like fault number analysis enables an accurate identification of extreme weather and addresses hidden troubles in traffic. This paper establishes the corresponding index system of risk prediction and puts forward effective countermeasures to guarantee urban residents regular travel.
Did Your Manager Choose Your Architecture?

Most engineers would cringe at the idea of their managers choosing their system’s software architecture, but back in 1968, this is exactly what Melvin Conway found to be true. Across many companies, Conway found that the organizational chart, chosen by managers, was replicated in the software systems of those companies.
ARIA: Adversarially Robust Image Attribution for Content Provenance

Image attribution – matching an image back to a trusted source – is an emerging tool in the fight against online misinformation. Deep visual fingerprinting models have recently been explored for this purpose. However, they are not robust to tiny input perturbations known as adversarial examples. First we illustrate how to generate valid adversarial images that can easily cause incorrect image attribution. Then we describe an approach to prevent imperceptible adversarial attacks on deep visual fingerprinting models, via robust contrastive learning. The proposed training procedure leverages training on ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> -bounded adversarial examples, it is conceptually simple and incurs only a small computational overhead. The resulting models are substantially more robust, are accurate even on unperturbed images, and perform well even over a database with millions of images. In particular, we achieve 91.6% standard and 85.1% adversarial recall under ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> -bounded perturbations on manipulated images compared to 80.1% and 0.0% from prior work. We also show that robustness generalizes to other types of imperceptible perturbations unseen during training. Finally, we show how to train an adversarially robust image comparator model for detecting editorial changes in matched images. Project page: https://max-andr.github.io/robust_image_attribution.
Low-Power Semantic Segmentation on Embedded Systems for Traffic in Asian Countries

In the real-world road scene, it needs a quick and powerful system to detect any traffic situation and compute prediction results in a limited hardware environment. In this paper, we present a semantic segmentation model that can perform well in the complex and dynamic Asian road scenes and evaluate the accuracy, power, and speed of MediaTek Dimensity 9000 platform. We use a model called Deep Dual-resolution Networks (DDRNets) in PyTorch, and deploy TensorFlow Lite format in MediaTek chip to assess our model. We choose the two-stage training strategy and utilize the decreasing training resolution technique to further improve results. Our team is the first-place winner in Low-power Deep Learning Semantic Segmentation Model Compression Competition for Traffic Scene in Asian Countries at IEEE International Conference on Multimedia & Expo (ICME) 2022.
Parallel Generative Adversarial Network for Third-person to First-person Image Generation

Cross-view image generation has been recently proposed to generate images of one view from another dramatically different view. In this paper, we investigate third-person (exocentric) view to first-person (egocentric) view image generation. This is a challenging task since egocentric view sometimes is remarkably different from exocentric view. Thus, transforming the appearances across the two views is a non-trivial task. To this end, we propose a novel Parallel Generative Adversarial Network (P-GAN) with a novel cross-cycle loss to learn the shared information for generating egocentric images from exocentric view. We also incorporate a novel contextual feature loss in the learning procedure to capture the contextual information in images. Extensive experiments on the Exo-Ego datasets [5] show that our model outperforms the state-of-the-art approaches.
Teacher Intelligence Training Based on Big Data and Artificial Intelligence

The purpose of this paper is to improve the theoretical system of teacher's intelligent training, build a teacher's intelligent training platform, build an intelligent training course resource and establish a teacher's intelligent training mechanism. This article mainly uses the methods of investigation and interviews to analyze the status quo of teachers' professional development, existing problems, and research needs. The results of the questionnaire survey showed that 46.7% of participants were in favor of smart training, but there were also 70 people who believed that the biggest difficulty in smart training was the lack of a relatively fixed and effective platform. Therefore, it is necessary to design an intelligent training platform to provide a good platform for teachers to learn.
Construction of Piano Performance Curriculum System Based on Convolutional Neural Network

Comprehensively promoting quality education and the all-round development of human beings is the focus of current educational work. When carrying out quality education at university, it is important to start from all aspects such as ideology and morality, physical and mental health, professional learning and personality cultivation, and to give full play to their potential and enhance their creativity. Music teaching is an important element of quality education and using it as an entry point can prevent it from being too abstract. However, music education is still a weak aspect of higher education in China.
Defense against adversarial attack by feature distillation and metric learning

In recent years, deep neural networks have achieved high accuracy in many classification tasks, including speech recognition, object detection, and image classification. Although the deep neural network is robust to random noise, when some special disturbances that cannot be detected by the human eye are added to the neural network input, these special disturbances will still cause the deep neural network model to output wrong predictions. For the defense method against adversarial samples, we propose an adversarial training method based on the combination of feature distillation and metric learning. This method is to pretrain a fixed teacher network training method and use clean sample training. The student network uses adversarial samples for adversarial training. During the training process, the clean samples are used in the middle layer features of the teacher network to guide the adversarial samples in the middle layer features of the student network, and the middle layer features of the adversarial samples are repaired in the student network to achieve good results. At the same time, considering the relationship between adversarial samples and clean samples in the student network, a metric learning loss is introduced in<sup>1</sup> the middle layer features of the student network, so that the distance between the adversarial samples and the clean samples is closer than that between the adversarial samples and the confused samples. This makes the deep neural network model more robust. Finally, we perform gray-box, white-box and black-box attacks to verify the effectiveness of our method. Our algorithm significantly outperforms state-of-the-art adversarial training algorithms.
Optimal Superposition Enhancement of a Mixed State by Stochastic Diagonal Superposition-Free Operations

The resource theory of superposition is a generalization of coherence theory. A rigorous framework for quantifying superposition of linear independent states has been given by previous studies. In this paper, the optimal superposition enhancement of a mixed state is studied. Based on the $l_{1}$ measure of superposition the optimal superposition enhancement value of a mixed state can be obtained under the stochastic diagonal superposition-free operations(or incomplete diagonal superposition-free operations). In addition, the optimal probability corresponding to the optimal superposition enhancement value of a mixed state is also discussed. The results show that although the superposition-free operations can decrease the superposition of a state, the superposition enhancement of a state is possible, a single mixed state can be probabilistically transformed into another state with higher superposition than before by superposition-free operations. These studies will give us a better understanding of the superposition manipulation and the resource theory of superposition.
Simulation-Driven Design of Smart Gloves for Gesture Recognition

Smart gloves for tracking hand movements and recognizing hand gestures have gained immense popularity due to the growing demand in entertainment, manufacturing, and rehabilitation. However, designing smart gloves has been a complex and costly process due to trial and error. In this paper, we propose an open simulation platform for creating smart glove designs, including optimal sensor placement and deep learning models for gesture recognition, with minimal costs and manual efforts. Our platform implements a pipeline that starts with 3D hand pose extraction from video data and extends to the refinement and conversion of the poses into hand joint angles based on inverse kinematics, the sensor placement optimization based on hand joint analysis, and the training of deep learning models using simulated sensor data. In comparison to the existing platforms that always require precise motion data as input, our platform takes monocular videos, which can be captured with widely available smartphones or web cameras, as input and integrates novel approaches to minimize the impact of the errors induced by imprecise motion extraction from videos. Moreover, our platform enables more efficient sensor placement selection by removing redundant or irrelevant sensor locations based on the analysis of hand joint independence and correlation before training and comparing the performance of classification models among all possible sensor layouts. We demonstrate how the pipeline works and how it delivers a sensible design for smart gloves in a real-life case study. We also evaluate the performance of each building block and its impact on the reliability of the generated design.
Real-time semantic segmentation of microvascular decompression images based on encoder-decoder structure

The predecessor of China Optics was China Optics and Applied Optics Digest, which was founded in 1985. At that time, it was the only retrieval journal in the field of optics in China. At the end of 2008, China Optics and Applied Optics Digest was renamed
Video Forensics for Object Removal Based on Darknet3D

To address the problems of insufficient analysis of time-domain information of tampered video by 2D convolutional neural networks and the loss of details in the pooling layer when processing frame images, a 3D video object removal tamper detection and localization model based on Darknet53 optimization network is proposed. While the Darknet53 network can fully retain the detail information in the frame, we try to give the two-dimensional Darknet53 network, which can only process spatial information, the ability to process time-domain information, and extend the two-dimensional convolutional layer into a three-dimensional convolutional layer, and also improve the detection efficiency by adjusting the network structure to reduce feature redundancy, making it more suitable for efficient processing of video tampering detection binary classification tasks. A D3D (Darknet3D) network is constructed to improve feature adequacy representation. Experimental results reveal that the temporal domain classification accuracy of the tamper detection model based on the Darknet3D is 98.9%, and the average Intersection over Union of spatial localization and tamper area labeling is 49.7%, which can effectively detect and locate the object removal tampering.
<i>DeepContext:</i> Mobile Context Modeling and Prediction Via HMMs and Deep Learning

Mobile context determination is an important step for many context-aware services such as location-based services, enterprise policy enforcement, building/room occupancy detection for power/HVAC operation, etc. Especially in enterprise scenarios where policies (e.g., attending a confidential meeting only when the user is in &#x201C;Location X&#x201D;) are defined based on mobile context, it is paramount to verify the accuracy of the mobile context. Most of the existing solutions rely on context obtained directly from sensors which could be hacked, noisy or insufficient, which cannot be relied upon for security applications. In this article, we take a different approach by modeling mobile context based on past context data of related users and considering its unique challenges such as missing features. To this end, we propose three models for modeling mobile context based on symbolic time series data of feature-value pairs&#x2014;two stochastic models based on the theory of Hidden Markov Models&#x00A0;(HMMs) and one model based on deep learning&#x2014;<i>personalized model</i> (<i>HPContext</i>), <i>collaborative filtering model</i> (<i>(HCFContext)</i>), and <i>deep learning model</i> (<i>DeepContext</i>) to eventually replace <i>HPContext</i>. <i>HPContext</i> and <i>DeepContext</i> predict the current context using sequential history of the user&#x0027;s own past context observations; while <i>HCFContext</i> enhances the former with collaborative filtering features, which enables it to predict the current context of the primary user based on the context observations of users related to the primary user, e.g., same team colleagues in the company, gym friends, family members, etc. <i>DeepContext</i> models mobile context based on symbolic (i.e., categorical valued rather than continuous-valued) time series data using deep learning techniques. These models are then used to determine the context of the primary user at the current instant or some timesteps in to the future. Each of the proposed models can also be used to enhance or complement the context obtained from sensors. Finally, these models are thoroughly validated on a real-life dataset.
Brightformer: A Transformer to Brighten the Image

Low-light image enhancement algorithms need to recover the overall information of the image, including local details and global information. However, existing image enhancement methods mainly focus on local details or global information. Therefore, balancing these two aspects at the same time is challenging. This paper proposes a local dual-branch network (BrightFormer) for image enhancement that combines convolutions and transformers to solve this problem.The salient features of this paper are: (1) Use convolution to refine high-frequency information so that local features are preserved and propagated throughout the network. (2) Combining the gating parameters with the prior information on illumination (ill-map) in self-attention not only improves the flexibility of feature expression but also allows BrightFormer to extract global features easily. (3) The obtained local details and global features are fused by spatial and channel attention in the Feature equalization fusion unit (FEFU). (4) The fused features utilize a Deep feedforward network (DFN) to encode the location information between adjacent pixels. And use the GELU activation function to retain useful features and eliminate useless features with an attention-like mechanism.Experimental results show that BrightFormer achieves competitive performance on quantitative metrics and visual perception on the LOW-Light dataset.