Reviewerly/exHarmony huggingface dataset:
Fast fourier transform based new pooling layer for deep learning

Abstract Convolution is considered most significant layer in deep learning because it can extract best features of data through the network but it may result in huge volume of data. This problem can be solved by using pooling. In this paper, A novel pooling method is proposed by using discrete Fourier transform (DFT), this method is used DFT technique to transform the data from spatial domain into frequency domain to preserve the most important information from the details coefficients, where the details information of the image is less significant, therefore it can be discarded to down sample the size of dimensions. Its effect will be great with advantage of reducing the eliminated details information as compared with other standard methods. After applying DFT, the most significant coefficients, which represent most important features are cropped while less important details will be discarded then the data are reconstructed by applying inverse DF, therefore the high quality of features are extracted, which solve the problem of losing significant information during the pooling layer. Different methods are proposed based on the scenario of using DFT. The proposed methods are tested by extracting pooled image then the original images were retrieved using only the pooled images. Then the retrieved images are compared with original images by using different measures such as SNR, correlation and SSIM. Then the proposed layers used for image classification for two different datasets. The results proved that the proposed methods outperformed standard methods, thus it can be used for deep learning application.
Learning Through SMS in Saudi EFL Classroom: An Inter-University Study of Learner Perceptions and Achievement of Autonomy

Technology as a learning tool or otherwise holds great attraction for learners today. The current study explored the impact of Short Messages as Learning Tool (SMLT) on EFL Saudi learners learning confused English words. It also gauges learners' satisfactions towards using such tools on their autonomy and language proficiency. The study pursued a quasi-experiment research design. It recruited 80 EFL learners across Najran University and Qassim University, KSA. To ensure parity of existing language proficiency and learning success, the Oxford Placement Test is administered once before and once at the end of the intervention to all of the 80 participants to obtain comparative values.. Furthermore, a semi-structured interview is also used with three randomly selected participants from each of the experimental groups to obtain data on individual perceptions of the EFL learners to the use of MALL in the EFL classroom. Content analysis is used to identify dominant themes in these. Findings revealed that learners acquisition for confusing words were developed to a great extent in both universities. Moreover, the study found that there is no significant difference in the students' achievement attributed to the learning sittings, Z = .935. Finally the students expressed satisfaction in terms of their autonomy ratings and complementary points of view on the use of SMS, based on the semi-structured interviews. The current research is useful as its findings can apply to mobile teaching and text messaging in the English classroom for EFL curriculum developers and English language teachers.
CoCoSoDa: Effective Contrastive Learning for Code Search

Code search aims to retrieve semantically relevant code snippets for a given natural language query. Recently, many approaches employing contrastive learning have shown promising results on code representation learning and greatly improved the performance of code search. However, there is still a lot of room for improvement in using contrastive learning for code search. In this paper, we propose CoCoSoDa to effectively utilize contrastive learning for code search via two key factors in contrastive learning: data augmentation and negative samples. Specifically, soft data augmentation is to dynamically masking or replacing some tokens with their types for input sequences to generate positive samples. Momentum mechanism is used to generate large and consistent representations of negative samples in a mini-batch through maintaining a queue and a momentum encoder. In addition, multimodal contrastive learning is used to pull together representations of code-query pairs and push apart the unpaired code snippets and queries. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages. Experimental results show that: (1) CoCoSoDa outperforms 14 baselines and especially exceeds CodeBERT, GraphCodeBERT, and UniXcoder by 13.3%, 10.5%, and 5.9% on average MRR scores, respectively. (2) The ablation studies show the effectiveness of each component of our approach. (3) We adapt our techniques to several different pre-trained models such as RoBERTa, CodeBERT, and GraphCodeBERT and observe a significant boost in their performance in code search. (4) Our model performs robustly under different hyper-parameters. Furthermore, we perform qualitative and quantitative analyses to explore reasons behind the good performance of our model.
Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets

In the current era of the internet, where social media platforms are easily accessible for everyone, people often have to deal with threats, identity attacks, hate, and bullying due to their association with a cast, creed, gender, religion, or even acceptance or rejection of a notion. Existing works in hate speech detection primarily focus on individual comment classification as a sequence labeling task and often fail to consider the context of the conversation. The context of a conversation often plays a substantial role when determining the author's intent and sentiment behind the tweet. This paper describes the system proposed by team MIDAS-IIITD for HASOC 2021 subtask 2, one of the first shared tasks focusing on detecting hate speech from Hindi-English code-mixed conversations on Twitter. We approach this problem using neural networks, leveraging the transformer's cross-lingual embeddings and further finetuning them for low-resource hate-speech classification in transliterated Hindi text. Our best performing system, a hard voting ensemble of Indic-BERT, XLM-RoBERTa, and Multilingual BERT, achieved a macro F1 score of 0.7253, placing us first on the overall leaderboard standings.
Exploring the Temporal Cues to Enhance Video Retrieval on Standardized CDVA

As the demand for large-scale video analysis increases, video retrieval research is also becoming more active. In 2014, ISO/IEC MPEG began standardizing compact descriptors for video analysis, known as CDVA, and it is now adopted as a standard. However, the standardized CDVA is not easily compared to other methods because the MPEG-CDVA dataset used for performance verification is not disclosed, despite the fact that follow-up studies are underway with multiple versions of the CDVA experimental model. In addition, analyses of modules constituting the CDVA framework are insufficient in previous studies. Therefore, we conduct self-evaluations of CDVA to analyze the impact of each module on the retrieval task. Furthermore, to overcome the obstacles identified through these self-evaluations, we suggest temporal nested invariance pooling, abbreviated as TNIP, which implies temporal robustness realized by improving nested invariance pooling, abbreviated as NIP, one of the features in CDVA. Finally, benchmarks of the existing CDVA and the proposed approach are provided on several public datasets. Through this, we show that the CDVA framework is capable of boosting the retrieval performance if utilizing the proposed approach.
A novel approach to capture the similarity in summarized text using embedded model

Abstract The presence of near duplicate textual content imposes great challenges while extracting information from it. To handle these challenges, detection of near duplicates is a prime research concern. Existing research mostly uses text clustering, classification and retrieval algorithms for detection of near duplicates. Text summarization, an important tool of text mining, is not explored yet for the detection of near duplicates. Instead of using the whole document, the proposed method uses its summary as it saves both time and storage. Experimental results show that traditional similarity algorithms were able to capture similarity relatedness to a great extent even on the summarized text with a similarity score of 44.685%. Moreover, degree of similarity capture was greater (0.52%) in case of use of embedding models with better text representation as compared to traditional methods. Also, this paper highlights the research status of various similarity measures in terms of concept involved, merits and demerits.
Hamming Neural Network in Discrete Form

Hamming artificial neural network is used to solve problems of classification of binary input vectors. Its work is based on procedures aimed at choosing, as a solution to the classification problem, one of the reference images closest to the noisy input image supplied to the network input and assigning this image to the corresponding class. At the same time, difficulties were identified in implementing calculations of the Hamming neural network paradigm. One of the ways to simplify computations in the considered neural network paradigm is to transition to integer computations since integer multiplication is computed several times faster than real data type. To simplify the computation model, a Hamming neural network in discrete form was proposed in this paper. A discrete model of the Hamming neural network is proposed, making it possible to simplify the implementation of computations significantly. These results can be applied to solve various problems, including computer vision applications.
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning

Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and chaining lower-level skills. Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions. Hierarchies can further improve on this by abstracting the space states as well. We posit that a suitable state abstraction should depend on the capabilities of the available lower-level policies. We propose Value Function Spaces: a simple approach that produces such a representation by using the value functions corresponding to each lower-level skill. These value functions capture the affordances of the scene, thus forming a representation that compactly abstracts task relevant information and robustly ignores distractors. Empirical evaluations for maze-solving and robotic manipulation tasks demonstrate that our approach improves long-horizon performance and enables better zero-shot generalization than alternative model-free and model-based methods.
Solving ImageNet: a Unified Scheme for Training any Backbone to Top Results

ImageNet serves as the primary dataset for evaluating the quality of computer-vision models. The common practice today is training each architecture with a tailor-made scheme, designed and tuned by an expert. In this paper, we present a unified scheme for training any backbone on ImageNet. The scheme, named USI (Unified Scheme for ImageNet), is based on knowledge distillation and modern tricks. It requires no adjustments or hyper-parameters tuning between different models, and is efficient in terms of training times. We test USI on a wide variety of architectures, including CNNs, Transformers, Mobile-oriented and MLP-only. On all models tested, USI outperforms previous state-of-the-art results. Hence, we are able to transform training on ImageNet from an expert-oriented task to an automatic seamless routine. Since USI accepts any backbone and trains it to top results, it also enables to perform methodical comparisons, and identify the most efficient backbones along the speed-accuracy Pareto curve. Implementation is available at:https://github.com/Alibaba-MIIL/Solving_ImageNet
Towards Efficient and Scalable Sharpness-Aware Minimization

Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training large-scale models such as vision transformers. However, the update rule of SAM requires two sequential (non-parallelizable) gradient computations at each step, which can double the computational overhead. In this paper, we propose a novel algorithm LookSAM - that only periodically calculates the inner gradient ascent, to significantly reduce the additional training cost of SAM. The empirical results illustrate that LookSAM achieves similar accuracy gains to SAM while being tremendously faster - it enjoys comparable computational complexity with first-order optimizers such as SGD or Adam. To further evaluate the performance and scalability of LookSAM, we incorporate a layer-wise modification and perform experiments in the large-batch training scenario, which is more prone to converge to sharp local minima. We are the first to successfully scale up the batch size when training Vision Transformers (ViTs). With a 64k batch size, we are able to train ViTs from scratch in minutes while maintaining competitive performance.
Weakly Supervised Bolt Detection Model Based on Attention Mechanism

At present, a high-precision annotated data set is still the key factor for object detection model to achieve good performance. However, in many practical scenarios such as power production, the high cost of image acquisition and the difficulty of expert annotation make the acquisition of high quality image annotation very time-consuming and laborious. Therefore, a weakly supervised bolt detection model based on attention mechanism is proposed in this paper. Firstly, the method in this paper uses the ability of the attention mechanism to filter information to strengthen the object positioning properties of the convolutional neural network itself, and on this basis, a model capable of rough bolt detection is trained under the weakly supervised bolt data set. Secondly, in view of the problem that the attention mechanism is easy to fall into the local optimal solution during training, the attention saliency maps are expanded by using a method similar to data augment, so as to realize the purpose of decentralizing the focus area of the attention mechanism. The experimental results show that the proposed model can achieve an average accuracy of 76.5% on the weakly supervised bolt data set. Compared with the traditional strongly supervised object detection algorithm, the proposed model can achieve the purpose of training high-quality bolt detection model by only using the low-quality training data set by sacrificing certain accuracy.
Website-Based Student Achievement Book Using the Waterfall Method

Al Qur'an Education Park (TPA) Al Badar has approximately thirty students, but that number will continue to grow. The recording of achievement cards is done manually, and everything is done by the Al-Badar TPA teacher. Usually, teachers record student learning outcomes using achievement cards, and achievement cards are filled out after students learn to read Iqro or the Qur'an and handed over to their students. However, amid the current COVID-19 pandemic, face-to-face meetings are very difficult, especially between teachers and participants who are undergoing online teaching. Based on the problems that occurred at the Al Badar TPA regarding the recording of student achievement and the importance of technology for education, it is necessary to have an information system at the Al Badar TPA. The method used in this research is the waterfall method approach in the manufacture of information systems with several stages, including problem identification, system requirements analysis, system design, system creation, and system testing, and also system analysis. In making the system, PHP and MySQL are used with the CodeIgniter framework. In system testing, the author uses two methods of black-box testing and white-box system testing. While the results of the analysis of the system in terms of testing the quality ofservice users, namely by using usability testing. The results of this study are a website-based student achievement book as an online learning achievement record book that can function well, both systemically and from the user's side of the application and greatly assists the needs of the Al-Badar TPA during the pandemic. The results obtained from the usability testing analysis are 90%, so it can conclude that the system is feasible and easy to use. So, a website-based student achievement book as book’s achievement record online can function properly and greatly help the needs of the TPA Al-Badar during the pandemic.
CrowdIoT: The Crowd-Sourcing Test System for IoT Devices Based on Blockchain

In view of some problems existing in traditional software crowdsourcing tests and Internet of Things device tests, we developed a CrowdIoT crowdsourcing test system for the Internet of Things devices based on a block chain.Crow-dIoT uses the core technology of blockchain to focus its testing on physical devices on the Internet of Things.CrowdIoT uses two different testing methods for different testing needs: online testing and offline testing.Online remote testing is our key function and research object.By opening the testing interface of Internet of Things devices to testers, testers can test the devices in the CrowdIoT system interface without having to get the hardware.At the same time, CrowdIoT introduced multi-threaded parallel testing technology to solve the conflict problem of multiple testers competing for hardware resources in online testing.Offline testing, as a supplement to online testing, is to send Internet of Things devices to testers with high credibility in deposit guarantee, so that testers can fully test the hardware devices and dig out as many vulnerabilities of the devices as possible.CrowdIoT has its own complete and scientific credibility system, combined with relevant incentive mechanism, consensus mechanism and connection mechanism, which not only effectively solves the centralization problem existing in traditional crowdsourcing testing platforms, but also solves the pain point of the lack of remote testing of Internet of Things devices in the market.Then it solves the problems of equipment limitation and testing cost limits in the field of Internet of Things equipment testing, and provides a platform for security testing and use for the Internet of Things participants.
Comparative Analysis of Machine Learning-Based Algorithms for Detection of Anomalies in IIoT

With the enormous increase in data, anomaly detection plays a prominent role in the finer analysis process. IIoT represents the Industrial Internet of Things that at first chiefly alluded to a mechanical system whereby an enormous number of devices or machines are associated and synchronized using programming devices and third stage advancements in a machine-to-machine and Internet of Things, later an Industry 4.0. The data produced by multiple huge numbers of sensors are incredibly complicated, diverse, and massive in IIoT and is raw. These may contain anomalies which are needed to be identified for better data analysis. In this research, we compare the Machine Learning algorithms of classification for detecting anomalies. The algorithms being compared here are Random Forest (RF), Logistic Regression (LR), Light Gradient Boosting Machine (LightGBM), Decision Trees (DT), K Nearest Neighbors (KNN). Three IIOT benchmark datasets were taken into consideration for analysis. The results have shown that RF has outperformed other algorithms used for the detection of anomalies in IIoT data.
The Response Shift Paradigm to Quantify Human Trust in AI Recommendations

Explainability, interpretability and how much they affect human trust in AI systems are ultimately problems of human cognition as much as machine learning, yet the effectiveness of AI recommendations and the trust afforded by end-users are typically not evaluated quantitatively. We developed and validated a general purpose Human-AI interaction paradigm which quantifies the impact of AI recommendations on human decisions. In our paradigm we confronted human users with quantitative prediction tasks: asking them for a first response, before confronting them with an AI's recommendations (and explanation), and then asking the human user to provide an updated final response. The difference between final and first responses constitutes the shift or sway in the human decision which we use as metric of the AI's recommendation impact on the human, representing the trust they place on the AI. We evaluated this paradigm on hundreds of users through Amazon Mechanical Turk using a multi-branched experiment confronting users with good/poor AI systems that had good, poor or no explainability. Our proof-of-principle paradigm allows one to quantitatively compare the rapidly growing set of XAI/IAI approaches in terms of their effect on the end-user and opens up the possibility of (machine) learning trust.
Predicting sea surface temperature based on a parallel autoreservoir computing approach with short-term measured data

Sea surface temperature (SST) is an essential parameter for observing the marine environment. It directly reflects the state of heat storage and release in the ocean. The change in SST will cause many phenomena that profoundly affect human production and life. Therefore, predicting SST accurately and efficiently can help us avoid many risks. In this article, we present a method based on neural networks. First, we propose a finite-dimensional description of SST, which investigates SST in the finite-dimensional phase space. Based on phase space reconstruction technology and the autoreservoir neural network (ARNN), the Spatial Parallel ARNN (SPARNN) is proposed for predicting SST. Unlike the previous machine learning methods that require big data, our approach only needs a small amount of locally short-term data to catch the dynamic features of the SST field. It also has excellent parallelism and is easy to run on a large-scale computer platform.
Lecture Notes: Programming Quantum Computers

Quantum computing is a new emerging computer technology. Current quantum computing devices are at a development stage where they are gradually becoming suitable for small real-world applications. This lecture is devoted to the practical aspects of programming such quantum computing devices. The first part of these lecture notes focuses on programming gate-based quantum computers, and the second part shows how to program quantum annealers.