Reviewerly/exHarmony huggingface dataset:
MARS: Multimacro Architecture SRAM CIM-Based Accelerator With Co-Designed Compressed Neural Networks

Convolutional neural networks (CNNs) play a key role in deep learning applications. However, the large storage overheads and the substantial computation cost of CNNs are problematic in hardware accelerators. Computing-in-memory (CIM) architecture has demonstrated great potential to effectively compute large-scale matrix-vector multiplication. However, the intensive multiply and accumulation (MAC) operations executed at the crossbar array and the limited capacity of CIM macros remain bottlenecks for further improvement of energy efficiency and throughput. To reduce computation costs, network pruning and quantization are two widely studied compression methods to shrink the model size. However, most of the model compression algorithms can only be implemented in digital-based CNN accelerators. For implementation in a static random access memory (SRAM) CIM-based accelerator, the model compression algorithm must consider the hardware limitations of CIM macros, such as the number of word lines and bit lines that can be turned on at the same time, as well as how to map the weight to the SRAM CIM macro. In this study, a software and hardware co-design approach is proposed to design an SRAM CIM-based CNN accelerator and an SRAM CIM-aware model compression algorithm. To lessen the high-precision MAC required by batch normalization (BN), a quantization algorithm that can fuse BN into the weights is proposed. Furthermore, to reduce the number of network parameters, a sparsity algorithm that considers a CIM architecture is proposed. Last, MARS, a CIM-based CNN accelerator that can utilize multiple SRAM CIM macros as processing units and support a sparsity neural network, is proposed.
Detecting and Measuring the Exposure of Children and Adolescents to Inappropriate Comments in YouTube

Social media platforms have been growing at a rapid pace, attracting users engagement with contents due to their convenience facilitated by many usable features. Such platforms provide users with interactive options such as likes, dislikes as well as a way of expressing their opinions in the form of text (i.e., comments). The ability of posting comments on these online platforms has allowed some users to post racist, obscene, as well as to spread hate on these platforms. In some cases, this kind of toxic behavior might turn the comment section from a space where users can share their views to a place where hate and profanity are spread. Such issues are observed across various social media platforms and many users are often exposed to these kinds of behaviors which requires comment moderators to spend a lot of time filtering out such inappropriate comments. Moreover, such textual "inappropriate contents" can be targeted towards users irrespective of age, concerning variety of topics (not only controversial), and triggered by various events. My doctoral dissertation work, therefore, is primarily focused on studying, detecting and analyzing users exposure to this kind of toxicity on different social media platforms utilizing the state-of-art techniques in deep learning and natural language processing. This paper presents one example of my works on detecting and measuring kids exposure to inappropriate comments posted on YouTube videos targeting young users. In the meantime, the same pipeline is being examined for measuring users interaction with mainstream news media and sentiment towards various topics in the public discourse in light of the Coronavirus disease 2019 (COVID'19).
A Comparison of Quantum and Traditional Fourier Transform Computations

The quantum Fourier transform (QFT) can calculate the Fourier transform of a vector of size $N$N with time complexity $\mathcal {O}(\log ^2~N)$O(log2N) as compared to the classical complexity of $\mathcal {O}(N \;\log N)$O(NlogN). However, if one wanted to measure the full output state, then the QFT complexity becomes $\mathcal {O}(N \;\log ^2~N)$O(Nlog2N), thus losing its apparent advantage, indicating that the advantage is fully exploited for algorithms when only a limited number of samples is required from the output vector, as is the case in many quantum algorithms. Moreover, the computational complexity worsens if one considers the complexity of constructing the initial state. In this article, this issue is better illustrated by providing a concrete implementation of these algorithms and discussing their complexities as well as the complexity of the simulation of the QFT in matlab.
Prevalence, Contents and Automatic Detection of KL-SATD

When developers use different keywords such as TODO and FIXME in source code comments to describe self-admitted technical debt (SATD), we refer it as Keyword-Labeled SATD (KL-SATD). We study KL-SATD from 33 software repositories with 13,588 KL-SATD comments. We find that the median percentage of KL-SATD comments among all comments is only 1,52%. We find that KL-SATD comment contents include words expressing code changes and uncertainty, such as remove, fix, maybe and probably. This makes them different compared to other comments. KL-SATD comment contents are similar to manually labeled SATD comments of prior work. Our machine learning classifier using logistic Lasso regression has good performance in detecting KL-SATD comments (AUC-ROC 0.88). Finally, we demonstrate that using machine learning we can identify comments that are currently missing but which should have a SATD keyword in them. Automating SATD identification of comments that lack SATD keywords can save time and effort by replacing manual identification of comments. Using KL-SATD offers a potential to bootstrap a complete SATD detector.
Image Captioning with Internal and External Knowledge

Automatically generating a human-like description for a given image is a potential research in artificial intelligence, which has attracted a great of attention recently. Most of the existing attention methods explore the mapping relationships between words in sentence and regions in image, such unpredictable matching manner sometimes causes inharmonious alignments that may reduce the quality of generated captions. In this paper, we make our efforts to reason about more accurate and meaningful captions. We first propose word attention to improve the correctness of visual attention when generating sequential descriptions word-by-word. The special word attention emphasizes on word importance when focusing on different regions of the input image, and makes full use of the internal annotation knowledge to assist the calculation of visual attention. Then, in order to reveal those incomprehensible intentions that cannot be expressed straightforwardly by machines, we inject external knowledge extracted from knowledge graph into the encoder-decoder framework to facilitate meaningful captioning. We validate our model on two freely available captioning benchmarks: Microsoft COCO dataset and Flickr30k dataset. The results demonstrate that our approach achieves state-of-the-art performance and outperforms many of the existing approaches.
Neural Relation Extraction on Wikipedia Tables for Augmenting Knowledge Graphs

Knowledge Graph Augmentation is the task of adding missing facts to an incomplete knowledge graph to improve its effectiveness in applications such as web search and question answering. State-of-the-art methods rely on information extraction from running text, leaving rich sources of facts such as tables behind. We help close this gap with a neural method that uses contextual information surrounding a table in a Wikipedia article to extract relations between entities appearing in the same row of a table or between the entity of said article and entities appearing in the table. We trained and tested our method on a much larger dataset compared to previous work which we have made public and observed experimentally that our method is very promising for the task.
Tracking Knowledge Structures and Proficiencies of Students With Learning Transfer

In online intelligent education systems, to offer proactive studying services to students (e.g., learning path recommendation), a crucial demand is to track students’ knowledge mastery levels over time. However, existing works ignore the impact of learning transfer on knowledge tracing and only track knowledge proficiency. Knowledge proficiency alone cannot fully reflect students’ knowledge mastery levels. A student’s knowledge structure (the similarities and differences within knowledge concepts) and abstract principle mastery level (common attributes among knowledge concepts, such as learning methods) also need to be tracked. To this end, we propose a novel multilevel Knowledge Tracing model with Learning Transfer (KTLT) to track students’ knowledge mastery levels. First, we clarify the relationships among abstract principles, knowledge structure, and knowledge proficiency by utilizing the learning transfer theory in educational psychology. Then, we associate each problem with a knowledge vector in which each element represents an explicit knowledge concept by leveraging educational priors (i.e., a Q-matrix). Correspondingly, each student is represented as a knowledge vector (knowledge proficiency) and an abstract principle vector (abstract principle mastery). Given a student’s knowledge and abstract principle vector over time, we use the learning and forgetting curve as priors to capture the student’s knowledge proficiency and abstract principle mastery level over time. Furthermore, we embed the knowledge concept by using a student’s abstract principle mastery level and obtain a personalized knowledge relevance matrix (a student’s knowledge structure) by calculating the cosine similarity among the knowledge embedding results. Finally, we design a probabilistic matrix factorization framework by combining student and problem priors for tracking a student’s knowledge mastery. Extensive experiments on two real-world datasets demonstrate both the effectiveness and explanatory ability of the KTLT.
An Improved RRT Robot Autonomous Exploration and SLAM Construction Method

When the indoor environment is unknown, how to make robots carry out effective autonomous exploration and construct related maps is one of the key issues in the field of mobile robots. In view of the fact that the real-world environment is usually partially observable and uncertain, an improved method of autonomous exploration and SLAM construction based on Rapid-exploration Random Tree (RRT) is proposed. In the local RRT exploration part, the autonomous exploration problem is regarded as a partially observable Markov decision process (POMDP) problem. Boundary points are extracted in the boundary area of known space and unknown space, and the robot is directed to the unexplored area. At the same time, during the exploration process, the global RRT tree with adaptive step values is established to explore the boundary points at the far end of the robot. The two methods are merged to speed up the search for boundary points. After the best advantage is obtained, the robot is continuously moved to the best advantage through closed-loop control. Based on the original construction of a 2D grid map, a “parallel construction” idea was proposed to construct a 3D octree map at the same time. The effectiveness of the proposed method is verified by simulation experiments and actual scenarios in a robot operating system (ROS).
What's going on in the chat flow? A move analysis of e-commerce customer service webchat exchange

The business world is fast entering a virtual environment where technology is applied in all areas to improve efficiency. For example, web-based synchronous chat has become a popular way of communication for customer support. However, we note that business managers and customers bemoan the amount of time taken to serve customers via webchat, caused in some part perhaps by the textual route to get to a resolution using this new media. This has received little attention from the research world. We therefore collected 15 customer service webchat exchanges from an international online retail company and explored the challenges by looking at the generic structural moves, their impact on the time to resolution and the linguistic affordances of webchat. It was found that webchat moves highly depend on the type of business, company policies and division of labor in contact centre. Some webchat affordances such as allowing for URLs to lengthy webpages and long lists of instructions can lead to communication breakdowns. The results of this analysis can be a valuable resource for the pre-employment training and coaching of online customer service representatives as well as for business communication courses in higher education where little attention is given to such genre.
Artificial Neural Network based Indian Sign Language Recognition using hand crafted features

Sign language is a medium of communication of Divyangjans (Deaf and mute people). Sign language can be used for effective communication only if understood by both the people trying to communicate. When one person is unaware of meaning of the sign gestures an interpreter is required who can translate the gestures into spoken language. This article presents a methodology to recognize Indian Sign Language (SL) gestures and translate them into English. SL Recognition systems can be useful for facilitating the conversation. There are various systems developed by researchers for implementing a SL recognition system. Being in its developing stage, the grammar rules of Indian Sign Language (ISL) are not documented making the recognition process a challenge. This approach employs hand crafted feature extraction technique and uses Artificial Neural Network for classification of the gestures. The accuracy of model achieved is as high as 98% using this methodology.
Image enhancement in a quaternion framework for remote sensing applications

Image enhancement refers to processing images to make them more suitable for display or further image analysis. So, image enhancement is a problem-oriented procedure for improving the visual appearance of the remote sensing image. In this paper, we adopt the quaternion framework for representing color satellite images since it offers scope to process color images holistically, rather than as separate color space components, and thereby handles the coupling between the color channels. We present a new image enhancement algorithm based on multi-scale block-rooting processing. The basic idea is to apply the frequency domain image enhancement approach for different size image blocks. Also, we used a nonreference quality measure to choose the presented algorithm parameters optimally. Computer simulations on the remote sensing image dataset show that the new enhancement algorithm exhibits better results compared to state-of-the-art techniques.
Evaluation of Proof-of-Work Consensus Algorithm for Blockchain Networks

Blockchain is an immutable database which is decentralized and distributed. This blockchain is shared by all the nodes in the network and every node has the same state. The transactions in this open distributed ledger are verified and confirmed with the help of consensus algorithms. The transactions, once executed and entered, can never be changed or erased. Bitcoin, the first usecase of blockchain, as a digital money, is based on Proof-of-Work (PoW) consensus protocol. PoW has worked tremendously well and can be useful for many industries. The opportunities are high in this innovative technology, and evolution in this area is just a beginning. This paper has described blockchain technology, the underlying concepts of blockchain and the implementation of Proof-of-Work consensus algorithm along with its performance analysis.
Multimodal Sentiment Analysis of #MeToo Tweets using Focal Loss (Grand Challenge)

The #MeToo trend has led to people talking about personal experiences of harassment more openly. This work attempts to aggregate such experiences of sexual abuse to facilitate a better understanding of social media constructs and to bring about social change [1]. We propose an approach to multimodal sentiment analysis using deep neural networks combining visual analysis and natural language processing. Our goal is different than the standard sentiment analysis goal of predicting whether a sentence expresses positive or negative sentiment; instead we try to detect the stand of a person on the topic and deduce the emotions conveyed. We have made use of a Multimodal Bi-Transformer (MMBT) model [2] which combines both image and text features to produce an optimal prediction of a tweet's stand and sentiments on the #MeToo campaign.
A Semantic-Based Approach for Privacy-Preserving in Trajectory Publishing

The increasing abundance of data in the trajectories of personal movement opens up new opportunities for analyzing and mining human mobility. But the inherent personal information in the data raises the privacy concern. In this paper, the trajectories are not simply considered as a sequence of the coordinates in Euclidean space, they combine the semantics-aware information with the background knowledge of underlying map for the location points. A novel approach is then proposed to conceal the actually visited sensitive places. To this end, a cloaking region for each personalized sensitive place is built before a reasonable dummy trajectory in the cloaking region is constructed. The performance of our approach is evaluated based on a real-world trajectories dataset. And the results show that the proposed approach outperforms the existing privacy-preserving approaches.
A Multispectral Light Field Dataset and Framework for Light Field Deep Learning

Deep learning undoubtedly has had a huge impact on the computer vision community in recent years. In light field imaging, machine learning-based applications have significantly outperformed their conventional counterparts. Furthermore, multi- and hyperspectral light fields have shown promising results in light field-related applications such as disparity or shape estimation. Yet, a multispectral light field dataset, enabling data-driven approaches, is missing. Therefore, we propose a new synthetic multispectral light field dataset with depth and disparity ground truth. The dataset consists of a training, validation and test dataset, containing light fields of randomly generated scenes, as well as a challenge dataset rendered from hand-crafted scenes enabling detailed performance assessment. Additionally, we present a Python framework for light field deep learning. The goal of this framework is to ensure reproducibility of light field deep learning research and to provide a unified platform to accelerate the development of new architectures. The dataset is made available under dx.doi.org/10.21227/y90t-xk47. The framework is maintained at gitlab.com/iiit-public/lfcnn.
DAKOTA: Continuous Authentication with Behavioral Biometrics in a Mobile Banking Application

Behavioral biometrics focus on identifying patterns in human activities or behaviors. Tracking and identifying interaction behavior of a user with a computing device is relatively a novel approach for user authentication on computer systems. In this paper, we present a continuous authentication scheme (DAKOTA) proposed for a mobile banking application. The currently used banking application is modified to track the behavioral biometrics of the client. A data logger is added to collect touch screen data while a user interacts with the application, as well as sensor data from accelerometer, gyroscope and magnetometer to capture the hand movements of the user. 30 volunteers participated for initial data collection. A one-class SVM model is trained for authenticating each user. Classification results show that, we can achieve 79% correct user identification, 13% false acceptance rate and 11.5% equal error rate. We also implemented an end-to-end authentication scheme, where data are collected on the client while using the mobile banking application and transferred to a server where the features are extracted and the SVM model for the particular user is invoked. We also present the resource usage results of DAKOTA in comparison to the original banking application, in terms of energy, CPU and memory as well as delay and data size.
A Novel Framework for Decentralized C2C E-commerce using Smart Contract

Security and users privacy have become an important factor in today's customer-centric world. Most of the trade nowadays is carried on e-commerce as it's easy and hassle proof. In C2C (Customer to Customer) e-commerce (Olx, eBay, etc) which acts as an intermediate between sellers and buyers, when a user buys products from these e-commerce that deals with a used products like Olx, eBay, etc cost evaluation becomes difficult and we end up buying it at the hiked price as there is no authority to verify that price quoted by seller is justified or not. These e-commerce websites often sell user data and influence users to buy specific products, they also charge transition fees for every good sold. In past years a lot of fraud has been seen on these e-commerce platforms as they don't have any existing model to prevent fraudulent people from joining their network. Besides these websites also control and manage the product reviews given by the genuine buyer. Keeping this scenario in mind our research focuses on various issues related to existing C2C e-commerce and overcoming them by using blockchain technology along with cost evaluation model. The use of blockchain in this scenario can help to move this centric model to a distributed model helping the user to trade better without letting share their personal information to others and getting products at the right price. This research work is aimed to provide better cost evaluation of used goods, provide data privacy and a sense of trust to end-users by leveraging various aspects of blockchain technology.
In-Vehicle Occupancy Detection And Classification Using Machine Learning

Occupancy detection is a difficult problem. There are several mechanisms exists for occupancy detection in vehicles, particularly in Automobiles. Now, safety has become an important and necessary aspect of the automobile industry. Airbag became a basic and important safety measure in cars. Even though airbag is a vehicle safety device, it can kill children below 12 years due to its rapid action by the exerting lot of force. This project explains about detecting the number of passengers sitting in the car and then classifying each person whether he/she is a child or an adult by processing the image taken from the camera. So that the deployment of airbags can be avoided near children. Each time car speeds from 0 Kmph to 20 Kmph, occupancy of the car is determined and each one is classified again. We are using widely used technique Haar Cascades, for detection. First, we detect faces and then classify each occupant adult or child.