Reviewerly/exHarmony huggingface dataset:
EFFGAN: Ensembles of fine-tuned federated GANs

Decentralized machine learning tackles the problem of learning useful models when data is distributed among several clients. The most prevalent decentralized setting today is federated learning (FL), where a central server orchestrates the learning among clients. In this work, we contribute to the relatively understudied sub-field of generative modelling in the FL framework.We study the task of how to train generative adversarial networks (GANs) when training data is heterogeneously distributed (non-iid) over clients and cannot be shared. Our objective is to train a generator that is able to sample from the collective data distribution centrally, while the client data never leaves the clients and user privacy is respected. We show using standard benchmark image datasets that existing approaches fail in this setting, experiencing so-called client drift when the local number of epochs becomes to large and local parameters drift too far away in parameter space. To tackle this challenge, we propose a novel approach named EFFGAN: Ensembles of fine-tuned federated GANs. Being an ensemble of local expert generators, EFFGAN is able to learn the data distribution over all clients and mitigate client drift. It is able to train with a large number of local epochs, making it more communication efficient than previous works.
ASRC:A Knowledge Graph Relation Construction Model based on Active Learning and Semantic Recognition

Domain-specific knowledge graph is constructed based on domain knowledge. The knowledge graph consists of entities and relations, where relation construction is the core. Traditional domain-specific knowledge graph relation construction requires manual construction by experts familiar with the relevant domain, which is heavily labor cost consuming. To solve this problem, this paper proposes a knowledge graph relation construction model ASRC based on active learning and semantic recognition, which consists of relation existence mining module and relation category prediction module. ASRC is training to mine entity pairs with possible relation, and recommends the entity pairs to experts for relation labeling, and then predicts the specific category of relation. ASRC improves model quality through multiple iterations, and more relation annotation can be labeled with the same number of annotations compared to randomly selected entity pairs.
Blockchain cross-chain technology research

At present, as most blockchain projects are based on different application scenarios and design concepts and develop heterogeneous blockchain projects through different technical frameworks, resulting in a large number of blockchain projects forming isolated value systems. With the increasing demand for the integration of blockchain technology between scenarios, the need for cross-chain value circulation and application synergy is becoming more and more obvious. How to realize the interconnection between blockchain chains and establish an efficient, secure and universal cross-chain technology system. So as to meet the requirements of safe and efficient data sharing and business collaboration between chains, which has become the top priority of current blockchain research. This paper first outlines the principles of blockchain cross-chain technology, discusses the current mainstream cross-chain mechanisms, Then the three mainstream cross-chain models are compared and analyzed, and the problems of the current mainstream cross-chain technologies are summarized, Finally, a summary and outlook on cross-chain technology.
A Business Workflow Architecture for Predictive Maintenance using Real-Time Anomaly Prediction On Streaming IoT Data

The Internet of Things (IoT) usually consists of many fully synchronized devices. Since these devices sense data from outside, they are usually outside and sometimes in places that are difficult to access in case of failure. Sometimes, late-recognized errors cause the system not to work and cause significant damage to the system. For this reason, it is crucial to avoid faulty situations with minor damage or even prevent these situations before they occur. Although there are studies on the topic in the literature, there is an emerging need to examine this problem comprehensively and target predictive maintenance and self-healing systems utilizing IoT systems. In this study, we propose a business workflow architecture using streaming-based machine learning algorithms to provide predictive maintenance utilizing IoT systems. In the proposed predictive maintenance workflow, we utilized various machine learning algorithms such as Adaptive Random Forest, Hoeffding Tree, Leveraging Bagging, SPegasos, and Single Drift Classifier classification algorithms. To show the usability of the proposed business workflow, we provide a prototype implementation. We provide an experimental study on the prototype to investigate the prediction success. We also investigate how fast the system can learn from the streaming data. We conduct this evaluation with unbalanced and balanced data. In this manuscript, we report the results of our experimental study.
StreamFlow: A System for Summarizing and Learning Over Industrial Big Data Streams

The growing need for predictive analytics over streaming data in the industry requires a flexible and continuously scalable big data system. In real-time big data applications (cybersecurity, AIOps, anomaly detection, predictive maintenance, IoT etc.), efficient machine learning models must be trained and industrialized within existing data processing plat-forms and industrial tools. This requires interoperability between various components: data collection, processing, summarization, modelling and analytics. Existing works focus on building AI models for big data, neglecting real-world challenges when integrating such models into an existing industrial production framework. In this paper, we propose StreamFlow, an operational data pipeline to address industrial challenges for continuous learning over big data streams. We also propose an online method using sliding windows to summarize high-velocity data. The final result of the framework is a feature vector that describes the underlying processes and is ready to use in machine learning tasks. Moreover, we showcase real-world applications such as automated feature engineering for real-time monitoring and online machine learning for event classification. The proposed system has been deployed within production in a banking system, processing billions of daily traffic operations. Our experiments demonstrate the effectiveness and performance of our approach by evaluating it at different levels: processing, summarization, improvement of machine learning performance and effectiveness in an industrial setting. In the case of downstream machine learning tasks, using summarized data generated by StreamFlow results in up to 2 orders of magnitude speedups in training time without compromising predictive performance.
Deep Video Decaptioning via Subtitle Mask Prediction and Inpainting

Video decaptioning is the process of automatically removing subtitles from video frames and inpainting the captioned regions. However, directly transferring the deep-learning-based image inpainting methods to video decaptioning scenarios always requires the masks of subtitled regions, which is unavailable for subtitled video frames. To address these issues, we propose a two-stage lightweight framework. The first caption mask prediction stage uses an encoder-decoder full convolutional network with residual blocks to predict the caption mask. The second background inpainting stage uses an encoder-decoder structure with attention modules and the skip connection to repair the background areas. Extensive experiments demonstrate that our proposed model can produce better visual results and outperform state-of-the-art methods.
Systematic Discovery of Bias in Data

Detecting bias in data is an integral component of trustworthy and responsible ML. For researchers and data scientists, investigating, detecting, and becoming aware of biases present in data is an important step to correcting and making better ML decisions. Bias exists in the form of subsets that deviate from global expectations. Typically, researchers begin with a set of pre-defined protected/sensitive attributes and use them as the basis upon which deviation from expectation is examined. For instance, a researcher may examine under- or over-representation of a particular gender or race and adjust ML models accordingly. While this works for most settings, it is suboptimal, because it does not cover the true scale of all possible enumerations of subsets in the data. In this paper, we argue for a different approach to bias discovery. Instead of performing stratification across a pre-defined set of features, we ask the more open-ended question — which subset has the highest deviation between observed and expected outcomes? To answer this question, we leverage subset scanning, which efficiently maximizes measures of divergence over exponentially many combinations of feature values. We demonstrate the capabilities and advantages of subset scanning over pre-defined stratification by analyzing scanning results on the Stanford Open Policing dataset. In so doing, we uncover anomalous subsets within the data which, to the best of our knowledge, have not been discovered before and show that it is impossible to uncover such anomalies by stratifying across a set of pre-defined features.
Auto-Regressive Self-Attention Models for Diagnosis Prediction on Electronic Health Records

Insufficient data and data lacking the diversity to represent the general public is a common challenge when modelling diagnosis prediction. We consider a much larger and more diverse database of commercial Electronic Health Records than what is prevalent in the literature. We formulate a simplified version of diagnosis prediction that focuses on major developments in medical histories of patients. To this end, we leverage Auto-Regressive Self-Attention models that have seen promising applications in language modelling and extend them to incorporate ontological representations of medical codes. Additionally, we include time-intervals between diagnoses into the attention calculation. We evaluate models and baselines at different levels of diagnostic granularity and our results suggest that using very detailed clinical classifications does not significantly degrade performance, possibly allowing their use in practice. Our model outperforms all baselines and we suggest that leveraging the ontology for generating diagnosis representations is mostly helpful for rare diagnoses.
Alternatives to Classic BM25-IDF based on a New Information Theoretical Framework

The IDF (Inverse Document Frequency) term weighting method is a classic treatment of a term’s significance in information retrieval and text analytics. IDF can be derived from the information-theoretic Kullback-Leibler (KL) Divergence and has given rise to competitive methods such as TF*IDF and Okapi BM25, which is the default scoring function of ElasticSearch. We developed a new information metric called DLITE and derived from it an alternative to IDF, namely iDL, for term weighting and scoring in ranked information retrieval. In a series of experiments we conducted on multiple benchmark Text REtrieval Conference (TREC) collections, iDL methods consistently outperformed BM25, a very competitive baseline, for ad hoc retrieval. We outline the theoretical properties of DLITE that support the effectiveness of iDL. As a general information measure, we expect DLITE to be applicable in many other areas of big-data analytics and machine learning where further research will be valuable.
A Reinforcement Learning Approach for Adaptive Covariance Tuning in the Kalman Filter

State estimation and localization for the autonomous vehicle are essential for accurate navigation and safe maneuvers. The commonly used method is Kalman filtering, but its performance is affected by the noise covariance. An inappropriate set value will decrease the estimation accuracy and even makes the filter diverge. The noise covariance estimation problem has long been considered a tough issue because there is too much uncertainty in where the noise comes from and therefore unable to model it systematically. In recent years, Deep Reinforcement Learning (DRL) has made astonishing progress and is an excellent choice for tackling the problem that cannot be solved by conventional techniques, such as parameter estimation. By finely abstracting the problem as an MDP, we can use the DRL methods to solve it without too many prior assumptions. We propose an adaptive covariance tuning method applied to the Error State Extend Kalman Filter by taking advantage of DRL, called Reinforcement Learning Aided Covariance Tuning. The preliminary experiment result indicates that our method achieves a 14.73% estimation accuracy improvement on average compared with the vanilla fixed-covariance method and bound the estimation error within 0.4 m.
Towards XAI in the SOC – a user centric study of explainable alerts with SHAP and LIME

Many studies of the adoption of machine learning (ML) in Security Operation Centres (SOCs) have pointed to a lack of transparency and explanation – and thus trust – as a barrier to ML adoption, and have suggested eXplainable Artificial Intelligence (XAI) as a possible solution. However, there is a lack of studies addressing to which degree XAI indeed helps SOC analysts. Focusing on two XAI-techniques, SHAP and LIME, we have interviewed several SOC analysts to understand how XAI can be used and adapted to explain ML-generated alerts. The results show that XAI can provide valuable insights for the analyst by highlighting features and information deemed important for a given alert. As far as we are aware, we are the first to conduct such a user study of XAI usage in a SOC and this short paper provides our initial findings.
Design for Data Structures: Data Unification and Federation with Wikibase

Thanks to the open government initiative movement, many base registries have been made available and beneficial for all citizens to ensure accountability and transparency. All data are easy to access and process by people for effective, efficient public oversight. However, most base registries publish their data in a different data model or structure. As a result, building an AI system to utilize these data is challenging since they are machine-unreadable, poorly managed, and scattered. This paper proposes a design method for base registry data structures that uses unification and federation to improve the accessibility and availability of data. We also use Wikibase as an underlying knowledge graph to inherit common sense knowledge available in Wikidata to improve its usability.
Tree-based Text-Vision BERT for Video Search in Baidu Video Advertising

The advancement of the communication technology and the popularity of the smart phones foster the booming of video ads. Baidu, as one of the leading search engine companies in the world, receives billions of search queries per day. How to pair the video ads with the user search is the core task of Baidu video advertising. Due to the modality gap, the query-to-video retrieval is much more challenging than traditional query-to-document retrieval and image-to-image search. Traditionally, the query-to-video retrieval is tackled by the query-to-title retrieval, which is not reliable when the quality of tiles are not high. With the rapid progress achieved in computer vision and natural language processing in recent years, content-based search methods becomes promising for the query-to-video retrieval. Benefited f rom p retraining o n l arge-scale datasets, some visionBERT methods based on cross-modal attention have achieved excellent performance in many vision-language tasks not only in academia but also in industry. Nevertheless, the expensive computation cost of cross-modal attention makes it impractical for large-scale search in industrial applications. In this work, we present a tree-based combo-attention network (TCAN) which has been recently launched in Baidu’s dynamic video advertising platform. It provides a practical solution to deploy the heavy cross-modal attention for the large-scale query-to-video search. After the method was deployed, we observe that click-through rate gets improved by 2.29% and conversion rate by 2.63%.
Improving Representation Learning for Session-based Recommendation

Session-based recommendation aims to predict the next item in an anonymous session. Recent advances have shown the importance of exploiting inter-session dependencies, such as item-item transitions and session-session similarities. However, the existing methods either ignore the relative order of item co-occurrences or assign the same importance to co-occurrence patterns at all distances. Besides, they are prone to extracting wrong signals to learn user preferences from dependencies between sessions. To solve these problems, we propose a model called FOCOL to better exploit the intersession dependencies by considering Fine-grained item co-Occurrences and applying the COntrastive Learning framework. Specifically, to capture inter-session item-item dependencies, we propose a component called FOGCN (Fine-grained co-Occurrence Graph Convolution Network) to automatically learn the importance of item co-occurrence patterns from a global graph that encodes the detailed information about item co-occurrences such as relative order and distance. To directly capture dependencies between sessions, we view the recommendation task as a clustering problem, and propose a component called CSRL (Contrastive Session Representation Learning) to implicitly group similar sessions (i.e., sessions with the same next item) into the same cluster and push apart sessions at different clusters. Extensive experiments conducted on three public datasets show that the proposed model is superior to the state-of-the-art methods and the proposed two components can learn more informative item and session representations by considering the fine-grained item co-occurrences and directly capturing dependencies between sessions.
A Novel GPU-Accelerated Algorithm to Discover Periodic-Frequent Patterns in Temporal Databases

Periodic-frequent pattern mining is a vital knowledge discovery technique that aims to find all regularly occurring patterns in a temporal database. Previous studies focused on developing CPU-centric algorithms by disregarding the speedups offered by the GPUs. Furthermore, existing GPU-based frequent pattern mining algorithms cannot be employed to find periodic-frequent patterns because they ignore the items' temporal occurrence information in the database, and the multi-threaded sum-reduction technique cannot be employed to determine the periodicity of a pattern in a database. With this motivation, this paper proposes an efficient GPU-accelerated depth-first search algorithm, GPU Periodic Frequent-Miner (gPF-Miner), to find the desired patterns. Our algorithm employs a novel flattened array structure to effectively record the temporal occurrence information of every item in a database. Our algorithm also introduces a new multi-threaded parallelization technique to calculate the support and periodicity of a pattern in a GPU. This technique’s best and worst-case time complexities are O(1) and O(n), where n represents the data size. Experimental results demonstrate that gPF-Miner outperforms the existing CPU-based and naive GPU-based algorithms by a vast margin.
Named Entity Recognition in Long Documents: An End-to-end Case Study in the Legal Domain

Named entity recognition (NER) is a fundamental task for several important applications such as knowledge base construction and semantic search. So far, the focus has been on building machine learning models, which identify generic named entities (e.g., person, date). Such models can be used off-the-shelf without requiring ground truth labels for training. However, such models cannot generalize to specialized domains that have domain-specific named entities (e.g., the legal domain). In these cases, it is inevitable to generate ground truth data and experiment with a variety of models in order to achieve good performance. Motivated by a real use case from the financial sector, we discuss the approach and lessons learned when solving the NER problem in the legal domain. This task is particularly challenging because it requires extensive human expertise to produce high quality ground-truth labels. For solving the legal domain NER problem, we first crawl a large dataset of legal documents and then introduce a semi-automated process to generate high-quality labels for a set of eleven predefined named entities. We validate that the proposed approach achieves high quality labels that outperform popular out-of-the-box NER methods. On top of that, our method once followed, can generate ground truth labels for the pre-defined named entities for an unbounded number of documents. Next, we experiment with a set of models and training procedures and report their performance on the NER task. Our experimental evaluation confirms that most of the models can generalize very well, achieving F1-score between 86% and 98.9%. The dataset, the labels produced by human annotators and our semi-supervised approach, as well as our code are made available to the research community.
End-to-end Multiple Instance Learning with Gradient Accumulation

Being able to learn on weakly labeled data and provide interpretability are two of the main reasons why attention-based deep multiple instance learning (ABMIL) methods have become particularly popular for classification of histopathological images. Such image data usually come in the form of gigapixel-sized whole-slide-images (WSI) that are cropped into smaller patches (instances). However, the sheer volume of the data poses a practical big data challenge: All the instances from one WSI cannot fit the GPU memory of conventional deep-learning models. Existing solutions compromise training by relying on pre-trained models, strategic selection of instances, sub-sampling, or self-supervised pre-training. We propose a training strategy based on gradient accumulation that enables direct end-to-end training of ABMIL models without being limited by GPU memory. We conduct experiments on both QMNIST and Imagenette to investigate the performance and training time and compare with the conventional memory-expensive baseline as well as a recent sampled-based approach. This memory-efficient approach, although slower, reaches performance indistinguishable from the memory-expensive baseline.
MATE: Summarizing Alerts to Interpretable Outcomes with MITRE ATT&amp;CK

Enterprise security operations centers are inundated with a volume of alerts. Whether fact-based or anomaly-based, a multitude of alerts from security products and services are difficult to investigate when each is viewed in isolation. Pivoting from one alert to another in an attempt to connect the dots during investigation is a time-consuming and labor-intensive process. This paper introduces a practical system that automatically organizes and summarizes alerts to cases for prioritization and investigation. The system outputs interpretable threat candidates with a timeline of activities modeled after attack stages grounded in the MITRE ATT&CK framework.