Reviewerly/exHarmony huggingface dataset:
Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning

The lottery ticket hypothesis states that a highly sparsified sub-network can be trained in isolation, given the appropriate weight initialization. This paper extends that hypothesis from one-shot task learning, and demonstrates for the first time that such extremely compact and independently trainable sub-networks can be also identified in the lifelong learning scenario, which we call lifelong tickets. We show that the resulting lifelong ticket can further be leveraged to improve the performance of learning over continual tasks. However, it is highly non-trivial to conduct network pruning in the lifelong setting. Two critical roadblocks arise: i) As many tasks now arrive sequentially, finding tickets in a greedy weight pruning fashion will inevitably suffer from the intrinsic bias, that the earlier emerging tasks impact more; ii) As lifelong learning is consistently challenged by catastrophic forgetting, the compact network capacity of tickets might amplify the risk of forgetting. In view of those, we introduce two pruning options, e.g., top-down and bottom-up, for finding lifelong tickets. Compared to the top-down pruning that extends vanilla (iterative) pruning over sequential tasks, we show that the bottom-up one, which can dynamically shrink and (re-)expand model capacity, effectively avoids the undesirable excessive pruning in the early stage. We additionally introduce lottery teaching that further overcomes forgetting via knowledge distillation aided by external unlabeled data. Unifying those ingredients, we demonstrate the existence of very competitive lifelong tickets, e.g., achieving 3-8% of the dense model size with even higher accuracy, compared to strong class-incremental learning baselines on CIFAR-10/CIFAR-100/Tiny-ImageNet datasets. Codes available at https://github.com/VITA-Group/Lifelong-Learning-LTH.
Multivariate Time Series Forecast in Industrial Process Based on XGBoost and GRU

In this paper, a time series prediction model that merges eXtreme Gradient Boosting (XGBoost) and Gate Recurrent Unit (GRU), XGB-GRU model, is proposed for multivariate time series prediction in industry. The XGB-GRU model uses XGBoost's strong feature extraction capabilities to extract the hidden information of multiple control variables in industrial data. Next, the model uses GRU's unique gating unit to extract the timing information in the industrial data. Finally, the importance of XGBoost output variables to guide actual production and solve the problem of inexplicability of neural networks. Predicting the temperature of the heating furnace verifies that the proposed XGB-GRU is better than a single XGBoost and GRU model. And the model has a good fit to the predicted value.
Prosodic Features Control by Symbols as Input of Sequence-to-Sequence Acoustic Modeling for Neural TTS

This paper describes a method to control prosodic features using phonetic and prosodic symbols as input of attention-based sequence-to-sequence (seq2seq) acoustic modeling (AM) for neural text-to-speech (TTS). The method involves inserting a sequence of prosodic symbols between phonetic symbols that are then used to reproduce prosodic acoustic features, i.e. accents, pauses, accent breaks, and sentence endings, in several seq2seq AM methods. The proposed phonetic and prosodic labels have simple descriptions and a low production cost. By contrast, the labels of conventional statistical parametric speech synthesis methods are complicated, and the cost of time alignments such as aligning the boundaries of phonemes is high. The proposed method does not need the boundary positions of phonemes. We propose an automatic conversion method for conventional labels and show how to automatically reproduce pitch accents and phonemes. The results of objective and subjective evaluations show the effectiveness of our method.
Constructive Aggression? Multiple Roles of Aggressive Content in Political Discourse on Russian YouTube

Today, aggressive verbal behavior is generally perceived as a threat to integrity and democratic quality of public discussions, including those online. However, we argue that, in more restrictive political regimes, communicative aggression may play constructive roles in both discussion dynamics and empowerment of political groups. This might be especially true for restrictive political and legal environments like Russia, where obscene speech is prohibited by law in registered media and the political environment does not give much space for voicing discontent. Taking Russian YouTube as an example, we explore the roles of two under-researched types of communicative aggression—obscene speech and politically motivated hate speech—within the publics of video commenters. For that, we use the case of the Moscow protests of 2019 against non-admission of independent and oppositional candidates to run for the Moscow city parliament. The sample of over 77,000 comments for 13 videos of more than 100,000 views has undergone pre-processing and vocabulary-based detection of aggression. To assess the impact of hate speech upon the dynamics of the discussions, we have used Granger tests and assessment of discussion histograms; we have also assessed the selected groups of posts in an exploratory manner. Our findings demonstrate that communicative aggression helps to express immediate support and solidarity. It also contextualizes the criticism towards both the authorities and regime challengers, as well as demarcates the counter-public.
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention

Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences -- a topic being actively studied in the community. To address this limitation, we propose Nystr\"{o}mformer -- a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nystr\"{o}m method to approximate standard self-attention with $O(n)$ complexity. The scalability of Nystr\"{o}mformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and IMDB reviews with standard sequence length, and find that our Nystr\"{o}mformer performs comparably, or in a few cases, even slightly better, than standard self-attention. On longer sequence tasks in the Long Range Arena (LRA) benchmark, Nystr\"{o}mformer performs favorably relative to other efficient self-attention methods. Our code is available at https://github.com/mlpen/Nystromformer.
Bloch–Landau–Zener dynamics induced by a synthetic field in a photonic quantum walk

Quantum walks are processes that model dynamics in coherent systems. Their experimental implementations proved key to unveil novel phenomena in Floquet topological insulators. Here we realize a photonic quantum walk in the presence of a synthetic gauge field, which mimics the action of an electric field on a charged particle. By tuning the energy gaps between the two quasi-energy bands, we investigate intriguing system dynamics characterized by the interplay between Bloch oscillations and Landau-Zener transitions. When both gaps at quasi-energy values 0 and $\pi$ are vanishingly small, the Floquet dynamics follows a ballistic spreading.
Photonic Convolution Neural Network Based on Interleaved Time-Wavelength Modulation

Convolution neural network (CNN), as one of the most powerful and popular technologies, has achieved remarkable progress for image and video classification since its invention in 1989. However, with the high definition video-data explosion, convolution layers in the CNN architecture will occupy a great amount of computing time and memory resources due to high computation complexity of matrix multiply accumulate operation. In this paper, a novel integrated photonic CNN is proposed based on double correlation operations through interleaved time-wavelength modulation. Micro-ring based multi-wavelength manipulation and single dispersion medium are utilized to realize convolution operation and replace the conventional optical delay lines. 200 images are tested in MNIST datasets with accuracy of 85.5% in our photonic CNN versus 86.5% in 64-bit computer.We also analyze the computing error of photonic CNN caused by various micro-ring parameters, operation baud rates and the characteristics of micro-ring weighting bank. Furthermore, a tensor processing unit based on 4x4 mesh with 1.2 TOPS (operation per second when 100% utilization) computing capability at 20G baud rate is proposed and analyzed to form a paralleled photonic CNN.
The effects of national culture on financial sector development: Evidence from emerging and developing economies

This study investigates the effects of national culture on financial sector development in emerging and developing economies. Prior literature on culture and finance has placed little focus on the multidimensionality of national culture and its potential implications for financial sector development. Therefore, this study contributes to the ongoing debate surrounding culture and finance by exploring broader cultural perspectives. The empirical findings show that national culture significantly explains cross-country differences in financial sector development. Various dimensions of national culture have a distinctive impact on financial sector development; for example, individualism and masculinity play boosting role, whereas uncertainty avoidance hampers it. The overall findings are reliable to a series of robustness checks and offer useful policy implications for governments, regulatory agencies, and other stakeholders.
Quantum Speedup for Protein Structure Prediction

Protein structure prediction (PSP) predicts the native conformation for a given protein sequence. Classically, the problem has been shown to belong to the NP-complete complexity class. Its applications range from physics, through bioinformatics to medicine and quantum biology. It is possible however to speed it up with quantum computational methods, as we show in this paper. Here we develop a fast quantum algorithm for PSP in three-dimensional hydrophobic-hydrophilic model on body-centered cubic lattice with quadratic speedup over its classical counterparts. Given a protein sequence of n amino acids, our algorithm reduces the temporal and spatial complexities to, respectively, [Formula: see text] and O(n2 logn) . With respect to oracle-related quantum algorithms for the NP-complete problems, we identify our algorithm as optimal. To justify the feasibility of the proposed algorithm we successfully solve the problem on IBM quantum simulator involving 21 and 25 qubits. We confirm the experimentally obtained high probability of success in finding the desired conformation by calculating the theoretical probability estimations.
Software product-line evaluation in the large

Abstract Software product-line engineering is arguably one of the most successful methods for establishing large portfolios of software variants in an application domain. However, despite the benefits, establishing a product line requires substantial upfront investments into a software platform with a proper product-line architecture, into new software-engineering processes (domain engineering and application engineering), into business strategies with commercially successful product-line visions and financial planning, as well as into re-organization of development teams. Moreover, establishing a full-fledged product line is not always possible or desired, and thus organizations often adopt product-line engineering only to an extent that deemed necessary or was possible. However, understanding the current state of adoption, namely, the maturity or performance of product-line engineering in an organization, is challenging, while being crucial to steer investments. To this end, several measurement methods have been proposed in the literature, with the most prominent one being the Family Evaluation Framework (FEF), introduced almost two decades ago. Unfortunately, applying it is not straightforward, and the benefits of using it have not been assessed so far. We present an experience report of applying the FEF to nine medium- to large-scale product lines in the avionics domain. We discuss how we tailored and executed the FEF, together with the relevant adaptations and extensions we needed to perform. Specifically, we elicited the data for the FEF assessment with 27 interviews over a period of 11 months. We discuss experiences and assess the benefits of using the FEF, aiming at helping other organizations assessing their practices for engineering their portfolios of software variants.
Making Sense of Blockchain for AI Deepfakes Technology

Deepfakes generally refers to a new breed of adversarial deep learning technology to create non-consensual contents (mostly videos) for nefarious purposes. Most researches focus on the `detection' of deepfakes using AI-assisted approaches to take on this problem. This has been the common method operandi used by researchers thus far. However, there is one missing aspect of the deepfake problem, which is `authentication'. Instead of attempting to detect what content is fake, in this paper, we focus on techniques to provide tamper-proof evidence of what content is real. Blockchain has been advocated to be helpful with the authentication aspect of many real-world scenarios. Despite the scattered efforts around such solutions, there are no studies that can shed light on where it makes sense to adopt blockchain technology to better take on the deepfake problem. This paper aims to provide a one-stop guide to using blockchain to navigate deepfake artificial intelligence. We discuss potential use cases and solutions to tackle deepfakes technology via blockchain functionalities and features.
HERMES: Fault-Tolerant Middleware for Blockchain Interoperability

&lt;div&gt;The emergence of blockchain interoperability is reducing the risk of investing in blockchain by avoiding vendor lock-in, leveraging interoperation, and providing migration capabilities. However, to fully unlock the internet of blockchains, it is necessary to provide enterprise interoperability mechanisms that allow service providers to comply with different regulations, e.g., data privacy regulations. Each blockchain can be reached via a gateway, allowing to interconnect value, to provide different services, and to enable self-sovereignty. To realize this vision, we propose Hermes, a fault-tolerant middleware that connects blockchain networks and is based on the Open Digital Asset Protocol (ODAP). Hermes is crash fault-tolerant by allying a new protocol, ODAP-2PC, with a log storage API that can leverage blockchain to secure logs, providing them transparency, auditability, availability, and non-repudiation. We introduce a use case benefiting from Hermes, digital cross-jurisdiction promissory notes. We show that cross-chain transactions can be achieved securely with Hermes, given that gateways are complying with legal frameworks.&lt;/div&gt;&lt;div&gt;&lt;br&gt;&lt;/div&gt;
Outlier detection strategies for WSNs: A survey

Wireless Sensor Networks (WSNs) are developed significantly from the last decades and attracted the attention of scientific and industrial domains. In WSNs, sensor nodes distributed autonomously in harsh environments are easily vulnerable to faults and attacks that cause sensor readings unreliable and inaccurate. In this scenario, sensor readings that have differed considerably from healthy behaviors will be considered abnormal data or anomalies/outliers. The inclusion of such outliers in data analytics will inevitably affect the outcome of the decision-making process. Thus, detecting outliers in WSNs using data-driven approaches becomes a novel technique among the Machine Learning (ML) communities. Meanwhile, various research issues are there in measuring the performance of the deployed ML algorithms in detecting outliers in WSNs, which generally contains minimum resources in terms of computational capability and power sources to ensure data quality. Hence, this paper presents a comprehensive overview of the state-of-the-art Statistical and Artificial Intelligence (AI) based techniques used in WSNs to detect outliers in the view of architecture, type, degree, approach, and detection mode. Furthermore, each aforementioned outlier detection approach is presented with detailed discussions and future scope for developments.
A modified batch intrinsic plasticity method for pre-training the random coefficients of extreme learning machines

In extreme learning machines (ELM) the hidden-layer coefficients are randomly set and fixed, while the output-layer coefficients of the neural network are computed by a least squares method. The randomly-assigned coefficients in ELM are known to influence its performance and accuracy significantly. In this paper we present a modified batch intrinsic plasticity (modBIP) method for pre-training the random coefficients in the ELM neural networks. The current method is devised based on the same principle as the batch intrinsic plasticity (BIP) method, namely, by enhancing the information transmission in every node of the neural network. It differs from BIP in two prominent aspects. First, modBIP does not involve the activation function in its algorithm, and it can be applied with any activation function in the neural network. In contrast, BIP employs the inverse of the activation function in its construction, and requires the activation function to be invertible (or monotonic). The modBIP method can work with the often-used non-monotonic activation functions (e.g. Gaussian, swish, Gaussian error linear unit, and radial-basis type functions), with which BIP breaks down. Second, modBIP generates target samples on random intervals with a minimum size, which leads to highly accurate computation results when combined with ELM. The combined ELM/modBIP method is markedly more accurate than ELM/BIP in numerical simulations. Ample numerical experiments are presented with shallow and deep neural networks for function approximation and boundary/initial value problems with partial differential equations. They demonstrate that the combined ELM/modBIP method produces highly accurate simulation results, and that its accuracy is insensitive to the random-coefficient initializations in the neural network. This is in sharp contrast with the ELM results without pre-training of the random coefficients.
Estimating expectation values using approximate quantum states

We introduce an approximate description of an <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>N</mml:mi></mml:math>-qubit state, which contains sufficient information to estimate the expectation value of any observable to a precision that is upper bounded by the ratio of a suitably-defined seminorm of the observable to the square root of the number of the system's identical preparations <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>M</mml:mi></mml:math>, with no explicit dependence on <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>N</mml:mi></mml:math>. We describe an operational procedure for constructing the approximate description of the state that requires, besides the quantum state preparation, only single-qubit rotations followed by single-qubit measurements. We show that following this procedure, the cardinality of the resulting description of the state grows as <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>3</mml:mn><mml:mi>M</mml:mi><mml:mi>N</mml:mi></mml:math>. We test the proposed method on Rigetti's quantum processor unit with 12, 16 and 25 qubits for random states and random observables, and find an excellent agreement with the theory, despite experimental errors.
Modern Pedagogical Techniques in Teaching French to Prepare Engineering University Students for Academic Mobility

The paper focuses on the practice of preparing Kazan National Research Technological University students for academic mobility to universities of France and development of a language training program to facilitate the students’ academic mobility. Every year, several KNRTU students, after completing the undergraduate studies, leave to do the Masters at French universities. In order to better prepare students for academic mobility to French universities, French teachers have developed a special training program, aiming to make it easier for students to continue their studies at universities and everyday life in France. The paper shows the methodology of the educational program of preparation for academic mobility. Besides, KNRTU French teachers have developed a program to prepare students for international DELF (B1, B2) exam to gain an international diploma, as students have the right to study at French universities only with these international diplomas. The paper discusses the pedagogical technologies that are used at French classes to prepare students for academic mobility, namely: multi-level training, problem-oriented learning technologies, collaborative training, project-based studies, game-based learning technologies, information and computer technologies. The teacher should choose such teaching methods that would allow the students to show their activity and master the practical aspects of a foreign language.
Deep Reinforcement Learning-Based Path Planning for Multi-Arm Manipulators with Periodically Moving Obstacles

In the workspace of robot manipulators in practice, it is common that there are both static and periodic moving obstacles. Existing results in the literature have been focusing mainly on the static obstacles. This paper is concerned with multi-arm manipulators with periodically moving obstacles. Due to the high-dimensional property and the moving obstacles, existing results suffer from finding the optimal path for given arbitrary starting and goal points. To solve the path planning problem, this paper presents a SAC-based (Soft actor–critic) path planning algorithm for multi-arm manipulators with periodically moving obstacles. In particular, the deep neural networks in the SAC are designed such that they utilize the position information of the moving obstacles over the past finite time horizon. In addition, the hindsight experience replay (HER) technique is employed to use the training data efficiently. In order to show the performance of the proposed SAC-based path planning, both simulation and experiment results using open manipulators are given.