Reviewerly/exHarmony huggingface dataset:
Reinforced EM Algorithm for Clustering with Gaussian Mixture Models

Methods that employ the EM algorithm for parameter estimation typically face a notorious yet unsolved problem that the initialization input significantly impacts the algorithm output. We here develop a Reinforced Expectation Maximization (REM) algorithm for cluster analysis using Gaussian mixture models. The competence of REM is achieved by introducing two innovative strategies into the EM framework: (1) a mode-finding strategy for initialization that detects non-trivial modes in the data, and (2) a mode-pruning strategy for detecting true modes/mixture components of the population. The pruning strategy is well-justified in the context of mixture modelling, and we present theoretical guarantees on the quality of the initialization. Extensive experimental studies on both synthetic and real datasets show that our approach achieves better performance compared to state-of-the-art methods.
Beyond The Evidence Lower Bound: Dual Variational Graph Auto-Encoders For Node Clustering

Variational Graph Auto-Encoders (VGAEs) have achieved promising performance in several applications. Some recent models incorporate the clustering inductive bias by imposing non-Gaussian prior distributions. However, the regularization term is practically insufficient to learn the clustering structures due to the mismatch between the target and the learned distributions. Thus, we formulate a new variational lower bound that incorporates an explicit clustering objective function. The introduction of a clustering objective leads to two problems. First, the latent information destroyed by the clustering process is critical for generating the between-cluster edges. Second, the noisy and sparse input graph does not benefit from the information learned during the clustering process. To address the first problem, we identify a new term overlooked by existing Evidence Lower BOunds (ELBOs). This term accounts for the difference between the variational posterior used for the clustering task and the variational posterior associated with the generation task. Furthermore, we find that the new term increases resistance to posterior collapse. Theoretically, we demonstrate that our lower bound is a tighter approximation of the log-likelihood function. To address the second problem, we propose a graph update algorithm that reduces the over-segmentation and under-segmentation problems. We conduct several experiments to validate the merits of our approach. Our results show that the proposed method considerably improves the clustering quality compared to state-of-the-art VGAE models.
Node ranking in labeled networks

The entities in directed networks arising from real-world interactions are often naturally organized under some hierarchical structure. Given a directed, weighted, graph with edges and node labels, we introduce ranking problem where the obtained hierarchy should be described using node labels. Such method has the advantage to not only rank the nodes but also provide an explanation for such ranking. To this end, we define a binary tree called label tree, where each leaf represents a rank and each non-leaf contains a single label, which is then used to partition, and consequently, rank the nodes in the input graph. We measure the quality of trees using agony score, a penalty score that penalizes the edges from higher ranks to lower ranks based on the severity of the violation. We show that the problem is NP-hard, and even inapproximable if we limit the size of the label tree. Therefore, we resort to heuristics, and design a divide- and-conquer algorithm which runs in O((n + m) log n + ℓR), where R is the number of node-label pairs in the given graph, ℓ is the number of nodes in the resulting label tree, and n and m denote the number of nodes and edges respectively. We also report an experimental study that shows that our algorithm can be applied to large networks, that it can find ground truth in synthetic datasets, and can produce explainable hierarchies in real-world datasets.*This research is supported by the Academy of Finland projects MALSOME (343045)
Fisher Scoring Method for Neural Networks Optimization

First-order methods based on the stochastic gradient descent and variants are popularly used in training neural networks. The large dimension of the parameter space prevents the use of second-order methods in current practice. The empirical Fisher information matrix is a readily available estimate of the Hessian matrix that has been used recently to guide informative dropout approaches in deep learning. In this paper, we propose efficient ways to dynamically estimate the empirical Fisher information matrix to speed up the optimization of deep learning loss functions. We propose two different methods, both using rank-1 updates for the empirical Fisher information matrix. The first one is FisherExp and it is based on exponential smoothing using Sherman-Woodbury-Morrison matrix inversion formula. The second one is FisherFIFO, which uses a circular gradient buffer using the Sherman-Woodbury-Morrison formula twice every time a new gradient is replaced. We found that FisherFIFO scales better and we further improve scaling by proposing a partitioning strategy for the empirical Fisher Information matrix. Our methods can be used in conjunction with existing optimizers that leverage momentum-based information to improve them. We compare the performance of our methods with alternative baselines in image classification problems and found that they produce better results. Despite the overhead incurred by using second-order information, the partitioning strategy combined with parallel block updates allows us to reduce the total training time of FisherFIFO relative to the baselines.
MAPNEWS: A Framework for Aggregating and Organizing Online News Articles

In recent years, digital news has become increasingly prevalent, with many people getting their news and information from online sources rather than traditional print or broadcast media. This shift has been driven, in part, by the convenience and accessibility of digital platforms, as well as the ability to personalize and customize news feeds. Digital news also allows for greater interactivity and engagement with readers and can reach a global audience almost instantly. News articles contain a plethora of hidden spatial information that, when shared with readers, increases comprehension of current events. Only a few news aggregation systems make this information available to users. Many stories, on the other hand, are not clearly geotagged with their spatial information. In this work, we propose the MapNews framework, a novel system that gathers, analyzes, and presents news articles on a map interface, allowing users to take advantage of their underlying spatial information. MapNews pulls content from several different internet news sources and, using a custom-built geotagger, it extracts geographic content from articles. A rapid online clustering method is used to organize articles into story clusters. Panning and zooming MapNews' map interface allows readers to receive news based on geographic location and category importance, and they will view distinct articles depending on their location. MapNews achieved an ARI score of 0.89 for clustering and an accuracy of 95% in usability testing
Chapter 5 Auxiliary selection in heritage speakers of Italian

The primary focus of many investigations into heritage languages is to provide an explanation about how and why heritage grammar may differ from the standard variety as its language of origin. Various theories (Scontras et al. 2015) share the common goal of understanding what contributes to the features of certain areas of the heritage grammar, often defined as “particularly vulnerable to reanalysis” (Scontras et al. 2015: 9). Simplified features of the heritage system are viewed as evidence of incomplete acquisition (Polinsky 2006), or of divergent attainment (Scontras et al. 2015), due to reduced input (Montrul 2008). The present study moves away from these perspectives and adopts the notion of heritage grammar as an independent linguistic system, which was born from the source language of input but then developed its own set of rules. The study specifically investigates the behavior of heritage speakers of Italian living in the US in auxiliary selection between essere (to be) and avere (to have), within the Italian compound past tense passato prossimo. Data is collected through a Forced-Choice (FC) Judgment Task administered to parents and children of the same family nucleus, for a total of seven children as input receivers and six parents as native input providers of Standard Italian. The language of the parents is compared to the language of the children in order to identify possible similarities and/or differences between two independent linguistic systems, the heritage language (HL) and the source language (SL). Results show that heritage speakers of Italian select the auxiliary according to the semantic properties of the verb and that they are sensitive to the unaccusativity gradient, as identified by Sorace (2000) in native monolingual speakers of Italian. The study contributes to highlight the notion that heritage speakers are guided by a native linguistic sensitivity, which operates within the structures of their heritage grammar. More studies are needed to support the idea that the process of reanalysis to which heritage grammar is subject in specific grammatical areas may not be triggered by the reduced properties of input, but by the specific nature of the source language and its inherent degree of variation, as well as language specific properties
Chapter 9 Discourse markers in heritage Italian spoken in Flanders

This paper offers an analysis of discourse markers (DMs) produced by speakers of Italian as a heritage language (HLS) in Flanders (Belgium), where the dominant language is Belgian Dutch. We compare the DMs used in spontaneous oral productions of HLS with those employed by monolingual native speakers of Italian (MonoL1S), and by students of Italian as a second language (L2S). We investigate how the DMs used by HLS differ from those used by MonoL1S and L2S, in terms of typology, frequency, pragmatic functions, and syntactic positions. Our findings provide different interesting insights on the relationship between the multiple linguistic systems in the mind of the multilingual speaker. As for the analysis of the data, firstly, no significant difference was found among the three groups of speakers in terms of the overall quantity of DMs used; however, the qualitative analysis reveals that HLS produce DMs mainly with an interactional function. We explain this result in light of the difference in the linguistic input that HLS receive with respect to L2S. Secondly, we found several code-switching phenomena in the DMs used both by HLS and L2S. However, it is especially among HLS that code-switched DMs are implemented as “metalanguaging” devices and/or as a way to lower the cognitive load during the interaction. We conclude the paper arguing that our findings provide evidence in favor of the simultaneous access hypothesis (Muysken 2000) and for the fact that bilingual speakers treat DMs as a unified category. Adding to the debate on the categorization of DMs, we propose that the boundaries of the “fuzzy” category of DMs can be found only by considering both their functional and formal characteristics, highlighting the need for a reconciliation between the “onomasiological” and the “semasiological” approaches to study DMs as a unified linguistic category.
Reliable experimental manipulation of quantum steering direction

Noise-adding methods have been widely used to manipulate the direction of quantum steering, but all related experimental schemes only worked under the assumption that Gaussian measurements were performed and ideal target states were accurately prepared. Here, we prove, and then experimentally observe, that a class of two-qubit states can be flexibly changed among two-way steerable, one-way steerable and no-way steerable, by adding either phase damping noise or depolarization noise. The steering direction is determined by measuring steering radius and critical radius, each of which represents a necessary and sufficient steering criterion valid for general projective measurements and actually prepared states. Our work provides a more efficient and rigorous way to manipulate the direction of quantum steering, and can also be employed to manipulate other types of quantum correlations.
Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimation

Inverse path tracing has recently been applied to joint material and lighting estimation, given geometry and multi-view HDR observations of an indoor scene. However, it has two major limitations: path tracing is expensive to compute, and ambiguities exist between reflection and emission. Our Factorized Inverse Path Tracing (FIPT) addresses these challenges by using a factored light transport formulation and finds emitters driven by rendering errors. Our algorithm enables accurate material and lighting optimization faster than previous work, and is more effective at resolving ambiguities. The exhaustive experiments on synthetic scenes show that our method (1) outperforms state-of-the-art indoor inverse rendering and relighting methods particularly in the presence of complex illumination effects; (2) speeds up inverse path tracing optimization to less than an hour. We further demonstrate robustness to noisy inputs through material and lighting estimates that allow plausible relighting in a real scene. The source code is available at: https://github.com/lwwu2/fipt
VidStyleODE: Disentangled Video Editing via StyleGAN and NeuralODEs

We propose $\textbf{VidStyleODE}$, a spatiotemporally continuous disentangled $\textbf{Vid}$eo representation based upon $\textbf{Style}$GAN and Neural-$\textbf{ODE}$s. Effective traversal of the latent space learned by Generative Adversarial Networks (GANs) has been the basis for recent breakthroughs in image editing. However, the applicability of such advancements to the video domain has been hindered by the difficulty of representing and controlling videos in the latent space of GANs. In particular, videos are composed of content (i.e., appearance) and complex motion components that require a special mechanism to disentangle and control. To achieve this, VidStyleODE encodes the video content in a pre-trained StyleGAN $\mathcal{W}_+$ space and benefits from a latent ODE component to summarize the spatiotemporal dynamics of the input video. Our novel continuous video generation process then combines the two to generate high-quality and temporally consistent videos with varying frame rates. We show that our proposed method enables a variety of applications on real videos: text-guided appearance manipulation, motion manipulation, image animation, and video interpolation and extrapolation. Project website: https://cyberiada.github.io/VidStyleODE
Feature filtering module of convolutional neural networks for image recognition system

Recent research has shown that attention mechanisms can help convolutional networks train and infer more efficiently and accurately. However, the current attention mechanism mainly focuses on the relationship between global features and ignores the relationship between local features. A Feature Filtering Module (FFM) for convolutional neural networks is proposed in this paper. The FFM module uses attention mechanisms in both spatial and channel dimensions and fuses the attention feature maps of the two branches into a 3D attention feature map to help effective feature information flow more efficiently. Extensive tests on the CIFAR-100, and MS COCO show that FFM improves baseline network performance under various models and tasks, demonstrating FFM's versatility.
Contrast clustering based on representation learning

As an important research field of machine learning, clustering aims to partition the samples of unsupervised data into clusters. Traditional clustering methods, like K-means, are concise and simple to be used. But they have very limited performance on various real-world data. Recently, clustering methods based on neural networks have been proposed to improve the processing of complicated and large-scale data. However, these methods are time-consuming and still difficult to extract effective features, which limits this application. Besides, most of the clustering methods cannot be applied to online clustering. To solve these problems, we propose Representation Contrast Clustering (RCC) method in this paper. To enhance the ability to extract features, we propose to introduce contrast learning into clustering, which makes it possible to extract clustering-friendly and effective features from complex data. Without labels, contrast learning is comparable to supervised learning in extracting features. Moreover, it designed a “pre-training & fine-tuning” structure, which can save clustering time and be used for online clustering. In the pre-training phase, a contrast learning framework combining data augmentation and neural networks is used to extract a cluster-friendly representation. In the fine-tuning phase, the representations are clustered by the strategy of “label-as-representation”. Experimental results show our proposed RCC method achieves state-of-the-art performance on most datasets. For example, RCC’s NMI of 0.764 on the cifar10 dataset exceeds the known best methods by 6 percentage points, while the ACC of 0.855 is at least 6 percentage points higher than others. In summary, the RCC method does not require data integrity in either pre-training or fine-tuning stages, so RCC can run on large data and be used for online clustering. This method is very efficient and requires very little training time for a downstream task, i.e. clustering after pre-training is completed, which also achieves state-of-the-art performance on most datasets in the clustering experiments.
Video description combining visual and audio features

Video description has become a research hotspot in recent years because of its wide application value. Single visual feature information can not accurately guide the generation of accurate video description, resulting in the mismatch between the generated description text and video content. To solve this problem, a video description text generation algorithm combining visual and voice features is proposed, which enhances the accuracy of the generated description text by combining visual and voice features. First, the vision transformer model is used to extract the visual feature vector, and the Mel-Frequency Cepstral Coefficients is used to extract the audio feature vector. After the two feature vectors are spliced, the average pooling process is performed to obtain the global feature information; Secondly, the processed feature information is sent to the transformer encoder for encoding. Finally, the encoded results are sent to the transformer decoder to finally generate the video description text. The transformer framework contains a multi head self-attention mechanism, which can focus on more important video feature information while acquiring temporal feature information, making the generated text description more accurate. The method proposed in this paper has been tested on the public data sets MSRVTT and MSVD and has achieved good results in four different evaluation standards.
Semantic segmentation network based on global and local multiscale feature fusion

Semantic segmentation networks require huge computing resources, and the main reason is that they have too many feature fusions. In this paper, we propose a multi-scale feature fusion network with global and local fusion. The network is a lightweight network with strong competitiveness in speed and model size, and the segmentation accuracy is also improved to a certain extent.
Energy Policy for Clean and Sustainable Energy Production: Application of Solar Tracking Systems to Obtain Maximum Efficiency from Solar Energy in the Çukurova Region

Abstract The authors have requested that this preprint be removed from Research Square.
A method analysis of multi-modal fusion channel clustering based on attention mechanism

In the case where prior knowledge such as frequency, bandwidth, and modulation mode is unknown, the RF(radio frequency) signal from the receiver must be demodulated to a bitstream. In this paper, unsupervised clustering of logical channels such as communication system signals, service synchronization or physical layer signaling using deep learning methods is analyzed. After analysis, it is found that the current channel clustering mainly faces two problems: one is the need to manually annotate the key features of the signal dock; Second, different representations of signals, such as square spectrum and time-frequency graph, contain different feature information. At present, most of them are only analyzed for specific signal representations. Therefore, we propose to use the mixed-domain attention mechanism to automatically locate the areas that need to be focused on instead of manual participation. At the same time, use product layer for feature fusion of different signal representations (Identical/orthogonal format, Amplitude/Phase format, Time and Frequency diagram) to improve the recognition accuracy, which is a feasible direction for future research.
Unsupervised modulation classification based on multi-level deep subspace clustering

Automatic modulation classification (AMC) recognizes modulation patterns based on features extracted from received signals. Benefiting from its powerful feature extraction capabilities, deep learning is widely and successfully applied for AMC. However, existing AMC mainly based on supervised deep learning, which is highly dependent on labelled data and has difficulty constructing large-scale and well-labelled datasets, still faces challenges. To address this challenge, we propose an unsupervised learning architecture for AMC (USLAMC) by uniting feature extraction and deep subspace clustering (DSC). Feature extraction can utilize multi-level information to obtain the original dataset features. DSC can generate self-representation relationships for features, and use spectral clustering realizes unsupervised subspace feature clustering.
Interactive design of practical teaching platform of products based on augmented reality

AR technology has the potential to revolutionize the way we live or learn. The blackboard-style product structure teaching mode can no longer meet the real-time and immersive learning needs of students. Therefore, based on mobile AR technology, a platform for product practice teaching is designed. Combined with experiments, it focuses on the design of platform gesture, voice, and eye movement interaction. The platform usability test results show that it can well meet the learning needs and operating habits of students, increase the fun of learning, and provide new ideas for educational reform.
Versatile DMDs as input information platform and trainable weights in optical neural networks

Digital micromirror devices are versatile, high performance photonic components that combine high configurability with a large number of programmable parameters and high bandwidth. These are essential features in photonic neural networks. A DMD's mirrors can optically encode information to be injected into a photonics neural network, or they can even provide configurable connections between photonic neurons of the neural network itself. Their easy programmability makes them highly attractive, as through this feature DMDs act as the interface between the analogue world of the photonic neural network, and the digital world of programming languages as well as information processing. I will introduce several of such DMD-based operations in photonic AI will sketch future possibilities for the development of the field.