Miracle-dz/UniFAR-FTUs huggingface dataset:
Utilizing Redundancy in Cost Functions for Resilience in Distributed Optimization and Learning

This paper considers the problem of resilient distributed optimization and stochastic machine learning in a server-based architecture. The system comprises a server and multiple agents, where each agent has a local cost function. The agents collaborate with the server to find a minimum of their aggregate cost functions. We consider the case when some of the agents may be asynchronous and/or Byzantine faulty. In this case, the classical algorithm of distributed gradient descent (DGD) is rendered ineffective. Our goal is to design techniques improving the efficacy of DGD with asynchrony and Byzantine failures. To do so, we start by proposing a way to model the agents’ cost functions by the generic notion of (f, r; )-redundancy where f and r are the parameters of Byzantine failures and asynchrony, respectively, and characterizes the closeness between agents’ cost functions. This allows us to quantify the level of redundancy present amongst the agents’ cost functions, for any given distributed optimization problem. We demonstrate, both theoretically and empirically, the merits of our proposed redundancy model in improving the robustness of DGD against asynchronous and Byzantine agents, and their extensions to distributed stochastic gradient descent (D-SGD) for robust distributed machine learning with asynchronous and Byzantine agents. This report supersedes our previous report [36] as it contains the most of the results in it. Georgetown University. Email: sl1539@georgetown.edu. École Polytechnique Fédérale de Lausanne (EPFL). Email: nirupam.gupta@epfl.ch. Georgetown University. Email: nitin.vaidya@georgetown.edu. 1 ar X iv :2 11 0. 10 85 8v 1 [ cs . D C ] 2 1 O ct 2 02 1
SocialVisTUM: An Interactive Visualization Toolkit for Correlated Neural Topic Models on Social Media Opinion Mining

Recent research in opinion mining proposed word embedding-based topic modeling methods that provide superior coherence compared to traditional topic modeling. In this paper, we demonstrate how these methods can be used to display correlated topic models on social media texts using SocialVisTUM, our proposed interactive visualization toolkit. It displays a graph with topics as nodes and their correlations as edges. Further details are displayed interactively to support the exploration of large text collections, e.g., representative words and sentences of topics, topic and sentiment distributions, hierarchical topic clustering, and customizable, predefined topic labels. The toolkit optimizes automatically on custom data for optimal coherence. We show a working instance of the toolkit on data crawled from English social media discussions about organic food consumption. The visualization confirms findings of a qualitative consumer research study. SocialVisTUM and its training procedures are accessible online.
Solving the Real Robot Challenge Using Deep Reinforcement Learning

This paper details our winning submission to Phase 1 of the 2021 Real Robot Challenge; a challenge in which a three-fingered robot must carry a cube along specified goal trajectories. To solve Phase 1, we use a pure reinforcement learning approach which requires minimal expert knowledge of the robotic system, or of robotic grasping in general. A sparse, goal-based reward is employed in conjunction with Hindsight Experience Replay to teach the control policy to move the cube to the desired x and y coordinates of the goal. Simultaneously, a dense distance-based reward is employed to teach the policy to lift the cube to the z coordinate (the height component) of the goal. The policy is trained in simulation with domain randomisation before being transferred to the real robot for evaluation. Although performance tends to worsen after this transfer, our best policy can successfully lift the real cube along goal trajectories via an effective pinching grasp. Our approach outperforms all other submissions, including those leveraging more traditional robotic control techniques, and is the first pure learning-based method to solve this challenge.
Distributionally Robust Classifiers in Sentiment Analysis

In this paper, we propose sentiment classification models based on BERT integrated with DRO (Distributionally Robust Classifiers) to improve model performance on datasets with distributional shifts. We added 2-Layer Bi-LSTM, projection layer (onto simplex or Lp ball), and linear layer on top of BERT to achieve distributionally robustness. We considered one form of distributional shift (from IMDb dataset to Rotten Tomatoes dataset). We have confirmed through experiments that our DRO model does improve performance on our test set with distributional shift from the training set.
Pseudoinverse-free randomized block iterative algorithms for consistent and inconsistent linear systems

Randomized iterative algorithms have attracted much attention in recent years because they can approximately solve large-scale linear systems of equations without accessing the entire coefficient matrix. In this paper, we propose two novel pseudoinverse-free randomized block iterative algorithms for solving consistent and inconsistent linear systems. The proposed algorithms require two user-defined random matrices: one for row sampling and the other for column sampling. We can recover the well-known doubly stochastic Gauss--Seidel, randomized Kaczmarz, randomized coordinate descent, and randomized extended Kaczmarz algorithms by choosing appropriate random matrices used in our algorithms. Because our algorithms allow for a much wider selection of these two random matrices, a number of new specific algorithms can be obtained. We prove the linear convergence in the mean square sense of our algorithms. Numerical experiments for linear systems with synthetic and real-world coefficient matrices demonstrate the efficiency of some special cases of our algorithms.
Mixing and localization in random time-periodic quantum circuits of Clifford unitaries

How much do local and time-periodic dynamics resemble a random unitary? In the present work, we address this question by using the Clifford formalism from quantum computation. We analyze a Floquet model with disorder, characterized by a family of local, time-periodic, and random quantum circuits in one spatial dimension. We observe that the evolution operator enjoys an extra symmetry at times that are a half-integer multiple of the period. With this, we prove that after the scrambling time, namely, when any initial perturbation has propagated throughout the system, the evolution operator cannot be distinguished from a (Haar) random unitary when all qubits are measured with Pauli operators. This indistinguishability decreases as time goes on, which is in high contrast to the more studied case of (time-dependent) random circuits. We also prove that the evolution of Pauli operators displays a form of mixing. These results require the dimension of the local subsystem to be large. In the opposite regime, our system displays a novel form of localization, produced by the appearance of effective one-sided walls, which prevent perturbations from crossing the wall in one direction but not the other.
When Are Solutions Connected in Deep Networks?

The question of how and why the phenomenon of mode connectivity occurs in training deep neural networks has gained remarkable attention in the research community. From a theoretical perspective, two possible explanations have been proposed: (i) the loss function has connected sublevel sets, and (ii) the solutions found by stochastic gradient descent are dropout stable. While these explanations provide insights into the phenomenon, their assumptions are not always satisfied in practice. In particular, the first approach requires the network to have one layer with order of $N$ neurons ($N$ being the number of training samples), while the second one requires the loss to be almost invariant after removing half of the neurons at each layer (up to some rescaling of the remaining ones). In this work, we improve both conditions by exploiting the quality of the features at every intermediate layer together with a milder over-parameterization condition. More specifically, we show that: (i) under generic assumptions on the features of intermediate layers, it suffices that the last two hidden layers have order of $\sqrt{N}$ neurons, and (ii) if subsets of features at each layer are linearly separable, then no over-parameterization is needed to show the connectivity. Our experiments confirm that the proposed condition ensures the connectivity of solutions found by stochastic gradient descent, even in settings where the previous requirements do not hold.
Enochs' conjecture for small precovering classes of modules

Enochs’ conjecture asserts that each covering class of modules (over any fixed ring) has to be closed under direct limits. Although various special cases of the conjecture have been verified, the conjecture remains open in its full generality. In this short paper, we prove the validity of the conjecture for small precovering classes, i.e. the classes of the form Add(M) where M is any module, under a mild additional set-theoretic assumption which ensures that there are enough non-reflecting stationary sets. We even show that M has a perfect decomposition if Add(M) is a covering class. Finally, the additional set-theoretic assumption is shown to be redundant if there exists an n < ω such that M decomposes into a direct sum of אn-generated modules.
Risk factors associated with osteoporosis and fracture in psoriatic arthritis

Abstract Background: Although there are few studies mentioned there may be some relationship between psoriatic arthritis (PsA) and osteoporosis, clinical data in real world still need to be clarified in China. The aim of this study was to assess the areal and volumetric bone mineral density (BMD), frequency of fracture, and risk factors in patients with PsA. Methods: A total of one hundred PsA patients who visited Peking University First Hospital and one hundred age- and sex-matched healthy controls with DXA data were enrolled in the study. Patients with clinical fractures confirmed by X-ray during follow-up were also recorded. Clinical characteristics of the patients were recorded and compared between the abnormal BMD group and the normal BMD group, as well as between the fracture and non-fracture groups. Risk factors for fracture and low BMD were analyzed. Results: Mean BMD at the total hip and femoral neck was significantly lower in PsA patients than that in healthy controls (0.809 ± 0.193 vs. 0.901 ± 0.152 g/cm2, P = 0.041; 0.780 ± 0.146 vs. 0.865 ± 0.166 g/cm2, P = 0.037, respectively). Moreover, lumbar spine BMD was negatively correlated with psoriasis duration, swollen joint count and DAS28-CRP (r = –0.503, –0.580, –0.438; P < 0.05). Total hip BMD and femoral neck BMD were negatively correlated with HAQ (r = –0.521, –0.335; P < 0.05). Fractures occurred in 29 patients during the follow-up period. Logistic regression analysis showed that older age (OR 1.132 [95%CI: 1.026–1.248), P < 0.05], higher HAQ score (OR 1.493, 95%CI: 1.214–1.836, P < 0.01), higher disease activity index for psoriatic arthritis (OR 1.033, 95% CI: 1.002–1.679, P < 0.05) and hip joint involvement (OR 6.401, 95% CI: 4.012–44.180, P < 0.05) were risk factors for fracture in the multivariate model. Conclusions: Increased risks of osteoporosis and fracture were found in PsA patients compared to healthy controls. Besides age, high disease activity and hip joint involvement were risk factors for decreased BMD and fracture.
Effects of ethanol on the secondary structure specific hydration properties of Chymotrypsin Inhibitor 2 in its folded and unfolded forms

ABSTRACT Molecular dynamics simulations were performed for the protein Chymotrypsin Inhibitor 2 in water–ethanol binary mixtures at several ethanol concentrations at two different temperatures to explore the differential role of ethanol on the hydration properties of the secondary structural segments of the protein. We find that, compared to the water structure around the secondary structure segments of the unfolded protein in pure water, water molecules preferred to be structured more in presence of ethanol. The structuration of ethanol is noticed to be higher around helix and sheet, as compared to that around the loop. The differential slow structural relaxations of the hydrogen bonds involving the secondary structural segments of protein in water–ethanol mixed solutions, as compared to that in pure water infer that ethanol alters the water properties in a remarkable manner. The slow relaxations of the hydrogen bonds formed between ethanol and the secondary structural segments of the protein, in general, suggests that ethanol interacts directly with the protein. However, the relatively faster relaxation of the ethanol-segment hydrogen bonds involving unfolded protein as compared to that involving the folded protein suggests that the unfolded protein surface can remain dynamically free to interact with water more.
Obese, non-eosinophilic asthma: frequent exacerbators in a real-world setting

Abstract Objective In the UK, asthma deaths are at their highest level this century. Increased recognition of at-risk patients is needed. This study phenotyped frequent asthma exacerbators and used machine learning to predict frequent exacerbators. Methods Patients admitted to a district general hospital with an asthma exacerbation between 1st March 2018 and 1st March 2020 were included. Patients were organized into two groups: “Infrequent Exacerbators” (1 admission in the previous 12 months) and “Frequent Exacerbators” (≥2 admissions in the previous 12 months). Patient data were retrospectively collected from hospital and primary care records. Machine learning models were used to predict frequent exacerbators. Results 200 patients admitted for asthma exacerbations were randomly selected (73% female; mean age 47.8 years). Peripheral eosinophilia was uncommon in either group (21% vs 19%). More frequent exacerbators were being treated with high-dose ICS than infrequent exacerbators (46.5% vs 23.2%; P < 0.001), and frequent exacerbators used more SABA inhalers (10.9 vs 7.40; P = 0.01) in the year preceding the current admission. BMI was raised in both groups (34.2 vs 30.9). Logistic regression was the most accurate machine learning model for predicting frequent exacerbators (AUC = 0.80). Conclusions Patients admitted for asthma are predominately female, obese and non-eosinophilic. Patients who require multiple admissions per year have poorer asthma control at baseline. Machine learning algorithms can predict frequent exacerbators using clinical data available in primary care. Instead of simply increasing the dose of corticosteroids, multidisciplinary management targeting Th2-low inflammation should be considered for these patients.
Understanding degraded speech leads to perceptual gating of a brainstem reflex in human listeners

The ability to navigate “cocktail party” situations by focusing on sounds of interest over irrelevant, background sounds is often considered in terms of cortical mechanisms. However, subcortical circuits such as the pathway underlying the medial olivocochlear (MOC) reflex modulate the activity of the inner ear itself, supporting the extraction of salient features from auditory scene prior to any cortical processing. To understand the contribution of auditory subcortical nuclei and the cochlea in complex listening tasks, we made physiological recordings along the auditory pathway while listeners engaged in detecting non(sense) words in lists of words. Both naturally spoken and intrinsically noisy, vocoded speech—filtering that mimics processing by a cochlear implant (CI)—significantly activated the MOC reflex, but this was not the case for speech in background noise, which more engaged midbrain and cortical resources. A model of the initial stages of auditory processing reproduced specific effects of each form of speech degradation, providing a rationale for goal-directed gating of the MOC reflex based on enhancing the representation of the energy envelope of the acoustic waveform. Our data reveal the coexistence of 2 strategies in the auditory system that may facilitate speech understanding in situations where the signal is either intrinsically degraded or masked by extrinsic acoustic energy. Whereas intrinsically degraded streams recruit the MOC reflex to improve representation of speech cues peripherally, extrinsically masked streams rely more on higher auditory centres to denoise signals.
MDF-SA-DDI: predicting drug-drug interaction events based on multi-source drug fusion, multi-source feature fusion and transformer self-attention mechanism

One of the main problems with the joint use of multiple drugs is that it may cause adverse drug interactions and side effects that damage the body. Therefore, it is important to predict potential drug interactions. However, most of the available prediction methods can only predict whether two drugs interact or not, whereas few methods can predict interaction events between two drugs. Accurately predicting interaction events of two drugs is more useful for researchers to study the mechanism of the interaction of two drugs. In the present study, we propose a novel method, MDF-SA-DDI, which predicts drug-drug interaction (DDI) events based on multi-source drug fusion, multi-source feature fusion and transformer self-attention mechanism. MDF-SA-DDI is mainly composed of two parts: multi-source drug fusion and multi-source feature fusion. First, we combine two drugs in four different ways and input the combined drug feature representation into four different drug fusion networks (Siamese network, convolutional neural network and two auto-encoders) to obtain the latent feature vectors of the drug pairs, in which the two auto-encoders have the same structure, and their main difference is the number of neurons in the input layer of the two auto-encoders. Then, we use transformer blocks that include self-attention mechanism to perform latent feature fusion. We conducted experiments on three different tasks with two datasets. On the small dataset, the area under the precision-recall-curve (AUPR) and F1 scores of our method on task 1 reached 0.9737 and 0.8878, respectively, which were better than the state-of-the-art method. On the large dataset, the AUPR and F1 scores of our method on task 1 reached 0.9773 and 0.9117, respectively. In task 2 and task 3 of two datasets, our method also achieved the same or better performance as the state-of-the-art method. More importantly, the case studies on five DDI events are conducted and achieved satisfactory performance. The source codes and data are available at https://github.com/ShenggengLin/MDF-SA-DDI.
Adsorption of plantaricin Q7 on montmorillonite and application in feedback regulation of plantaricin Q7 synthesis by Lactobacillus plantarum Q7

Kieselguhr, bentonite, and montmorillonite were investigated as potential carriers of plantaricin Q7. Highest level of adsorption of plantaricin Q7 was obtained with montmorillonite. Meanwhile, visible inhibition zones were observed against Listeria monocytogenes for montmorillonite adsorbed with plantaricin Q7. Adsorption kinetics showed that the adsorption behaviour followed the pseudo‐first‐order and Weber's intra‐particle diffusion models, suggesting two steps had taken place during the adsorption process. X‐ray diffraction assays revealed that plantaricin Q7 was intercalated into the interlayer space of montmorillonites. Electrostatic, hydrogen bonding and hydrophobic interactions proved to play important roles in the mechanisms of interaction between montmorillonite and plantaricin Q7, as shown by infrared spectroscopy analysis. In addition, plantaricin Q7 production was inhibited by feedback regulation with its high concentrations. In order to remove product feedback inhibition in plantaricin Q7 production, a strategy was implemented for its adsorption onto montmorillonite during fermentation. The final plantaricin Q7 output reached 3713.40 IU/mL during fermentation using montmorillonite to adsorb plantaricin Q7, 41.61% higher than that of non‐ montmorillonite. These results indicate that montmorillonites are suitable carriers for plantaricin Q7 adsorption, and the adsorption of plantaricin Q7 onto montmorillonite during fermentation could be a good method to increase final plantaricin Q7 production.
The apparent need for better communication between clinicians and patients regarding elevated blood pressure among United States emerging adults

Abstract Objectives: To examine the discrepancy between ‘told’ and screened blood pressure (BP) category, and the association between body composition and told BP category. Methods: Between 08/2019-03/2020 college students (n = 1057) were asked what category they had previously been told their BP falls into prior to having their BP and body composition (waist circumference, body mass index (BMI), and body fat percentage) objectively assessed. Results: A disproportionate number (>80%) who were told that they had normal BP were categorized as having elevated BP. Men told they had borderline/high BP had less favorable body compositions than those told they had low/normal BP. Less favorable body composition was associated with increased likelihood of men being told that they had borderline/high BP. Conclusions: Better BP screening policies and procedures would ensure healthcare providers discuss elevated BP with emerging adults regardless of body composition and avoid missing opportunities to delivery of early and cost-effective interventions.
UV protection and antimicrobial finish on cotton khadi fabric using a mixture of nanoparticles of zinc oxide and poly-hydroxy-amino methyl silicone

A different percentage of nanoparticles of zinc oxide dispersed in a newer amino-silicone binder (poly-hydroxy-amino methyl silicone) were applied to bleached cotton khadi (handloom woven from handspun yarns) fabric to impart both ultraviolet protection and an antimicrobial finish in one step using the pad-dry-cure method, instead of using two processes for two different finishes. Amongst the varying dosages of nanoparticles of zinc oxide (1% to 5%) on the weight of fabric dispersed in poly-hydroxy-amino methyl silicone (2–10%) owf, 1% nanoparticles of zinc oxide and 4% poly-hydroxy-amino methyl silicone show ultraviolet protection factor 10 and 93–95% antibacterial reduction, whereas a 4% poly-hydroxy-amino methyl silicone and 5% nanoparticles of zinc oxide combination yields ultraviolet protection factor 20 and 99% antibacterial reduction. Thus, nanoparticles of zinc oxide at the level of 5% application with 4% poly-hydroxy-amino methyl silicone gives the best antimicrobial (99% bacterial reduction) and ultraviolet protection factor value of 20, balanced with 15–20% loss of fabric tenacity. Fourier transform infrared spectroscopy analysis reveals a complex formation between cellulose/oxy-cellulose and poly-hydroxy-amino methyl silicone that embeds nanoparticles of zinc oxide within it. Supporting reaction mechanisms proposed for both energy dispersive spectroscopy and atomic absorption spectrophotometry results further confirm the presence of zinc, potassium, and silicon on the treated cotton fabric. A wash stability test also shows the stability of the antimicrobial treatment for up to five wash cycles with 96% bacterial reduction and retention of ultraviolet protection factor of 15 after five washes. Thus, this single step combining ultraviolet protective and antimicrobial finishing of cotton fabric may be used for eco-fashion garments to protect the human skin from ultraviolet light and microbes alongside its possible uses in medical textiles to protect human body parts.