Reviewerly/exHarmony huggingface dataset:
Neural Video Portrait Relighting in Real-time via Consistency Modeling

Video portraits relighting is critical in user-facing human photography, especially for immersive VR/AR experience. Recent advances still fail to recover consistent relit result under dynamic illuminations from monocular RGB stream, suffering from the lack of video consistency supervision. In this paper, we propose a neural approach for real-time, high-quality and coherent video portrait relighting, which jointly models the semantic, temporal and lighting consistency using a new dynamic OLAT dataset. We propose a hybrid structure and lighting disentanglement in an encoder-decoder architecture, which combines a multi-task and adversarial training strategy for semantic-aware consistency modeling. We adopt a temporal modeling scheme via flow-based supervision to encode the conjugated temporal consistency in a cross manner. We also propose a lighting sampling strategy to model the illumination consistency and mutation for natural portrait light manipulation in real-world. Extensive experiments demonstrate the effectiveness of our approach for consistent video portrait light-editing and relighting, even using mobile computing.
Practical Isogeny-Based Key-Exchange with Optimal Tightness

We exploit the Diffie-Hellman-like structure of CSIDH to build a quantum-resistant authenticated key-exchange algorithm. Our security proof has optimal tightness, which means that the protocol is efficient even when instantiated with theoretically-sound security parameters. Compared to previous isogeny-based authenticated key-exchange protocols, our scheme is extremely simple, its security relies only on the underlying CSIDH-problem and it has optimal communication complexity for CSIDH-based protocols. Our security proof relies heavily on the re-randomizability of CSIDH-like problems and carries on in the ROM.
Improving Robustness of Facial Landmark Detection by Defending against Adversarial Attacks

Many recent developments in facial landmark detection have been driven by stacking model parameters or augmenting annotations. However, three subsequent challenges remain, including 1) an increase in computational overhead, 2) the risk of overfitting caused by increasing model parameters, and 3) the burden of labor-intensive annotation by humans. We argue that exploring the weaknesses of the detector so as to remedy them is a promising method of robust facial landmark detection. To achieve this, we propose a sample-adaptive adversarial training (SAAT) approach to interactively optimize an attacker and a detector, which improves facial landmark detection as a defense against sample-adaptive black-box attacks. By leveraging adversarial attacks, the proposed SAAT exploits adversarial perturbations beyond the handcrafted transformations to improve the detector. Specifically, an attacker generates adversarial perturbations to reflect the weakness of the detector. Then, the detector must improve its robustness to adversarial perturbations to defend against adversarial attacks. Moreover, a sample-adaptive weight is designed to balance the risks and benefits of augmenting adversarial examples to train the detector. We also introduce a masked face alignment dataset, Masked-300W, to evaluate our method. Experiments show that our SAAT performed comparably to existing state-of-the-art methods. The dataset and model are publicly available at https://github.com/zhuccly/SAAT.
Byzantine-Resilient Decentralized Stochastic Gradient Descent

Decentralized learning has gained great popularity to improve learning efficiency and preserve data privacy. Each computing node makes equal contribution to collaboratively learn a Deep Learning model. The elimination of centralized Parameter Servers (PS) can effectively address many issues such as privacy, performance bottleneck and single-point-failure. However, how to achieve Byzantine Fault Tolerance in decentralized learning systems is rarely explored, although this problem has been extensively studied in centralized systems. In this paper, we present an in-depth study towards the Byzantine resilience of decentralized learning systems with two contributions. First, from the adversarial perspective, we theoretically illustrate that Byzantine attacks are more dangerous and feasible in decentralized learning systems: even one malicious participant can arbitrarily alter the models of other participants by sending carefully crafted updates to its neighbors. Second, from the defense perspective, we propose Ubar, a novel algorithm to enhance decentralized learning with Byzantine Fault Tolerance. Specifically, Ubar provides a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</b> niform <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</b> yzantine-resilient <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> ggregation <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</b> ule for benign nodes to select the useful parameter updates and filter out the malicious ones in each training iteration. It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes. We conduct extensive experiments on standard image classification tasks and the results indicate that Ubar can effectively defeat both simple and sophisticated Byzantine attacks with higher performance efficiency than existing solutions.
Discriminative unimodal feature selection and fusion for RGB-D salient object detection

Most existing RGB-D salient object detectors make use of the complementary information of RGB-D images to overcome the challenging scenarios, e.g., low contrast, clutter backgrounds. However, these models generally neglect the fact that one of the input images may be poor in quality. This will adversely affect the discriminative ability of cross-modal features when the two channels are fused directly. To address this issue, a novel end-to-end RGB-D salient object detection model is proposed in this paper. At the core of our model is a Semantic-Guided Modality-Weight Map Generation (SG-MWMG) sub-network, producing modality-weight maps to indicate which regions on both modalities are high-quality regions, given input RGB-D images and the guidance of their semantic information. Based on it, a Bi-directional Multi-scale Cross-modal Feature Fusion (Bi-MCFF) module is presented, where the interactions of the features across different modalities and scales are exploited by using a novel bi-directional structure for better capturing cross-scale and cross-modal complementary information. The experimental results on several benchmark datasets verify the effectiveness and superiority of the proposed method over some state-of-the-art methods.
Analyzing DistilBERT for Sentiment Classification of Banking Financial News

In this paper, the sentiment classification approaches are introduced in Indian banking, governmental and global news. The study assesses state-of-art deep contextual language representation, DistilBERT, and traditional context-independent system, TF-IDF, on multiclass (positive, negative, and neutral) sentiment classification news-events. The DistilBERT model is fine-tuned and fed into four supervised machine learning classifiers Random Forest, Decision Tree, Logistic Regression, and Linear SVC, and similarly with baseline TF-IDF. The findings indicate that DistilBERT can transfer basic semantic understanding to further domains and lead to greater accuracy than the baseline TF-IDF. The results also suggest that Random Forest with DistilBERT leads to higher accuracy than other ML classifiers. The Random Forest with DistilBERT achieves 78% accuracy, which is 7% more than with TF-IDF.
Distilling Global and Local Logits with Densely Connected Relations

In prevalent knowledge distillation, logits in most image recognition models are computed by global average pooling, then used to learn to encode the high-level and task-relevant knowledge. In this work, we solve the limitation of this global logit transfer in this distillation context. We point out that it prevents the transfer of informative spatial information, which provides localized knowledge as well as rich relational information across contexts of an input scene. To exploit the rich spatial information, we propose a simple yet effective logit distillation approach. We add a local spatial pooling layer branch to the penultimate layer, thereby our method extends the standard logit distillation and enables learning of both finely-localized knowledge and holistic representation. Our proposed method shows favorable accuracy improvement against the state-of-the-art methods on several image classification datasets. We show that our distilled students trained on the image classification task can be successfully leveraged for object detection and semantic segmentation tasks; this result demonstrates our method’s high transferability.
Simplicial Convolutional Neural Networks

Graphs can model networked data by representing them as nodes and their pairwise relationships as edges.Recently, signal processing and neural networks have been extended to process and learn from data on graphs, with achievements in tasks like graph signal reconstruction, graph or node classifications, and link prediction.However, these methods are only suitable for data defined on the nodes of a graph.In this paper, we propose a simplicial convolutional neural network (SCNN) architecture to learn from data defined on simplices, e.g., nodes, edges, triangles, etc.We study the SCNN permutation and orientation equivariance, complexity, and spectral analysis.Finally, we test the SCNN performance for imputing citations on a coauthorship complex.
Class-Incremental Domain Adaptation with Smoothing and Calibration for Surgical Report Generation

Generating surgical reports aimed at surgical scene understanding in robot-assisted surgery can contribute to documenting entry tasks and post-operative analysis. Despite the impressive outcome, the deep learning model degrades the performance when applied to different domains encountering domain shifts. In addition, there are new instruments and variations in surgical tissues appeared in robotic surgery. In this work, we propose class-incremental domain adaptation (CIDA) with a multi-layer transformer-based model to tackle the new classes and domain shift in the target domain to generate surgical reports during robotic surgery. To adapt incremental classes and extract domain invariant features, a class-incremental (CI) learning method with supervised contrastive (SupCon) loss is incorporated with a feature extractor. To generate caption from the extracted feature, curriculum by one-dimensional gaussian smoothing (CBS) is integrated with a multi-layer transformer-based caption prediction model. CBS smoothes the features embedding using anti-aliasing and helps the model to learn domain invariant features. We also adopt label smoothing (LS) to calibrate prediction probability and obtain better feature representation with both feature extractor and captioning model. The proposed techniques are empirically evaluated by using the datasets of two surgical domains, such as nephrectomy operations and transoral robotic surgery. We observe that domain invariant feature learning and the well-calibrated network improves the surgical report generation performance in both source and target domain under domain shift and unseen classes in the manners of one-shot and few-shot learning. The code is publicly available at https://github.com/XuMengyaAmy/CIDACaptioning.
Image Splicing-Based Forgery Detection Using Discrete Wavelet Transform and Edge Weighted Local Binary Patterns

With the advancement of the multimedia technology, the extensive accessibility of image editing applications makes it easier to tamper the contents of digital images. Furthermore, the distribution of digital images over the open channel using information and communication technology (ICT) makes it more vulnerable to forgery. The vulnerabilities in telecommunication infrastructure open the doors for intruders to introduce deceiving changes in image data, which is hard to detect. The forged images can create severe social and legal troubles if altered with malicious purpose. Image forgery detection necessitates the development of sophisticated techniques that can efficiently detect the alterations in the digital image. Splicing forgery is commonly used to conceal the reality in images. Splicing introduces high contrast in the corners, smooth regions, and edges. We proposed a novel image forgery detection technique based on image splicing using Discrete Wavelet Transform and histograms of discriminative robust local binary patterns. First, a given color image is transformed in YCbCr color space and then Discrete Wavelet Transform (DWT) is applied on Cb and Cr components of the digital image. Texture variation in each subband of DWT is described using the dominant rotated local binary patterns (DRLBP). The DRLBP from each subband are concatenated to produce the final feature vector. Finally, a support vector machine is used to develop image forgery detection model. The performance and generalization of the proposed technique were evaluated on publicly available benchmark datasets. The proposed technique outperformed the state-of-the-art forgery detection techniques with 98.95% detection accuracy.
Progressively real-time video salient object detection via cascaded fully convolutional networks with motion attention

Semantics and motion are two cues of essence for the success in video salient object detection. Most existing deep-learning based approaches extract semantic features by the use of only one fully convolutional network with simple stacked encoders. They simulate motion patterns of video objects with two consecutive frames being simultaneously fed into a convolutional LSTM network or a weights-sharing fully convolutional network. However, such approaches have the shortcomings of producing a coarse predicted saliency map or requiring significant computational overheads. In this paper, we present a novel approach with cascaded fully convolutional networks involving motion attention (abbreviated as CFCN-MA), to achieve real-time saliency detection in videos. Our key idea is to construct twofold fully convolutional networks in order to gain a saliency map from coarse to fine. We devise an optical flow-based motion attention mechanism to improve the prediction accuracy of the initial fully convolutional networks, using the popular FlowNet2-SD model that is efficient and effective for motion pattern recognition of distinctive objects in videos. This method can obtain a fine saliency map with a refined region of interest. Moreover, we propose a means for calculating attention-guided intersection-over-union loss (shortnamed as AIoU) to supervise the CFCN-MA model in learning a saliency map with both clear edge and complete structure. Our approach is evaluated on three popular benchmark datasets, namely DAVIS, ViSal and FBMS. Experimental results demonstrate that our method outperforms many state-of-the-art techniques while meeting the real-time demand at 27 fps.
Fraud Detection Using the Fraud Triangle Theory and Data Mining Techniques: A Literature Review

Fraud entails deception in order to obtain illegal gains; thus, it is mainly evidenced within financial institutions and is a matter of general interest. The problem is particularly complex, since perpetrators of fraud could belong to any position, from top managers to payroll employees. Fraud detection has traditionally been performed by auditors, who mainly employ manual techniques. These could take too long to process fraud-related evidence. Data mining, machine learning, and, as of recently, deep learning strategies are being used to automate this type of processing. Many related techniques have been developed to analyze, detect, and prevent fraud-related behavior, with the fraud triangle associated with the classic auditing model being one of the most important of these. This work aims to review current work related to fraud detection that uses the fraud triangle in addition to machine learning and deep learning techniques. We used the Kitchenham methodology to analyze the research works related to fraud detection from the last decade. This review provides evidence that fraud is an area of active investigation. Several works related to fraud detection using machine learning techniques were identified without the evidence that they incorporated the fraud triangle as a method for more efficient analysis.
Sentiment Analysis of Rumor Spread Amid COVID-19: Based on Weibo Text

(1) Background: in early 2020, COVID-19 broke out. Driven by people's psychology of conformity, panic, group polarization, etc., various rumors appeared and spread wildly, and the Internet became a hotbed of rumors. (2) Methods: the study selected Weibo as the research media, using topic models, time series analysis, sentiment analysis, and Granger causality testing methods to analyze the social media texts related to COVID-19 rumors. (3) Results: in study 1, we obtained 21 topics related to "COVID-19 rumors" and "outbreak rumors" after conducting topic model analysis on Weibo texts; in study 2, we explored the emotional changes of netizens before and after rumor dispelling information was released and found people's positive emotions first declined and then rose; in study 3, we also explored the emotional changes of netizens before and after the "Wuhan lockdown" event and found positive sentiment of people in non-Wuhan areas increased, while negative sentiment of people in Wuhan increased; in study 4, we studied the relationship between rumor spread and emotional polarity and found negative sentiment and rumor spread was causally interrelated. (4) Conclusion: These findings could help us to intuitively understand the impact of rumors spread on people's emotions during the COVID-19 pandemic and help the government take measures to reduce panic.
Hinweise und Empfehlungen für die Konzeption von Lehrkräftefortbildungen zu digitalen Medien

For a reflective and purposeful use of information and communication technology (ICT) in and out of the classroom, school teachers need to be equipped with the necessary knowledge and skills. In addition to improving the technical infrastructure in schools, opportunities for support need to be provided. However, teachers at German schools have shown low participation rates in professional development activities concerning ICT related teaching and learning, which may indicate a lack of alignment with real needs. This article addresses teachers' preferences for topics and organizational design criteria of teachers' professional development regarding ICT use and reports results of a survey with secondary school teachers (N = 238). To obtain an information base for tailored results for the target group, a latent profile analysis utilizing technological and technological pedagogical knowledge, self-efficacy, and media use, yielded three profiles. The profiles were compared in their preferences for professional development. Results revealed a wide range of proposed topics by the teachers as well as the need for differential design of teacher professional development regarding ICT. Recommendations for the conceptual design of professional development programmes for teachers on the use of digital media are presented.
A Method Based on Attention Mechanism using Bidirectional Long-Short Term Memory(BLSTM) for Question Answering

Question answering (QA) enables the system to answer questions automatically. In recent years, much research has been done in this area. In most methods, question and answer words are given equal importance, which leads to poor model performance. This paper proposed Attention-Based Bidirectional Long-Short Term Memory(BLSTM) to select the answer to the question. In our model, first, word embedding is trained in several different ways. Then, we consider two BLSTM networks for question and answer. The outputs of these two networks and the difference between them are connected and entered into a feed-forward neural network. Finally, this network assigns a score to a question-answer pair. We evaluate our proposed model on the English and Persian datasets about Covid-19. The experiments demonstrate that our model achieves better results than other compared methods.
$$\mathsf {FLOD}$$: Oblivious Defender for Private Byzantine-Robust Federated Learning with Dishonest-Majority

Privacy and Byzantine-robustness are two major concerns of federated learning (FL), but mitigating both threats simultaneously is highly challenging: privacy-preserving strategies prohibit access to individual model updates to avoid leakage, while Byzantine-robust methods require access for comprehensive mathematical analysis. Besides, most Byzantine-robust methods only work in the honest-majority setting.We present \(\mathsf {FLOD}\), a novel oblivious defender for private Byzantine-robust FL in dishonest-majority setting. Basically, we propose a novel Hamming distance-based aggregation method to resist \(>1/2\) Byzantine attacks using a small root-dataset and server-model for bootstrapping trust. Furthermore, we employ two non-colluding servers and use additive homomorphic encryption (\(\mathsf {AHE}\)) and secure two-party computation (2PC) primitives to construct efficient privacy-preserving building blocks for secure aggregation, in which we propose two novel in-depth variants of Beaver Multiplication triples (MT) to reduce the overhead of Bit to Arithmetic (\(\mathsf {Bit2A}\)) conversion and vector weighted sum aggregation (\(\mathsf {VSWA}\)) significantly. Experiments on real-world and synthetic datasets demonstrate our effectiveness and efficiency: (i) \(\mathsf {FLOD}\) defeats known Byzantine attacks with a negligible effect on accuracy and convergence, (ii) achieves a reduction of \(\approx \)2\(\times \) for offline (resp. online) overhead of \(\mathsf {Bit2A}\) and \(\mathsf {VSWA}\) compared to \(\mathsf {ABY}\)-\(\mathsf {AHE}\) (resp. \(\mathsf {ABY}\)-\(\mathsf {MT}\)) based methods (NDSS’15), (iii) and reduces total online communication and run-time by 167–1416\(\times \) and 3.1–7.4\(\times \) compared to \(\mathsf {FLGUARD}\) (Crypto Eprint 2021/025).
Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction

Early detection of suicidal ideation in depressed individuals can allow for adequate medical attention and support, which can be life-saving. Recent NLP research focuses on classifying, from given text, if an individual is suicidal or clinically healthy. However, there have been no major attempts to differentiate between depression and suicidal ideation, which is a separate and important clinical challenge. Due to the scarce availability of EHR data, suicide notes, or other verified sources, web query data has emerged as a promising alternative. Online sources, such as Reddit, allow for anonymity, prompting honest disclosure of symptoms, making it a plausible source even in a clinical setting. However, online datasets also result in inherent noise in web-scraped labels, which necessitates a noise-removal process to improve performance. Thus, we propose SDCNL, a deep neural network approach for suicide versus depression classification. We utilize online content to train our algorithm, and to verify and correct noisy labels, we propose a novel unsupervised label correction method which, unlike previous work, does not require prior noise distribution information. Our extensive experimentation with various deep word embedding models and classifiers display strong performance of SDCNL as a new clinical application for a challenging problem (We make our supplemental, dataset, web-scraping script, and code (with hyperparameters) available at https://github.com/ayaanzhaque/SDCNL).