Reviewerly/exHarmony huggingface dataset:
Generalizing Deep Learning for Medical Image Segmentation to Unseen Domains via Deep Stacked Transformation

Recent advances in deep learning for medical image segmentation demonstrate expert-level accuracy. However, application of these models in clinically realistic environments can result in poor generalization and decreased accuracy, mainly due to the domain shift across different hospitals, scanner vendors, imaging protocols, and patient populations etc. Common transfer learning and domain adaptation techniques are proposed to address this bottleneck. However, these solutions require data (and annotations) from the target domain to retrain the model, and is therefore restrictive in practice for widespread model deployment. Ideally, we wish to have a trained (locked) model that can work uniformly well across unseen domains without further training. In this paper, we propose a deep stacked transformation approach for domain generalization. Specifically, a series of ${n}$ stacked transformations are applied to each image during network training. The underlying assumption is that the “expected” domain shift for a specific medical imaging modality could be simulated by applying extensive data augmentation on a single source domain, and consequently, a deep model trained on the augmented “big” data (BigAug) could generalize well on unseen domains. We exploit four surprisingly effective, but previously understudied, image-based characteristics for data augmentation to overcome the domain generalization problem. We train and evaluate the BigAug model (with ${n}={9}$ transformations) on three different 3D segmentation tasks (prostate gland, left atrial, left ventricle) covering two medical imaging modalities (MRI and ultrasound) involving eight publicly available challenge datasets. The results show that when training on relatively small dataset (n = 10~32 volumes, depending on the size of the available datasets) from a single source domain: (i) BigAug models degrade an average of 11%(Dice score change) from source to unseen domain, substantially better than conventional augmentation (degrading 39%) and CycleGAN-based domain adaptation method (degrading 25%), (ii) BigAug is better than “shallower” stacked transforms (i.e. those with fewer transforms) on unseen domains and demonstrates modest improvement to conventional augmentation on the source domain, (iii) after training with BigAug on one source domain, performance on an unseen domain is similar to training a model from scratch on that domain when using the same number of training samples. When training on large datasets (n = 465 volumes) with BigAug, (iv) application to unseen domains reaches the performance of state-of-the-art fully supervised models that are trained and tested on their source domains. These findings establish a strong benchmark for the study of domain generalization in medical imaging, and can be generalized to the design of highly robust deep segmentation models for clinical deployment.
Revisiting the Sibling Head in Object Detector

The "shared head for classification and localization'' (sibling head), firstly denominated in Fast RCNN, has been leading the fashion of the object detection community in the past five years. This paper provides the observation that the spatial misalignment between the two object functions in the sibling head can considerably hurt the training process, but this misalignment can be resolved by a very simple operator called task-aware spatial disentanglement (TSD). Considering the classification and regression, TSD decouples them from the spatial dimension by generating two disentangled proposals for them, which are estimated by the shared proposal. This is inspired by the natural insight that for one instance, the features in some salient area may have rich information for classification while these around the boundary may be good at bounding box regression. Surprisingly, this simple design can boost all backbones and models on both MS COCO and Google OpenImage consistently by ~3% mAP. Further, we propose a progressive constraint to enlarge the performance margin between the disentangled and the shared proposals, and gain ~1% more mAP. We show the TSD breaks through the upper bound of nowadays single-model detector by a large margin (mAP 49.4 with ResNet-101, 51.2 with SENet154), and is the core model of our 1st place solution on the Google OpenImage Challenge 2019.
Multi-attentional Deepfake Detection

Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. Recently, how to detect such forgery contents has become a hot research topic and many deepfake detection methods have been proposed. Most of them model deepfake detection as a vanilla binary classification problem, i.e, first use a backbone network to extract a global feature and then feed it into a binary classifier (real/fake). But since the difference between the real and fake images in this task is often subtle and local, we argue this vanilla solution is not optimal. In this paper, we instead formulate deepfake detection as a fine-grained classification problem and propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps. Moreover, to address the learning difficulty of this network, we further introduce a new regional independence loss and an attention guided data augmentation strategy. Through extensive experiments on different datasets, we demonstrate the superiority of our method over the vanilla binary classifier counterparts, and achieve state-of-the-art performance. The models will be released recently at https://github.com/yoctta/multiple-attention.
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs

We revisit large kernel design in modern convolutional neural networks (CNNs). Inspired by recent advances in vision transformers (ViTs), in this paper, we demonstrate that using a few large convolutional kernels instead of a stack of small kernels could be a more powerful paradigm. We suggested five guidelines, e.g., applying re-parameterized large depthwise convolutions, to design efficient high-performance large-kernel CNNs. Following the guidelines, we propose RepLKNet, a pure CNN architecture whose kernel size is as large as 31×31, in contrast to commonly used 3×3. RepLKNet greatly closes the performance gap between CNNs and ViTs, e.g., achieving comparable or superior results than Swin Transformer on ImageNet and a few typical downstream tasks, with lower latency. RepLKNet also shows nice scalability to big data and large models, obtaining 87.8% top-1 accuracy on ImageNet and 56.0% mIoU on ADE20K, which is very competitive among the state-of-the-arts with similar model sizes. Our study further reveals that, in contrast to small-kernel CNNs, large-kernel CNNs have much larger effective receptive fields and higher shape bias rather than texture bias. Code & models at https://github.com/megvii-research/RepLKNet.
Fuzzy Broad Learning System: A Novel Neuro-Fuzzy Model for Regression and Classification

A novel neuro-fuzzy model named fuzzy broad learning system (BLS) is proposed by merging the Takagi-Sugeno (TS) fuzzy system into BLS. The fuzzy BLS replaces the feature nodes of BLS with a group of TS fuzzy subsystems, and the input data are processed by each of them. Instead of aggregating the outputs of fuzzy rules produced by every fuzzy subsystem into one value immediately, all of them are sent to the enhancement layer for further nonlinear transformation to preserve the characteristic of inputs. The defuzzification outputs of all fuzzy subsystem and the outputs of enhancement layer are combined together to obtain the model output. The k -means method is employed to determine the centers of Gaussian membership functions in antecedent part and the number of fuzzy rules. The parameters that need to be calculated in a fuzzy BLS are the weights connecting the outputs of enhancement layer to model output and the randomly initialized coefficients of polynomials in consequent part in fuzzy subsystems, which can be calculated analytically. Therefore, fuzzy BLS retains the fast computational nature of BLS. The proposed fuzzy BLS is evaluated by some popular benchmarks for regression and classification, and compared with some state-of-the-art nonfuzzy and neuro-fuzzy approaches. The results indicate that fuzzy BLS outperforms other models involved. Moreover, fuzzy BLS shows advantages over neuro-fuzzy models regarding to the number of fuzzy rules and training time, which can ease the problem of rule explosion to some extent.
FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images

Estimating 3D hand pose from single RGB images is a highly ambiguous problem that relies on an unbiased training dataset. In this paper, we analyze cross-dataset generalization when training on existing datasets. We find that approaches perform well on the datasets they are trained on, but do not generalize to other datasets or in-the-wild scenarios. As a consequence, we introduce the first large-scale, multi-view hand dataset that is accompanied by both 3D hand pose and shape annotations. For annotating this real-world dataset, we propose an iterative, semi-automated `human-in-the-loop' approach, which includes hand fitting optimization to infer both the 3D pose and shape for each sample. We show that methods trained on our dataset consistently perform well when tested on other datasets. Moreover, the dataset allows us to train a network that predicts the full articulated hand shape from a single RGB image. The evaluation set can serve as a benchmark for articulated hand shape estimation.
Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View

Nowadays, in the international scientific community of machine learning, there exists an enormous discussion about the use of black-box models or explainable models; especially in practical problems.On the one hand, a part of the community defends that black-box models are more accurate than explainable models in some contexts, like image preprocessing.On the other hand, there exist another part of the community alleging that explainable models are better than black-box models because they can obtain comparable results and also they can explain these results in a language close to a human expert by using patterns.In this paper, advantages and weaknesses for each approach are shown; taking into account a stateof-the-art review for both approaches, their practical applications, trends, and future challenges.This paper shows that both approaches are suitable for solving practical problems, but experts in machine learning need to understand the input data, the problem to solve, and the best way for showing the output data before applying a machine learning model.Also, we propose some ideas for fusing both, explainable and blackbox, approaches to provide better solutions to experts in real-world domains.Additionally, we show one way to measure the effectiveness of the applied machine learning model by using expert opinions jointly with statistical methods.Throughout this paper, we show the impact of using explainable and black-box models on the security and medical applications.
Deep learning-based image recognition for autonomous driving

Various image recognition tasks were handled in the image recognition field prior to 2010 by combining image local features manually designed by researchers (called handcrafted features) and machine learning method. After entering the 2010, However, many image recognition methods that use deep learning have been proposed. The image recognition methods using deep learning are far superior to the methods used prior to the appearance of deep learning in general object recognition competitions. Hence, this paper will explain how deep learning is applied to the field of image recognition, and will also explain the latest trends of deep learning-based autonomous driving.
Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study

BackgroundMammography is the current standard for breast cancer screening. This study aimed to develop an artificial intelligence (AI) algorithm for diagnosis of breast cancer in mammography, and explore whether it could benefit radiologists by improving accuracy of diagnosis.MethodsIn this retrospective study, an AI algorithm was developed and validated with 170 230 mammography examinations collected from five institutions in South Korea, the USA, and the UK, including 36 468 cancer positive confirmed by biopsy, 59 544 benign confirmed by biopsy (8827 mammograms) or follow-up imaging (50 717 mammograms), and 74 218 normal. For the multicentre, observer-blinded, reader study, 320 mammograms (160 cancer positive, 64 benign, 96 normal) were independently obtained from two institutions. 14 radiologists participated as readers and assessed each mammogram in terms of likelihood of malignancy (LOM), location of malignancy, and necessity to recall the patient, first without and then with assistance of the AI algorithm. The performance of AI and radiologists was evaluated in terms of LOM-based area under the receiver operating characteristic curve (AUROC) and recall-based sensitivity and specificity.FindingsThe AI standalone performance was AUROC 0·959 (95% CI 0·952–0·966) overall, and 0·970 (0·963–0·978) in the South Korea dataset, 0·953 (0·938–0·968) in the USA dataset, and 0·938 (0·918–0·958) in the UK dataset. In the reader study, the performance level of AI was 0·940 (0·915–0·965), significantly higher than that of the radiologists without AI assistance (0·810, 95% CI 0·770–0·850; p<0·0001). With the assistance of AI, radiologists' performance was improved to 0·881 (0·850–0·911; p<0·0001). AI was more sensitive to detect cancers with mass (53 [90%] vs 46 [78%] of 59 cancers detected; p=0·044) or distortion or asymmetry (18 [90%] vs ten [50%] of 20 cancers detected; p=0·023) than radiologists. AI was better in detection of T1 cancers (73 [91%] vs 59 [74%] of 80; p=0·0039) or node-negative cancers (104 [87%] vs 88 [74%] of 119; p=0·0025) than radiologists.InterpretationThe AI algorithm developed with large-scale mammography data showed better diagnostic performance in breast cancer detection compared with radiologists. The significant improvement in radiologists' performance when aided by AI supports application of AI to mammograms as a diagnostic support tool.FundingLunit.
Pelican Optimization Algorithm: A Novel Nature-Inspired Algorithm for Engineering Applications

Optimization is an important and fundamental challenge to solve optimization problems in different scientific disciplines. In this paper, a new stochastic nature-inspired optimization algorithm called Pelican Optimization Algorithm (POA) is introduced. The main idea in designing the proposed POA is simulation of the natural behavior of pelicans during hunting. In POA, search agents are pelicans that search for food sources. The mathematical model of the POA is presented for use in solving optimization issues. The performance of POA is evaluated on twenty-three objective functions of different unimodal and multimodal types. The optimization results of unimodal functions show the high exploitation ability of POA to approach the optimal solution while the optimization results of multimodal functions indicate the high ability of POA exploration to find the main optimal area of the search space. Moreover, four engineering design issues are employed for estimating the efficacy of the POA in optimizing real-world applications. The findings of POA are compared with eight well-known metaheuristic algorithms to assess its competence in optimization. The simulation results and their analysis show that POA has a better and more competitive performance via striking a proportional balance between exploration and exploitation compared to eight competitor algorithms in providing optimal solutions for optimization problems.
Blockchain-enabled supply chain: An experimental study

Despite Information and Communication Technologies (ICT) have reduced the information asymmetry and increased the degree of interorganizational collaboration, the companies participating a supply chain are less inclined to share data when information is sensible and partners cannot be fully trusted. In such a context, Blockchain is a decentralized certificate authority that may provide economic and operational benefits but companies operating in a supply chain claim to have little knowledge about Blockchain due to its novelty and to the lack of use cases and application studies. In this work, a software connector has been designed and developed to connect an Ethereum-like blockchain with the enterprises' information systems to allow companies to share information with their partners with different levels of visibility and to check data authenticity, integrity and invariability over time through the blockchain, thus building trust. In order to explore the potential of deploying the blockchain in a supply chain, a simulation model has been developed to recreate the supply chain operations and integrated with the blockchain through the same software connector to carry out a scenario statistical analysis. Application results shows how blockchain technology is a convenient instrument to overcome collaboration and trust issues in a supply chain, to increase the supply chain overall performance, to minimize the negative consequences of information asymmetry over the echelons of a supply chain but also to discourage companies from any misconduct (e.g. counterfeiting data or low data accuracy).
3D Hand Shape and Pose From Images in the Wild

We present in this work the first end-to-end deep learning based method that predicts both 3D hand shape and pose from RGB images in the wild. Our network consists of the concatenation of a deep convolutional encoder, and a fixed model-based decoder. Given an input image, and optionally 2D joint detections obtained from an independent CNN, the encoder predicts a set of hand and view parameters. The decoder has two components: A pre-computed articulated mesh deformation hand model that generates a 3D mesh from the hand parameters, and a re-projection module controlled by the view parameters that projects the generated hand into the image domain. We show that using the shape and pose prior knowledge encoded in the hand model within a deep learning framework yields state-of-the-art performance in 3D pose prediction from images on standard benchmarks, and produces geometrically valid and plausible 3D reconstructions. Additionally, we show that training with weak supervision in the form of 2D joint annotations on datasets of images in the wild, in conjunction with full supervision in the form of 3D joint annotations on limited available datasets allows for good generalization to 3D shape and pose predictions on images in the wild.
Blockchain critical success factors for sustainable supply chain

Way back the development of Blockchain (BC) technology, Traditional supply chain (TSC) failed to meet the demand of customer of reasonable price with high quality. The current trend is the time to integrate and to make the existing system smarter, automated and sustainable. Researchers are working on the integration of information and communication technology with the supply chain (SC) and continuously endeavor to achieve an efficient SC. This paper is an attempt to study the use of BC technology and a try to develop efficient sustainable supply chain management (SSCM) rather than the inefficient design of supply chain management (SCM). Important variables related to BC are identified from the literature after going through discussion with academician and industry experts. These variables are further analyzed and modeled using Principal Component Analysis (PCA), Fuzzy-Decision making trial and evaluation laboratory (DEMATEL). PCA is applied to form the principal factor from these identified variables based on possible correlation. Although, the application of Fuzzy-DEMATEL is adopted to identify major causes that help to achieve sustainable supply chain (SSC) after integration of BC technology. The proposed integrated (incorporated) approach of PCA and Fuzzy-DEMATEL found six major causes namely Data safety and Decentralization, Accessibility, Laws and Policy, Documentation, Data management, and Quality. This study is revolving around the integrated approach of two streams namely BC-related Information technology and SC from Operations. This research represents the significant role of causes, which leads to the integration of BC with the SC resulting in achieving sustainability. BC technology is still in their nuance stage and this study will motivate researcher and industrial practitioners to achieve a more efficient and effective goal in SC practices to achieve sustainability.
Machine Learning Adoption in Blockchain-Based Smart Applications: The Challenges, and a Way Forward

In recent years, the emergence of blockchain technology (BT) has become a unique, most disruptive, and trending technology. The decentralized database in BT emphasizes data security and privacy. Also, the consensus mechanism in it makes sure that data is secured and legitimate. Still, it raises new security issues such as majority attack and double-spending. To handle the aforementioned issues, data analytics is required on blockchain based secure data. Analytics on these data raises the importance of arisen technology Machine Learning (ML). ML involves the rational amount of data to make precise decisions. Data reliability and its sharing are very crucial in ML to improve the accuracy of results. The combination of these two technologies (ML and BT) can provide highly precise results. In this paper, we present a detailed study on ML adoption for making BT-based smart applications more resilient against attacks. There are various traditional ML techniques, for instance, Support Vector Machines (SVM), clustering, bagging, and Deep Learning (DL) algorithms such as Convolutional Neural Network (CNN) and Long short-term memory (LSTM) can be used to analyse the attacks on a blockchain-based network. Further, we include how both the technologies can be applied in several smart applications such as Unmanned Aerial Vehicle (UAV), Smart Grid (SG), healthcare, and smart cities. Then, future research issues and challenges are explored. At last, a case study is presented with a conclusion.
The economic impact of universities: Evidence from across the globe

We develop a new dataset using UNESCO source materials on the location of nearly 15,000 universities in about 1,500 regions across 78 countries, some dating back to the 11th Century. We estimate fixed effects models at the sub-national level between 1950 and 2010 and find that increases in the number of universities are positively associated with future growth of GDP per capita (and this relationship is robust to controlling for a host of observables, as well as unobserved regional trends). Our estimates imply that a 10% increase in a region's number of universities per capita is associated with 0.4% higher future GDP per capita in that region. Furthermore, there appear to be positive spillover effects from universities to geographically close neighbouring regions. We show that the relationship between GDP per capita and universities is not simply driven by the direct expenditures of the university, its staff and students. Part of the effect of universities on growth is mediated through an increased supply of human capital and greater innovation. Furthermore, we find that within countries, higher historical university presence is associated with stronger pro-democratic attitudes.