Miracle-dz/UniFAR-FTUs huggingface dataset:
Soft-CP: A Credible and Effective Data Augmentation for Semantic Segmentation of Medical Lesions

The medical datasets are usually faced with the problem of scarcity and data imbalance. Moreover, annotating large datasets for semantic segmentation of medical lesions is domain-knowledge and time-consuming. In this paper, we propose a new object-blend method(short in soft-CP) that combines the Copy-Paste augmentation method for semantic segmentation of medical lesions offline, ensuring the correct edge information around the lession to solve the issue above-mentioned. We proved the method's validity with several datasets in different imaging modalities. In our experiments on the KiTS19[2] dataset, Soft-CP outperforms existing medical lesions synthesis approaches. The Soft-CP augementation provides gains of +26.5% DSC in the low data regime(10% of data) and +10.2% DSC in the high data regime(all of data), In offline training data, the ratio of real images to synthetic images is 3:1.
Unsupervised Heterophilous Network Embedding via r-Ego Network Discrimination

—Recently, supervised network embedding (NE) has emerged as a predominant technique for representing complex systems that take the form of networks, and various downstream node-and network-level tasks have beneﬁted from its remarkable developments. However, unsupervised NE still remains challenging due to the uncertainty in deﬁning a learning objective. In addition, it is still an unexplored research question whether existing NE methods adapt well to heterophilous networks . This paper introduces the ﬁrst empirical study on the inﬂuence of homophily ratio on the performance of existing unsupervised NE methods and reveals their limitations. Inspired by our empirical ﬁndings, we design unsupervised NE task as an r -ego network discrimination problem and further develop a SELf-supErvised Network Embedding (Selene) framework for learning useful node representations for both homophilous and heterophilous networks. Speciﬁcally, we propose a dual-channel feature embedding mechanism to fuse node attributes and network structure information and leverage a sampling and anonymisation strategy to break the implicit homophily assumption of existing embedding mechanisms. Lastly, we introduce a negative-sample-free SSL objective function to optimise the framework. We conduct extensive experiments and a series of ablation studies on 12 real-world datasets and 20 synthetic networks. Results demonstrate Selene’s superior performance and conﬁrm the effectiveness of each component. Code and data are available at https://github.com/zhiqiangzhongddu/Selene.
Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild

We propose a trainable Image Signal Processing (ISP) framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image. During inference, we predict the target color image by designing a color prediction network with efficient Global Context Transformer modules. The latter effectively leverage global information to learn consistent color and tone mappings. We further propose a robust masked aligned loss to identify and discard regions with inaccurate motion estimation during training. Lastly, we introduce the ISP in the Wild (ISPW) dataset, consisting of weakly paired phone RAW and DSLR sRGB images. We extensively evaluate our method, setting a new state-of-the-art on two datasets.
When regression coefficients change over time: A proposal

A common approach in forecasting problems is to estimate a least-squares regression (or other statistical learning models) from past data, which is then applied to predict future outcomes. An underlying assumption is that the same correlations that were observed in the past still hold for the future. We propose a model for situations when this assumption is not met: adopting methods from the state space literature, we model how regression coefficients change over time. Our approach can shed light on the large uncertainties associated with forecasting the future, and how much of this is due to changing dynamics of the past. Our simulation study shows that accurate estimates are obtained when the outcome is continuous, but the procedure fails for binary outcomes.
Blockchain for the Metaverse: A Review

Since Facebook officially changed its name to Metaverse in Oct. 2021, the metaverse has become a new norm of social networks and three-dimensional (3D) virtual worlds. The metaverse aims to bring 3D immersive and personalized experiences to users by leveraging many pertinent technologies. Despite great attention and benefits, a natural question in the metaverse is how to secure its users' digital content and data. In this regard, blockchain is a promising solution owing to its distinct features of decentralization, immutability, and transparency. To better understand the role of blockchain in the metaverse, we aim to provide an extensive survey on the applications of blockchain for the metaverse. We first present a preliminary to blockchain and the metaverse and highlight the motivations behind the use of blockchain for the metaverse. Next, we extensively discuss blockchain-based methods for the metaverse from technical perspectives, such as data acquisition, data storage, data sharing, data interoperability, and data privacy preservation. For each perspective, we first discuss the technical challenges of the metaverse and then highlight how blockchain can help. Moreover, we investigate the impact of blockchain on key-enabling technologies in the metaverse, including Internet-of-Things, digital twins, multi-sensory and immersive applications, artificial intelligence, and big data. We also present some major projects to showcase the role of blockchain in metaverse applications and services. Finally, we present some promising directions to drive further research innovations and developments towards the use of blockchain in the metaverse in the future.
ImageNet as a Representative Basis for Deriving Generally Effective CNN Architectures

Lukas Tuggener1,2, Jürgen Schmidhuber2,3,4, Thilo Stadelmann1,5 1ZHAW Centre for Artificial Intelligence, Winterthur, Switzerland, 2University of Lugano, Switzerland, 3The Swiss AI Lab IDSIA, Switzerland, 4 AI Initiative, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia, 5Fellow of the European Centre for Living Technology, Venice, Italy {tugg,stdm}@zhaw.ch, juergen@idsia.ch
Sample and Computationally Efficient Stochastic Kriging in High Dimensions

High-dimensional Simulation Metamodeling Stochastic kriging has been widely employed for simulation metamodeling to predict the response surface of complex simulation models. However, its use is limited to cases where the design space is low-dimensional because the sample complexity (i.e., the number of design points required to produce an accurate prediction) grows exponentially in the dimensionality of the design space. The large sample size results in both a prohibitive sample cost for running the simulation model and a severe computational challenge due to the need to invert large covariance matrices. To address this long-standing challenge, Liang Ding and Xiaowei Zhang, in their recent paper “Sample and Computationally Efficient Stochastic Kriging in High Dimensions”, develop a novel methodology — based on tensor Markov kernels and sparse grid experimental designs — that dramatically alleviates the curse of dimensionality. The proposed methodology has theoretical guarantees on both sample complexity and computational complexity and shows outstanding performance in numerical problems of as high as 16,675 dimensions.
A naive method to discover directions in the StyleGAN2 latent space

Several research groups have shown that Generative Adversarial Networks (GANs) can generate photo-realistic images in recent years. Using the GANs, a map is created between a latent code and a photo-realistic image. This process can also be reversed: given a photo as input, it is possible to obtain the corresponding latent code. In this paper, we will show how the inversion process can be easily exploited to interpret the latent space and control the output of StyleGAN2, a GAN architecture capable of generating photo-realistic faces. From a biological perspective, facial features such as nose size depend on important genetic factors, and we explore the latent spaces that correspond to such biological features, including masculinity and eye colour. We show the results obtained by applying the proposed method to a set of photos extracted from the CelebA-HQ database. We quantify some of these measures by utilizing two landmarking protocols, and evaluate their robustness through statistical analysis. Finally we correlate these measures with the input parameters used to perturb the latent spaces along those interpretable directions. Our results contribute towards building the groundwork of using such GAN architecture in forensics to generate photo-realistic faces that satisfy certain biological attributes.
Iwin: Human-Object Interaction Detection via Transformer with Irregular Windows

This paper presents a new vision Transformer, named Iwin Transformer, which is specifically designed for human-object interaction (HOI) detection, a detailed scene understanding task involving a sequential process of human/object detection and interaction recognition. Iwin Transformer is a hierarchical Transformer which progressively performs token representation learning and token agglomeration within irregular windows. The irregular windows, achieved by augmenting regular grid locations with learned offsets, 1) eliminate redundancy in token representation learning, which leads to efficient human/object detection, and 2) enable the agglomerated tokens to align with humans/objects with different shapes, which facilitates the acquisition of highly-abstracted visual semantics for interaction recognition. The effectiveness and efficiency of Iwin Transformer are verified on the two standard HOI detection benchmark datasets, HICO-DET and V-COCO. Results show our method outperforms existing Transformers-based methods by large margins (3.7 mAP gain on HICO-DET and 2.0 mAP gain on V-COCO) with fewer training epochs ($0.5 \times$).
Explicit User Manipulation in Reinforcement Learning Based Recommender Systems

Recommender systems are highly prevalent in the modern world due to their value to both users and platforms and services that employ them. Generally, they can improve the user experience and help to increase satisfaction, but they do not come without risks. One such risk is that of their effect on users and their ability to play an active role in shaping user preferences. This risk is more significant for reinforcement learning based recommender systems. These are capable of learning for instance, how recommended content shown to a user today may tamper that user's preference for other content recommended in the future. Reinforcement learning based recommendation systems can thus implicitly learn to influence users if that means maximizing clicks, engagement, or consumption. On social news and media platforms, in particular, this type of behavior is cause for alarm. Social media undoubtedly plays a role in public opinion and has been shown to be a contributing factor to increased political polarization. Recommender systems on such platforms, therefore, have great potential to influence users in undesirable ways. However, it may also be possible for this form of manipulation to be used intentionally. With advancements in political opinion dynamics modeling and larger collections of user data, explicit user manipulation in which the beliefs and opinions of users are tailored towards a certain end emerges as a significant concern in reinforcement learning based recommender systems.
Underwater Light Field Retention : Neural Rendering for Underwater Imaging

Underwater Image Rendering aims to generate a true-to-life underwater image from a given clean one, which could be applied to various practical applications such as underwater image enhancement, camera filter, and virtual gaming. We explore two less-touched but challenging problems in underwater image rendering, namely, i) how to render diverse underwater scenes by a single neural network? ii) how to adaptively learn the underwater light fields from natural exemplars, i,e. , realistic underwater images? To this end, we propose a neural rendering method for underwater imaging, dubbed UWNR (Underwater Neural Rendering). Specifically, UWNR is a data-driven neural network that implicitly learns the natural degenerated model from authentic underwater images, avoiding introducing erroneous biases by hand-craft imaging models.Compared with existing underwater image generation methods, UWNR utilizes the natural light field to simulate the main characteristics of the underwater scene. Thus, it is able to synthesize a wide variety of underwater images from one clean image with various realistic underwater images.Extensive experiments demonstrate that our approach achieves better visual effects and quantitative metrics over previous methods. Moreover, we adopt UWNR to build an open Large Neural Rendering Underwater Dataset containing various types of water quality, dubbed LNRUD. The source code and LNRUD are available at https://github.com/Ephemeral182/UWNR.
Bike Sharing Demand Prediction based on Knowledge Sharing across Modes: A Graph-based Deep Learning Approach

Bike sharing is an increasingly popular part of urban transportation systems. Accurate demand prediction is the key to support timely re-balancing and ensure service efficiency. Most existing models of bike-sharing demand prediction are solely based on its own historical demand variation, essentially regarding bike sharing as a closed system and neglecting the interaction between different transport modes. This is particularly important because bike sharing is often used to complement travel through other modes (e.g., public transit). Despite some recent efforts, there is no existing method capable of leveraging spatiotemporal information from multiple modes with heterogeneous spatial units. To address this research gap, this study proposes a graph-based deep learning approach for bike sharing demand prediction (B-MRGNN) with multimodal historical data as input. The spatial dependencies across modes are encoded with multiple intra- and inter-modal graphs. A multi-relational graph neural network (MRGNN) is introduced to capture correlations between spatial units across modes, such as bike sharing stations, subway stations, or ride-hailing zones. Extensive experiments are conducted using real-world bike sharing, subway and ride-hailing data from New York City, and the results demonstrate the superior performance of our proposed approach compared to existing methods.
TreatmentEstimatoR: a Dashboard for Estimating Treatment Effects from Observational Health Data

Observational health data can be leveraged to measure the real-world use and potential benefits or risks of existing medical interventions. However, lack of programming proficiency and advanced knowledge of causal inference methods excludes some clinicians and non-computational researchers from performing such analyses. Code-free dashboard tools provide accessible means to estimate and visualize treatment effects from observational health data. We present TreatmentEstimatoR, an R Shiny dashboard that facilitates the estimation of treatment effects from observational data without any programming knowledge required. The dashboard provides effect estimates from multiple algorithms simultaneously and accommodates binary, continuous, and time-to-event outcomes. TreatmentEstimatoR allows for flexible covariate selection for treatment and outcome models, comprehensive model performance metrics, and an exploratory data analysis tool. TreatmentEstimatoR is available at https://github.com/CollinSakal/TreatmentEstimatoR. We provide full installation instructions and detailed vignettes for how to best use the dashboard.
Image Classification on Accelerated Neural Networks

For image classification problems, various neural network models are commonly used due to their success in yielding high accuracies. Convolutional Neural Network (CNN) is one of the most frequently used deep learning methods for image classification applications. It may produce extraordinarily accurate results with regard to its complexity. However, the more complex the model is the longer it takes to train. In this paper, an acceleration design that uses the power of FPGA is given for a basic CNN model which consists of one convolutional layer and one fully connected layer for the training phase of the fully connected layer. Nonetheless, inference phase is also accelerated automatically due to the fact that training phase includes inference. In this design, the convolutional layer is calculated by the host computer and the fully connected layer is calculated by an FPGA board. It should be noted that the training of convolutional layer is not taken into account in this design and is left for future research. The results are quite encouraging as this FPGA design tops the performance of some of the state-of-the-art deep learning platforms such as Tensorflow on the host computer approximately 2 times in both training and inference.
Coordinated Pose Control of Mobile Manipulation With an Unstable Bikebot Platform

Bikebot manipulation has advantages of the single-track robot mobility and manipulation dexterity. We present a coordinated pose control of mobile manipulation with the stationary bikebot. The challenges of the bikebot manipulation include the limited steering balance capability of the unstable bikebot and kinematic redundancy of the manipulator. We first present the steering balance model to analyze and explore the maximum steering capability to balance the stationary platform. A balancing equilibrium manifold is then proposed to describe the necessary condition to fulfill simultaneous platform balance and posture control of the end-effector. A coordinated planning and control design is presented to determine the balance-prioritized posture control under kinematic and dynamic constraints. Extensive experiments are conducted to demonstrate the mechatronic design for autonomous plant inspection in agricultural applications. The results confirm the feasibility to use the bikebot manipulation for plant inspection with end-effector position and orientation errors about 5 mm and 0.3 degs, respectively.
Point3D: tracking actions as moving points with 3D CNNs

Spatio-temporal action recognition has been a challenging task that involves detecting where and when actions occur. Current state-of-the-art action detectors are mostly anchor-based, requiring sensitive anchor designs and huge computations due to calculating large numbers of anchor boxes. Motivated by nascent anchor-free approaches, we propose Point3D, a flexible and computationally efficient network with high precision for spatio-temporal action recognition. Our Point3D consists of a Point Head for action localization and a 3D Head for action classification. Firstly, Point Head is used to track center points and knot key points of humans to localize the bounding box of an action. These location features are then piped into a time-wise attention to learn long-range dependencies across frames. The 3D Head is later deployed for the final action classification. Our Point3D achieves state-of-the-art performance on the JHMDB, UCF101-24, and AVA benchmarks in terms of frame-mAP and video-mAP. Comprehensive ablation studies also demonstrate the effectiveness of each module proposed in our Point3D.
Laneformer: Object-aware Row-Column Transformers for Lane Detection

We present Laneformer, a conceptually simple yet powerful transformer-based architecture tailored for lane detection that is a long-standing research topic for visual perception in autonomous driving. The dominant paradigms rely on purely CNN-based architectures which often fail in incorporating relations of long-range lane points and global contexts induced by surrounding objects (e.g., pedestrians, vehicles). Inspired by recent advances of the transformer encoder-decoder architecture in various vision tasks, we move forwards to design a new end-to-end Laneformer architecture that revolutionizes the conventional transformers into better capturing the shape and semantic characteristics of lanes, with minimal overhead in latency. First, coupling with deformable pixel-wise self-attention in the encoder, Laneformer presents two new row and column self-attention operations to efficiently mine point context along with the lane shapes. Second, motivated by the appearing objects would affect the decision of predicting lane segments, Laneformer further includes the detected object instances as extra inputs of multi-head attention blocks in the encoder and decoder to facilitate the lane point detection by sensing semantic contexts. Specifically, the bounding box locations of objects are added into Key module to provide interaction with each pixel and query while the ROI-aligned features are inserted into Value module. Extensive experiments demonstrate our Laneformer achieves state-of-the-art performances on CULane benchmark, in terms of 77.1% F1 score. We hope our simple and effective Laneformer will serve as a strong baseline for future research in self-attention models for lane detection.
The Main Sequence of star-forming galaxies across cosmic times

By compiling a comprehensive census of literature studies, we investigate the evolution of the Main Sequence (MS) of star-forming galaxies (SFGs) in the widest range of redshift (0 < 𝑧 < 6) and stellar mass (10 8 . 5 − 10 11 . 5 𝑀 (cid:12) ) ever probed. We convert all observations to a common calibration and ﬁnd a remarkable consensus on the variation of the MS shape and normalization across cosmic time. The relation exhibits a curvature towards the high stellar masses at all redshifts. The best functional form is governed by two parameters: the evolution of the normalization and the turnover mass ( 𝑀 0 ( 𝑡 ) ), which both evolve as a power law of the Universe age. The turn-over mass determines the MS shape. It marginally evolves with time, making the MS slightly steeper towards 𝑧 ∼ 4 − 6. At stellar masses below 𝑀 0 ( 𝑡 ) , SFGs have a constant speciﬁc SFR (sSFR), while above 𝑀 0 ( 𝑡 ) the sSFR is suppressed. We ﬁnd that the MS is dominated by central galaxies. This allows to turn 𝑀 0 ( 𝑡 ) into the corresponding host halo mass. This evolves as the halo mass threshold between cold and hot accretion regimes, as predicted by the theory of accretion, where the central galaxy is fed or starved of cold gas supply, respectively. We, thus, argue that the progressive MS bending as a function of the Universe age is caused by the lower availability of cold gas in halos entering the hot accretion phase, in addition to black hole feedback. We also ﬁnd qualitatively the same trend in the largest sample of star-forming galaxies provided by the IllustrisTNG simulation. Nevertheless, we still note large quantitative discrepancies with respect to observations, in particular at the high mass end. These can not be easily ascribed to biases or systematics in the observed SFRs and the derived MS.