Miracle-dz/UniFAR-FTUs huggingface dataset:
CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data

Contrastive Language-Image Pre-training, benefiting from large-scale unlabeled text-image pairs, has demonstrated great performance in open-world vision understanding tasks. However, due to the limited Text-3D data pairs, adapting the success of 2D Vision-Language Models (VLM) to the 3D space remains an open problem. Existing works that leverage VLM for 3D understanding generally resort to constructing intermediate 2D representations for the 3D data, but at the cost of losing 3D geometry information. To take a step toward open-world 3D vision understanding, we propose Contrastive Language-Image-Point Cloud Pretraining (CLIP2) to directly learn the transferable 3D point cloud representation in realistic scenarios with a novel proxy alignment mechanism. Specifically, we exploit naturally-existed correspondences in 2D and 3D scenarios, and build well-aligned and instance-based text-image-point proxies from those complex scenarios. On top of that, we propose a cross-modal contrastive objective to learn semantic and instance-level aligned point cloud representation. Experimental results on both indoor and outdoor scenarios show that our learned 3D representation has great transfer ability in downstream tasks, including zero-shot and few-shot 3D recognition, which boosts the state-of-the-art methods by large margins. Furthermore, we provide analyses of the capability of different representations in real scenarios and present the optional ensemble scheme.
Convenient Calibration Procedure for Structured Light Projection System

We present a convenient calibration method for structured light projection system. The proposed clibration approach can realize 3D shape measurement without projector calibration, without system calibration, without precise linear z stage to be used, the relative position between camera and projector can be arbitrary, and the only involved device is a plane board. Experiment results validated that the accuracy of the proposed approach.
Multistable topological edge states in double-wave chains of polariton condensates

Topological edge states have been widely investigated in diﬀerent types of lattices. In the present work, we report on topological edge states in double-wave potential chains, which can be described by a generalized Aubry-Andr´e-Harper (AAH) model, applied to a driven-dissipative exciton polariton system. We show that in such potential chains, diﬀerent types of edge states can form. For resonant optical excitation, the nonlinearity resulting from the repulsive polariton-polariton interaction leads to a multistability of diﬀerent edge states that are stabilized for the same set of system and excitation parameters. This includes topologically protected edge states evolved directly from the individual linear eigenmodes of the system as well as additional edge states that originate from the localization of bulk states in the presence of nonlinearity. Generally, the interplay of topological localization and nonlinearity opens up exciting prospects for functional topological structures.
Text Semantics to Image Generation: A method of building facades design base on Stable Diffusion model

Stable Diffusion model has been extensively employed in the study of archi-tectural image generation, but there is still an opportunity to enhance in terms of the controllability of the generated image content. A multi-network combined text-to-building facade image generating method is proposed in this work. We first fine-tuned the Stable Diffusion model on the CMP Fa-cades dataset using the LoRA (Low-Rank Adaptation) approach, then we ap-ply the ControlNet model to further control the output. Finally, we contrast-ed the facade generating outcomes under various architectural style text con-tents and control strategies. The results demonstrate that the LoRA training approach significantly decreases the possibility of fine-tuning the Stable Dif-fusion large model, and the addition of the ControlNet model increases the controllability of the creation of text to building facade images. This pro-vides a foundation for subsequent studies on the generation of architectural images.
Conditional Instability and Shear for Six Hurricanes over the Atlantic Ocean

Abstract One hundred and thirty Omega dropwindsondes deployed within 500-km radius of the eye of six North Atlantic hurricanes are used to determine the magnitudes and trends in convective available potential energy, and 10–1500-m and 0–6-km shear of the horizontal wind as a function of radius, quadrant, and hurricane intensity. The moist convective instability found at large radii (400–500 km) decreases to near neutral stability by 75 km from the eyewall. Vertical shears increase as radius decreases, but maximum shear values are only one-half of those found over land. Scatter for both the conditional instability and the shear is influenced chiefly by hurricane intensity, but proximity to reflectivity features does modulate the pattern. The ratio of the conditional instability to the shear (bulk Richardson number) indicates that supercell formation is favored within 250 km of the circulation center, but helicity values are below the threshold to support strong waterspouts. The difference between these oce.
flippy: User friendly and open source framework for lipid membrane simulations

Animal cells are both encapsulated and subdivided by lipid bilayer membranes. Beyond just acting as boundaries, these membranes' shapes influence the function of cells and their compartments. Physically, membranes are two-dimensional fluids with complex elastic behavior, which makes it impossible, for all but a few simple cases, to predict membrane shapes analytically. Instead, the shape and behavior of biological membranes can be determined by simulations. However, the setup and use of such simulations require a significant programming background. The availability of open-source and user-friendly packages for simulating biological membranes needs improvement.Here, we present flippy, an open-source package for simulating lipid membrane shapes, their interaction with proteins or external particles, and the effect of external forces. Our goal is to provide a tool that is easy to use without sacrificing performance or versatility. flippy is an implementation of a dynamically triangulated membrane. We use a precise yet fast algorithm for calculating the geometric properties of membranes and can also account for local spontaneous curvature, a feature not all discretizations allow. Finally, in flippy we can also include regions of purely elastic (non-fluid) membranes and thus explore various shapes encountered in living systems.
Reducing Air Pollution through Machine Learning

This paper presents a data-driven approach to mitigate the effects of air pollution from industrial plants on nearby cities by linking operational decisions with weather conditions. Our method combines predictive and prescriptive machine learning models to forecast short-term wind speed and direction and recommend operational decisions to reduce or pause the industrial plant's production. We exhibit several trade-offs between reducing environmental impact and maintaining production activities. The predictive component of our framework employs various machine learning models, such as gradient-boosted tree-based models and ensemble methods, for time series forecasting. The prescriptive component utilizes interpretable optimal policy trees to propose multiple trade-offs, such as reducing dangerous emissions by 33-47% and unnecessary costs by 40-63%. Our deployed models significantly reduced forecasting errors, with a range of 38-52% for less than 12-hour lead time and 14-46% for 12 to 48-hour lead time compared to official weather forecasts. We have successfully implemented the predictive component at the OCP Safi site, which is Morocco's largest chemical industrial plant, and are currently in the process of deploying the prescriptive component. Our framework enables sustainable industrial development by eliminating the pollution-industrial activity trade-off through data-driven weather-based operational decisions, significantly enhancing factory optimization and sustainability. This modernizes factory planning and resource allocation while maintaining environmental compliance. The predictive component has boosted production efficiency, leading to cost savings and reduced environmental impact by minimizing air pollution.
Lack of scaling in global climate models

Detrended fluctuation analysis is used to test the performance of global climate models. We study the temperature data simulated by seven leading models for the greenhouse gas forcing only (GGFO) scenario and test their ability to reproduce the universal scaling (persistence) law found in the real records for four sites on the globe: (i) New York, (ii) Brookings, (iii) Tashkent and (iv) Saint Petersburg. We find that the models perform quite differently for the four sites and the data simulated by the models lack the universal persistence found in the observed data. We also compare the scaling behaviour of this scenario with that of the control run where the CO2 concentration is kept constant. Surprisingly, from the scaling point of view, the simple control run performs better than the more sophisticated GGFO scenario. This comparison indicates that the variation of the greenhouse gases affects not only trends but also fluctuations.
Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed Computational Miniature Mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model’s generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.
On a barrier height problem for RNA branching

The branching of an RNA molecule is an important structural characteristic yet difficult to predict correctly, especially for longer sequences. Using plane trees as a combinatorial model for RNA folding, we consider the thermodynamic cost, known as the barrier height, of transitioning between branching configurations. Using branching skew as a coarse energy approximation, we characterize various types of paths in the discrete configuration landscape. In particular, we give sufficient conditions for a path to have both minimal length and minimal branching skew. The proofs offer some biological insights, notably the potential importance of both hairpin stability and domain architecture to higher resolution RNA barrier height analyses.
Attenuating surface gravity waves by an array of submerged resonators: an experimental study

We report on an experimental study of a device composed by an array of submerged, reversed and periodic cylindrical pendula (resonators), whose objective is the attenuation of surface gravity waves. The idea is inspired by the concept of metamaterials, i.e. engineered structures designed to interact with waves and manipulate their propagation properties. The study is performed in a wave flume where single frequency waves are excited in a wide range of frequencies. We explore various configurations of the device, measuring the transmitted, reflected and dissipated energy of the waves. If the incoming wave frequencies are sufficiently close to the natural frequency of the pendula, we find a considerable wave attenuation effect, driven by viscous dissipative mechanisms. This behaviour is enhanced by the number of resonators in the array. Moreover, the device is also capable of reflecting the energy of selected frequencies of the incoming waves. These frequencies can be predicted by assuming the interactions involving at least three wave modes, including higher harmonics, and are therefore associated with the distance between the resonators. The presented results show promise for the development of a environmentally sustainable device for mitigating waves in coastal zones.
AIIPot: Adaptive Intelligent-Interaction Honeypot for IoT Devices

The proliferation of the Internet of Things (IoT) has raised concerns about the security of connected devices. There is a need to develop suitable and cost-efficient methods to identify vulnerabilities in IoT devices to address them before attackers seize opportunities to compromise them. The deception technique is a prominent approach to improving the security posture of IoT systems. Honeypot is a popular deception technique that mimics interaction in real fashion and encourages unauthorised users (attackers) to launch attacks. Due to the large number and the heterogeneity of IoT devices, manually crafting the low and high-interaction honeypots is not affordable. This has forced researchers to seek innovative ways to build honeypots for IoT devices. In this paper, we propose a honeypot for IoT devices that uses machine learning techniques to learn and interact with attackers automatically. The evaluation of the proposed model indicates that our system can improve the session length with attackers and capture more attacks on the IoT network.
Simulation Environment with Customized RISC-V Instructions for Logic-in-Memory Architectures

Nowadays, various memory-hungry applications like machine learning algorithms are knocking"the memory wall". Toward this, emerging memories featuring computational capacity are foreseen as a promising solution that performs data process inside the memory itself, so-called computation-in-memory, while eliminating the need for costly data movement. Recent research shows that utilizing the custom extension of RISC-V instruction set architecture to support computation-in-memory operations is effective. To evaluate the applicability of such methods further, this work enhances the standard GNU binary utilities to generate RISC-V executables with Logic-in-Memory (LiM) operations and develop a new gem5 simulation environment, which simulates the entire system (CPU, peripherals, etc.) in a cycle-accurate manner together with a user-defined LiM module integrated into the system. This work provides a modular testbed for the research community to evaluate potential LiM solutions and co-designs between hardware and software.
LFM-3D: Learnable Feature Matching Across Wide Baselines Using 3D Signals

Finding localized correspondences across different images of the same object is crucial to understand its geometry. In recent years, this problem has seen remarkable progress with the advent of deep learning-based local image features and learnable matchers. Still, learnable matchers often underperform when there exists only small regions of co-visibility between image pairs (i.e. wide camera baselines). To address this problem, we leverage recent progress in coarse single-view geometry estimation methods. We propose LFM-3D, a Learnable Feature Matching framework that uses models based on graph neural networks and enhances their capabilities by integrating noisy, estimated 3D signals to boost correspondence estimation. When integrating 3D signals into the matcher model, we show that a suitable positional encoding is critical to effectively make use of the low-dimensional 3D information. We experiment with two different 3D signals - normalized object coordinates and monocular depth estimates - and evaluate our method on large-scale (synthetic and real) datasets containing object-centric image pairs across wide baselines. We observe strong feature matching improvements compared to 2D-only methods, with up to +6% total recall and +28% precision at fixed recall. Additionally, we demonstrate that the resulting improved correspondences lead to much higher relative posing accuracy for in-the-wild image pairs - up to 8.6% compared to the 2D-only approach.
Road Extraction With Satellite Images and Partial Road Maps

Road extraction is a process of automatically generating road maps mainly from satellite images. Existing models all target to generate roads from the scratch despite that a large number of road maps, though incomplete, are publicly available [e.g., those from OpenStreetMap (OSM)] and can help with road extraction. In this article, we propose to conduct road extraction based on satellite images and partial road maps, which is new. We then propose a two-branch partial-to-complete network (P2CNet) for the task, which has two prominent components: gated self-attention module (GSAM) and missing part (MP) loss. GSAM leverages a channelwise self-attention module and a gate module to capture long-range semantics, filter out useless information, and better fuse the features from two branches. An MP loss is derived from the partial road maps, trying to give more attention to the road pixels that do not exist in partial road maps. Extensive experiments are conducted to demonstrate the effectiveness of our model, e.g., P2CNet achieves the state-of-the-art performance with the intersection over union (IoU) scores of 70.71% and 75.52%, respectively, on the SpaceNet and OSM datasets.
Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints

Estimation problems with constrained parameter spaces arise in various settings. In many of these problems, the observations available to the statistician can be modelled as arising from the noisy realization of the image of a random linear operator; an important special case is random design regression. We derive sharp rates of estimation for arbitrary compact elliptical parameter sets and demonstrate how they depend on the distribution of the random linear operator. Our main result is a functional that characterizes the minimax rate of estimation in terms of the noise level, the law of the random operator, and elliptical norms that define the error metric and the parameter space. This nonasymptotic result is sharp up to an explicit universal constant, and it becomes asymptotically exact as the radius of the parameter space is allowed to grow. We demonstrate the generality of the result by applying it to both parametric and nonparametric regression problems, including those involving distribution shift or dependent covariates.
DR.CPO: Diversified and Realistic 3D Augmentation via Iterative Construction, Random Placement, and HPR Occlusion

In autonomous driving, data augmentation is commonly used for improving 3D object detection. The most basic methods include insertion of copied objects and rotation and scaling of the entire training frame. Numerous variants have been developed as well. The existing methods, however, are considerably limited when compared to the variety of the real world possibilities. In this work, we develop a diversified and realistic augmentation method that can flexibly construct a whole-body object, freely locate and rotate the object, and apply self-occlusion and external-occlusion accordingly. To improve the diversity of the whole-body object construction, we develop an iterative method that stochastically combines multiple objects observed from the real world into a single object. Unlike the existing augmentation methods, the constructed objects can be randomly located and rotated in the training frame because proper occlusions can be reflected to the whole-body objects in the final step. Finally, proper self-occlusion at each local object level and external-occlusion at the global frame level are applied using the Hidden Point Removal (HPR) algorithm that is computationally efficient. HPR is also used for adaptively controlling the point density of each object according to the object's distance from the LiDAR. Experiment results show that the proposed DR.CPO algorithm is data-efficient and model-agnostic without incurring any computational overhead. Also, DR.CPO can improve mAP performance by 2.08% when compared to the best 3D detection result known for KITTI dataset.
Deployment of Image Analysis Algorithms under Prevalence Shifts

Domain gaps are among the most relevant roadblocks in the clinical translation of machine learning (ML)-based solutions for medical image analysis. While current research focuses on new training paradigms and network architectures, little attention is given to the specific effect of prevalence shifts on an algorithm deployed in practice. Such discrepancies between class frequencies in the data used for a method's development/validation and that in its deployment environment(s) are of great importance, for example in the context of artificial intelligence (AI) democratization, as disease prevalences may vary widely across time and location. Our contribution is twofold. First, we empirically demonstrate the potentially severe consequences of missing prevalence handling by analyzing (i) the extent of miscalibration, (ii) the deviation of the decision threshold from the optimum, and (iii) the ability of validation metrics to reflect neural network performance on the deployment population as a function of the discrepancy between development and deployment prevalence. Second, we propose a workflow for prevalence-aware image classification that uses estimated deployment prevalences to adjust a trained classifier to a new environment, without requiring additional annotated deployment data. Comprehensive experiments based on a diverse set of 30 medical classification tasks showcase the benefit of the proposed workflow in generating better classifier decisions and more reliable performance estimates compared to current practice.
Posthoc Interpretation via Quantization

In this paper, we introduce a new approach, called Posthoc Interpretation via Quantization (PIQ), for interpreting decisions made by trained classifiers. Our method utilizes vector quantization to transform the representations of a classifier into a discrete, class-specific latent space. The class-specific codebooks act as a bottleneck that forces the interpreter to focus on the parts of the input data deemed relevant by the classifier for making a prediction. Our model formulation also enables learning concepts by incorporating the supervision of pretrained annotation models such as state-of-the-art image segmentation models. We evaluated our method through quantitative and qualitative studies involving black-and-white images, color images, and audio. As a result of these studies we found that PIQ generates interpretations that are more easily understood by participants to our user studies when compared to several other interpretation methods in the literature.