Reviewerly/exHarmony huggingface dataset:
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer

Many machine learning models operate on images, but ignore the fact that images are 2D projections formed by 3D geometry interacting with light, in a process called rendering. Enabling ML models to understand image formation might be key for generalization. However, due to an essential rasterization step involving discrete assignment operations, rendering pipelines are non-differentiable and thus largely inaccessible to gradient-based ML techniques. In this paper, we present DIB-Render, a novel rendering framework through which gradients can be analytically computed. Key to our approach is to view rasterization as a weighted interpolation, allowing image gradients to back-propagate through various standard vertex shaders within a single framework. Our approach supports optimizing over vertex positions, colors, normals, light directions and texture coordinates, and allows us to incorporate various well-known lighting models from graphics. We showcase our approach in two ML applications: single-image 3D object prediction, and 3D textured object generation, both trained using exclusively 2D supervision.
Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume

Deep learning has shown to be effective for depth inference in multi-view stereo (MVS). However, the scalability and accuracy still remain an open problem in this domain. This can be attributed to the memory-consuming cost volume representation and inappropriate depth inference. Inspired by the group-wise correlation in stereo matching, we propose an average group-wise correlation similarity measure to construct a lightweight cost volume. This can not only reduce the memory consumption but also reduce the computational burden in the cost volume filtering. Based on our effective cost volume representation, we propose a cascade 3D U-Net module to regularize the cost volume to further boost the performance. Unlike the previous methods that treat multi-view depth inference as a depth regression problem or an inverse depth classification problem, we recast multi-view depth inference as an inverse depth regression task. This allows our network to achieve sub-pixel estimation and be applicable to large-scale scenes. Through extensive experiments on DTU dataset and Tanks and Temples dataset, we show that our proposed network with Correlation cost volume and Inverse DEpth Regression (CIDER1), achieves state-of-the-art results, demonstrating its superior performance on scalability and accuracy.
The Potential of Immersive Virtual Reality for Cognitive Training in Elderly

The number of elderly is growing rapidly, including those that struggle with cognitive decline and, consequently, activities of daily living. Developing interventions that slow down cognitive decline would greatly benefit the autonomy of these elderly, as well as their caretakers. Virtual reality (VR) is an immersive and interactive human-computer interface with real-time simulation that is suggested to be a promising tool to improve cognitive functioning. This literature review discusses the features of immersive VR, including its opportunities and limitations, that are of special relevance for improving cognition in elderly. First, the high flexibility of VR allows for personalized and safe training of potentially harmful situations and isolated aspects of tasks. Second, VR’s dynamic and real-time feedback on performance provides a labor-extensive tool for caretakers in elderly care. Third, the acquisition of high resolution data enables the exposure of subtle changes over time, relevant for detecting the nature and extent of cognitive changes in elderly, including those with dementia. Fourth, immersive VR has been related to cybersickness. Finally, VR is immersive and enables the perception and expression of emotions, motor behaviors, and multiple senses. Aged subjects are found to have sparing of emotional learning and show increased reliance on multisensory integration for learning. Moreover, they experience high levels of flow, motivation, and presence in VR setups. Despite the clear potential of immersive VR, the studies available are small scale and have not been replicated. The digital illiteracy of elderly and healthcare personnel form an additional barrier for implementation. Hence, while replication of outcomes and standardization of VR interventions is required before it can be used in standard care, VR may provide a particularly effective tool for training cognition in elderly.
A Survey on Blockchain Consensus with a Performance Comparison of PoW, PoS and Pure PoS

Blockchain technology started as the backbone for cryptocurriencies and it has emerged as one of the most interesting technologies of the last decade. It is a new paradigm able to modify the way how industries transact. Today, the industries’ concern is about their ability to handle a high volume of data transactions per second while preserving both decentralization and security. Both decentralization and security are guaranteed by the mathematical strength of cryptographic primitives. There are two main approaches to achieve consensus: the Proof-of-Work based blockchains—PoW—and the Proof-of-Stake—PoS. Both of them come with some pros and drawbacks, but both rely on cryptography. In this survey, we present a review of the main consensus procedures, including the new consensus proposed by Algorand: Pure Proof-of-Stake—Pure PoS. In this article, we provide a framework to compare the performances of PoW, PoS and the Pure PoS, based on throughput and scalability.
A Systematic Survey on Energy-Efficient Techniques in Sustainable Cloud Computing

Global warming is one of the most compelling environmental threats today, as the rise in energy consumption and CO2 emission caused a dreadful impact on our environment. The data centers, computing devices, network equipment, etc., consume vast amounts of energy that the thermal power plants mainly generate. Primarily fossil fuels like coal and oils are used for energy generation in these power plants that induce various environmental problems such as global warming ozone layer depletion, which can even become the cause of premature deaths of living beings. The recent research trend has shifted towards optimizing energy consumption and green fields since the world recognized the importance of these concepts. This paper aims to conduct a complete systematic mapping analysis on the impact of high energy consumption in cloud data centers and its effect on the environment. To answer the research questions identified in this paper, one hundred nineteen primary studies published until February 2022 were considered and further categorized. Some new developments in green cloud computing and the taxonomy of various energy efficiency techniques used in data centers have also been discussed. It includes techniques like VM Virtualization and Consolidation, Power-aware, Bio-inspired methods, Thermal-management techniques, and an effort to evaluate the cloud data center’s role in reducing energy consumption and CO2 footprints. Most of the researchers proposed software level techniques as with these techniques, massive infrastructures are not required as compared with hardware techniques, and it is less prone to failure and faults. Also, we disclose some dominant problems and provide suggestions for future enhancements in green computing.
Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework

The current popular two-stream, two-stage tracking framework extracts the template and the search region features separately and then performs relation modeling, thus the extracted features lack the awareness of the target and have limited target-background discriminability. To tackle the above issue, we propose a novel one-stream tracking (OSTrack) framework that unifies feature learning and relation modeling by bridging the template-search image pairs with bidirectional information flows. In this way, discriminative target-oriented features can be dynamically extracted by mutual guidance. Since no extra heavy relation modeling module is needed and the implementation is highly parallelized, the proposed tracker runs at a fast speed. To further improve the inference efficiency, an in-network candidate early elimination module is proposed based on the strong similarity prior calculated in the one-stream framework. As a unified framework, OSTrack achieves state-of-the-art performance on multiple benchmarks, in particular, it shows impressive results on the one-shot tracking benchmark GOT-10k, i.e.,, achieving 73.7% AO, improving the existing best result (SwinTrack) by 4.3%. Besides, our method maintains a good performance-speed trade-off and shows faster convergence. The code and models are available at https://github.com/botaoye/OSTrack .
Compute Trends Across Three Eras of Machine Learning

Compute, data, and algorithmic advances are the three fundamental factors that guide the progress of modern Machine Learning (ML).In this paper we study trends in the most readily quantified factor compute.We show that before 2010 training compute grew in line with Moore's law, doubling roughly every 20 months.Since the advent of Deep Learning in the early 2010s, the scaling of training compute has accelerated, doubling approximately every 6 months.In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute.Based on these observations we split the history of compute in ML into three eras: the Pre Deep Learning Era , the Deep Learning Era and the Large-Scale Era .Overall, our work highlights the fast-growing compute requirements for training advanced ML systems.
Multi-view underwater image enhancement method via embedded fusion mechanism

Due to wavelength-dependent light absorption and scattering, underwater images often appear with a colour cast and blurry details. Most existing deep learning methods utilize a single input end-to-end network structure, which leads to a single form and content of the extracted features. To address these problems, we present a novel multi-feature underwater image enhancement method via embedded fusion mechanism (MFEF). We find that the quality of reconstruction results is affected by the quality of the input image to some extent, and use pre-processing to obtain high-quality images, which can improve the final reconstruction effect. We introduce the white balance (WB) algorithm and the contrast-limited adaptive histogram equalization (CLAHE) algorithm employing multiple path inputs to extract different forms of rich features in multiple views. To fully interact with features from multiple views, we design a multi-feature fusion (MFF) module to fuse derived image features. We suggest a novel pixel-weighted channel attention module (PCAM) that calibrates the detailed features of the degraded images using a weight matrix to give diverse weights to the encoded features. Ultimately, our network utilizes a fusion mechanism-based encoder and decoder that can be applied to restore various underwater scenes. In the UIEB dataset, our PSNR increased by 10.2% compared to that of Ucolor. Extensive experimental results demonstrate that the MFEF method outperforms other state-of-the-art underwater image enhancement methods in various real-world datasets.
Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language

GitHub Copilot is an artificial intelligence tool for automatically generating source code from natural language problem descriptions. Since June 2022, Copilot has officially been available for free to all students as a plug-in to development environments like Visual Studio Code. Prior work exploring OpenAI Codex, the underlying model that powers Copilot, has shown it performs well on typical CS1 problems thus raising concerns about its potential impact on how introductory programming courses are taught. However, little is known about the types of problems for which Copilot does not perform well, or about the natural language interactions that a student might have with Copilot when resolving errors. We explore these questions by evaluating the performance of Copilot on a publicly available dataset of 166 programming problems. We find that it successfully solves around half of these problems on its very first attempt, and that it solves 60% of the remaining problems using only natural language changes to the problem description. We argue that this type of prompt engineering, which we believe will become a standard interaction between human and Copilot when it initially fails, is a potentially useful learning activity that promotes computational thinking skills, and is likely to change the nature of code writing skill development.
No More Strided Convolutions or Pooling: A New CNN Building Block for Low-Resolution Images and Small Objects

Convolutional neural networks (CNNs) have made resounding success in many computer vision tasks such as image classification and object detection. However, their performance degrades rapidly on tougher tasks where images are of low resolution or objects are small. In this paper, we point out that this roots in a defective yet common design in existing CNN architectures, namely the use of strided convolution and/or pooling layers, which results in a loss of fine-grained information and learning of less effective feature representations. To this end, we propose a new CNN building block called SPD-Conv in place of each strided convolution layer and each pooling layer (thus eliminates them altogether). SPD-Conv is comprised of a space-to-depth (SPD) layer followed by a non-strided convolution (Conv) layer, and can be applied in most if not all CNN architectures. We explain this new design under two most representative computer vision tasks: object detection and image classification. We then create new CNN architectures by applying SPD-Conv to YOLOv5 and ResNet, and empirically show that our approach significantly outperforms state-of-the-art deep learning models, especially on tougher tasks with low-resolution images and small objects. We have open-sourced our code at https://github.com/LabSAINT/SPD-Conv .
A Survey on the Metaverse: The State-of-the-Art, Technologies, Applications, and Challenges

In recent years, the concept of the Metaverse has attracted considerable attention. This article provides a comprehensive overview of the Metaverse. First, the development status of the Metaverse is presented. We summarize the policies of various countries, companies, and organizations relevant to the Metaverse, as well as statistics on the number of Metaverse-related publications. Characteristics of the Metaverse are identified: 1) multitechnology convergence; 2) sociality; and 3) hyper-spatio-temporality. For the multitechnology convergence of the Metaverse, we divide the technological framework of the Metaverse into five dimensions. For the sociality of the Metaverse, we focus on the Metaverse as a virtual social world. Regarding the characteristic of hyper-spatio-temporality, we introduce the Metaverse as an open, immersive, and interactive 3-D virtual world which can break through the constraints of time and space in the real world. The challenges of the Metaverse are also discussed.
Convolutional Neural Networks: A Survey

Artificial intelligence (AI) has become a cornerstone of modern technology, revolutionizing industries from healthcare to finance. Convolutional neural networks (CNNs) are a subset of AI that have emerged as a powerful tool for various tasks including image recognition, speech recognition, natural language processing (NLP), and even in the field of genomics, where they have been utilized to classify DNA sequences. This paper provides a comprehensive overview of CNNs and their applications in image recognition tasks. It first introduces the fundamentals of CNNs, including the layers of CNNs, convolution operation (Conv_Op), Feat_Maps, activation functions (Activ_Func), and training methods. It then discusses several popular CNN architectures such as LeNet, AlexNet, VGG, ResNet, and InceptionNet, and compares their performance. It also examines when to use CNNs, their advantages and limitations, and provides recommendations for developers and data scientists, including preprocessing the data, choosing appropriate hyperparameters (Hyper_Param), and evaluating model performance. It further explores the existing platforms and libraries for CNNs such as TensorFlow, Keras, PyTorch, Caffe, and MXNet, and compares their features and functionalities. Moreover, it estimates the cost of using CNNs and discusses potential cost-saving strategies. Finally, it reviews recent developments in CNNs, including attention mechanisms, capsule networks, transfer learning, adversarial training, quantization and compression, and enhancing the reliability and efficiency of CNNs through formal methods. The paper is concluded by summarizing the key takeaways and discussing the future directions of CNN research and development.
Adapting Feature Selection Algorithms for the Classification of Chinese Texts

Text classification has been highlighted as the key process to organize online texts for better communication in the Digital Media Age. Text classification establishes classification rules based on text features, so the accuracy of feature selection is the basis of text classification. Facing fast-increasing Chinese electronic documents in the digital environment, scholars have accumulated quite a few algorithms for the feature selection for the automatic classification of Chinese texts in recent years. However, discussion about how to adapt existing feature selection algorithms for various types of Chinese texts is still inadequate. To address this, this study proposes three improved feature selection algorithms and tests their performance on different types of Chinese texts. These include an enhanced CHI square with mutual information (MI) algorithm, which simultaneously introduces word frequency and term adjustment (CHMI); a term frequency–CHI square (TF–CHI) algorithm, which enhances weight calculation; and a term frequency–inverse document frequency (TF–IDF) algorithm enhanced with the extreme gradient boosting (XGBoost) algorithm, which improves the algorithm’s ability of word filtering (TF–XGBoost). This study randomly chooses 3000 texts from six different categories of the Sogou news corpus to obtain the confusion matrix and evaluate the performance of the new algorithms with precision and the F1-score. Experimental comparisons are conducted on support vector machine (SVM) and naive Bayes (NB) classifiers. The experimental results demonstrate that the feature selection algorithms proposed in this paper improve performance across various news corpora, although the best feature selection schemes for each type of corpus are different. Further studies of the application of the improved feature selection methods in other languages and the improvement in classifiers are suggested.
Time-Dependent Graphs: Definitions, Applications, and Algorithms

Abstract A time-dependent graph is, informally speaking, a graph structure dynamically changes with time. In such graphs, the weights associated with edges dynamically change over time, that is, the edges in such graphs are activated by sequences of time-dependent elements. Many real-life scenarios can be better modeled by time-dependent graphs, such as bioinformatics networks, transportation networks, and social networks. In particular, the time-dependent graph is a very broad concept, which is reflected in the related research with many names, including temporal graphs, evolving graphs, time-varying graphs, historical graphs, and so on. Though static graphs have been extensively studied, for their time-dependent generalizations, we are still far from a complete and mature theory of models and algorithms. In this paper, we discuss the definition and topological structure of time-dependent graphs, as well as models for their relationship to dynamic systems. In addition, we review some classic problems on time-dependent graphs, e.g., route planning, social analysis, and subgraph problem (including matching and mining). We also introduce existing time-dependent systems and summarize their advantages and limitations. We try to keep the descriptions consistent as much as possible and we hope the survey can help practitioners to understand existing time-dependent techniques.
Framework for Efficient Medical Image Encryption Using Dynamic S-Boxes and Chaotic Maps

Protecting patient privacy and medical records is a legal requirement. Traditional encryption methods fall short of handling the large volume of medical image data and their peculiar statistical properties. In this paper, we propose a generic medical image encryption framework based on a novel arrangement of two very efficient constructs, dynamic substitution boxes (S-boxes) and chaotic maps. The arrangement of S-box substitution before and after chaotic substitution is shown to successfully resist chosen plaintext and chosen ciphertext attacks. Special precautions are taken to fend off the reset attack against pseudorandom number generators. We show how to implement the generic framework using any key-dependent dynamic S-box construction method and any chaotic map. Experimental results show that the proposed framework successfully passes all security tests regardless of the chaotic map used for implementation. Based on speed analysis, we recommend the use of the classical Baker map or Henon map to achieve encryption throughput approaching 90 MB/s on a modern PC without hardware acceleration.
Efficient One-Pass End-to-End Entity Linking for Questions

We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass. Evaluated on WebQSP and GraphQuestions with extended annotations that cover multiple entities per question, ELQ outperforms the previous state of the art by a large margin of +12.7% and +19.6% F1, respectively. With a very fast inference time (1.57 examples/s on a single CPU), ELQ can be useful for downstream question answering systems. In a proof-of-concept experiment, we demonstrate that using ELQ significantly improves the downstream QA performance of GraphRetriever.