OpenRaiser/Intern-Atlas huggingface dataset:
ZeroSearch: Local Image Search from Text with Zero Shot Learning
Venue: arXiv cs.CV 2023
Year: 2023

The problem of organizing and finding images in a user's directory has become increasingly challenging due to the rapid growth in the number of images captured on personal devices. This paper presents a solution that utilizes zero shot learning to create image queries with only user provided text descriptions. The paper's primary contribution is the development of an algorithm that utilizes pre-trained models to extract features from images. The algorithm uses OWL to check for the presence of bounding boxes and sorts images based on cosine similarity scores. The algorithm's output is a list of images sorted in descending order of similarity, helping users to locate specific images more efficiently. The paper's experiments were conducted using a custom dataset to simulate a user's image directory and evaluated the accuracy, inference time, and size of the models. The results showed that the VGG model achieved the highest accuracy, while the Resnet50 and InceptionV3 models had the lowest inference time and size. The papers proposed algorithm provides an effective and efficient solution for organizing and finding images in a users local directory. The algorithm's performance and flexibility make it suitable for various applications, including personal image organization and search engines. Code and dataset for zero-search are available at: https://github.com/NainaniJatinZ/zero-search
Molecular Design Based on Integer Programming and Splitting Data Sets by Hyperplanes
Venue: arXiv cs.CE 2023
Year: 2023

A novel framework for designing the molecular structure of chemical compounds with a desired chemical property has recently been proposed. The framework infers a desired chemical graph by solving a mixed integer linear program (MILP) that simulates the computation process of a feature function defined by a two-layered model on chemical graphs and a prediction function constructed by a machine learning method. To improve the learning performance of prediction functions in the framework, we design a method that splits a given data set $\mathcal{C}$ into two subsets $\mathcal{C}^{(i)},i=1,2$ by a hyperplane in a chemical space so that most compounds in the first (resp., second) subset have observed values lower (resp., higher) than a threshold $θ$. We construct a prediction function $ψ$ to the data set $\mathcal{C}$ by combining prediction functions $ψ_i,i=1,2$ each of which is constructed on $\mathcal{C}^{(i)}$ independently. The results of our computational experiments suggest that the proposed method improved the learning performance for several chemical properties to which a good prediction function has been difficult to construct.
Neurosymbolic AI -- Why, What, and How
Venue: arXiv cs.AI 2023
Year: 2023

Humans interact with the environment using a combination of perception - transforming sensory inputs from their environment into symbols, and cognition - mapping symbols to knowledge about the environment for supporting abstraction, reasoning by analogy, and long-term planning. Human perception-inspired machine perception, in the context of AI, refers to large-scale pattern recognition from raw data using neural networks trained using self-supervised learning objectives such as next-word prediction or object recognition. On the other hand, machine cognition encompasses more complex computations, such as using knowledge of the environment to guide reasoning, analogy, and long-term planning. Humans can also control and explain their cognitive functions. This seems to require the retention of symbolic mappings from perception outputs to knowledge about their environment. For example, humans can follow and explain the guidelines and safety constraints driving their decision-making in safety-critical applications such as healthcare, criminal justice, and autonomous driving. This article introduces the rapidly emerging paradigm of Neurosymbolic AI combines neural networks and knowledge-guided symbolic approaches to create more capable and flexible AI systems. These systems have immense potential to advance both algorithm-level (e.g., abstraction, analogy, reasoning) and application-level (e.g., explainable and safety-constrained decision-making) capabilities of AI systems.
Empowering Learner-Centered Instruction: Integrating ChatGPT Python API and Tinker Learning for Enhanced Creativity and Problem-Solving Skills
Venue: arXiv cs.CY 2023
Year: 2023

The ChatGPT Python API plays a crucial role in promoting Learner-Centered Instruction (LCI) and aligns with the principles of Tinker Learning, allowing students to discover their learning strategies. LCI emphasizes the importance of active, hands-on learning experiences and encourages students to take responsibility for their learning journey. By integrating the ChatGPT Python API into the educational process, students can explore various resources, generate new ideas, and create content in a more personalized manner. This innovative approach enables students to engage with the learning material deeper, fostering a sense of ownership and motivation. As they work through the Creative Learning Spiral, students develop essential skills such as critical thinking, problem-solving, and creativity. The ChatGPT Python API is a valuable tool for students to explore different solutions, evaluate alternatives, and make informed decisions, all while encouraging self-directed learning. In Tinker Learning environments, the integration of ChatGPT Python API empowers students to experiment and iterate, allowing them to find the most effective learning strategies that cater to their individual needs and preferences. This personalized approach helps students to become more confident in their abilities, leading to tremendous academic success and long-term skill development. By leveraging the capabilities of the ChatGPT Python API, educational institutions can create a more engaging, supportive, and dynamic learning environment. This approach aligns with the principles of Learner-Centered Instruction and Tinker Learning, promoting a culture of curiosity, exploration, and creativity among students while preparing them for the challenges of the fast-paced, ever-changing world.
Supporting Contextual Conversational Agent-Based Software Development
Venue: arXiv cs.SE 2023
Year: 2023

Software Development (SD) is remarkably dynamic and is critically dependent on the knowledge acquired by the project's software developers as the project progresses. Software developers need to understand large amounts of information related to the tasks at hand. This information (context) is often not explicit, as it can be lost in large documentation repositories, a team member's brain, or beyond their cognitive memory capacity. These contexts include tool features, integration strategies, data structures, code syntax, approaches to tasks, project definitions, and even implicit or tacit contexts, which add significant complexity to the SD process. Current software development practices still lack sufficient techniques using the existing SD execution information and context to provide developers with relevant process guidance, augmenting their capacity to do their job using available applicable information. This paper presents ongoing and future research on an approach to support conversational agent-based knowledge-augmented software development. Developers benefit by receiving recommendations about task-related information and workflows they need to execute. This work advances human-computer interaction patterns in workflow engines, from graphical user interfaces to conversational patterns in software engineering.
Designing Adaptive Developer-Chatbot Interactions: Context Integration, Experimental Studies, and Levels of Automation
Venue: arXiv cs.SE 2023
Year: 2023

The growing demand for software developers and the increasing development complexity have emphasized the need for support in software engineering projects. This is especially relevant in light of advancements in artificial intelligence, such as conversational systems. A significant contributor to the complexity of software development is the multitude of tools and methods used, creating various contexts in which software developers must operate. Moreover, there has been limited investigation into the interaction between context-based chatbots and software developers through experimental user studies. Assisting software developers in their work becomes essential. In particular, understanding the context surrounding software development and integrating this context into chatbots can lead to novel insight into what software developers expect concerning these human-chatbot interactions and their levels of automation. In my research, I study the design of context-based adaptive interactions between software developers and chatbots to foster solutions and knowledge to support software developers at work.
Freeform Templates: Combining Freeform Curation with Structured Templates
Venue: arXiv cs.HC 2023
Year: 2023

Online whiteboards are becoming a popular way to facilitate collaborative design work, providing a free-form environment to curate ideas. However, as templates are increasingly being used to scaffold contributions from non-experts designers, it is crucial to understand their impact on the creative process. In this paper, we present the results from a study with 114 students in a large introductory design course. Our results confirm prior findings that templates benefit students by providing a starting point, a shared process, and the ability to access their own work from previous steps. While prior research has criticized templates for being too rigid, we discovered that using templates within a free-form environment resulted in visual patterns of free-form curation where concepts were spatially organized, clustered, color-coded, and connected using arrows and lines. We introduce the concept of "Free-form Templates" to illustrate how templates and free-form curation can be synergistic.
Behavioral Forensics in Social Networks: Identifying Misinformation, Disinformation and Refutation Spreaders Using Machine Learning
Venue: arXiv cs.SI 2023
Year: 2023

With the ever-increasing spread of misinformation on online social networks, it has become very important to identify the spreaders of misinformation (unintentional), disinformation (intentional), and misinformation refutation. It can help in educating the first, stopping the second, and soliciting the help of the third category, respectively, in the overall effort to counter misinformation spread. Existing research to identify spreaders is limited to binary classification (true vs false information spreaders). However, people's intention (whether naive or malicious) behind sharing misinformation can only be understood after observing their behavior after exposure to both the misinformation and its refutation which the existing literature lacks to consider. In this paper, we propose a labeling mechanism to label people as one of the five defined categories based on the behavioral actions they exhibit when exposed to misinformation and its refutation. However, everyone does not show behavioral actions but is part of a network. Therefore, we use their network features, extracted through deep learning-based graph embedding models, to train a machine learning model for the prediction of the classes. We name our approach behavioral forensics since it is an evidence-based investigation of suspicious behavior which is spreading misinformation and disinformation in our case. After evaluating our proposed model on a real-world Twitter dataset, we achieved 77.45% precision and 75.80% recall in detecting the malicious actors, who shared the misinformation even after receiving its refutation. Such behavior shows intention, and hence these actors can rightfully be called agents of disinformation spread.
Generalization for slowly mixing processes
Venue: arXiv cs.LG 2023
Year: 2023

A bound uniform over various loss-classes is given for data generated by stationary and phi-mixing processes, where the mixing time (the time needed to obtain approximate independence) enters the sample complexity only in an additive way. For slowly mixing processes this can be a considerable advantage over results with multiplicative dependence on the mixing time. The admissible loss-classes include functions with prescribed Lipschitz norms or smoothness parameters. The bound can also be applied to be uniform over unconstrained loss-classes, where it depends on local Lipschitz properties of the function on the sample path.
Exploring the Privacy Concerns in Permissionless Blockchain Networks and Potential Solutions
Venue: arXiv cs.CR 2023
Year: 2023

In recent years, permissionless blockchains have gained significant attention for their ability to secure and provide transparency in transactions. The development of blockchain technology has shifted from cryptocurrency to decentralized finance, benefiting millions of unbanked individuals, and serving as the foundation of Web3, which aims to provide the next generation of the internet with data ownership for users. The rise of NFTs has also helped artists and creative workers to protect their intellectual property and reap the benefits of their work. However, privacy risks associated with permissionless blockchains have become a major concern for individuals and institutions. The role of blockchain in the transition from Web2 to Web3 is crucial, as it is rapidly evolving. As more individuals, institutions, and organizations adopt this technology, it becomes increasingly important to closely monitor the new risks associated with permissionless blockchains and provide updated solutions to mitigate them. This paper endeavors to examine the privacy risks inherent in permissionless blockchains, including Remote Procedure Call (RPC) issues, Ethereum Name Service (ENS), miner extractable value (MEV) bots, on-chain data analysis, data breaches, transaction linking, transaction metadata, and others. The existing solutions to these privacy risks, such as zero-knowledge proofs, ring signatures, Hyperledger Fabric, and stealth addresses, shall be analyzed. Finally, suggestions for the future improvement of privacy solutions in the permissionless blockchain space shall be put forward.
Robust Communication Complexity of Matching: EDCS Achieves 5/6 Approximation
Venue: arXiv cs.DS 2023
Year: 2023

We study the robust communication complexity of maximum matching. Edges of an arbitrary $n$-vertex graph $G$ are randomly partitioned between Alice and Bob independently and uniformly. Alice has to send a single message to Bob such that Bob can find an (approximate) maximum matching of the whole graph $G$. We specifically study the best approximation ratio achievable via protocols where Alice communicates only $\widetilde{O}(n)$ bits to Bob. There has been a growing interest on the robust communication model due to its connections to the random-order streaming model. An algorithm of Assadi and Behnezhad [ICALP'21] implies a $(2/3+ε_0 \sim .667)$-approximation for a small constant $0 < ε_0 < 10^{-18}$, which remains the best-known approximation for general graphs. For bipartite graphs, Assadi and Behnezhad [Random'21] improved the approximation to .716 albeit with a computationally inefficient (i.e., exponential time) protocol. In this paper, we study a natural and efficient protocol implied by a random-order streaming algorithm of Bernstein [ICALP'20] which is based on edge-degree constrained subgraphs (EDCS) [Bernstein and Stein; ICALP'15]. The result of Bernstein immediately implies that this protocol achieves an (almost) $(2/3 \sim .666)$-approximation in the robust communication model. We present a new analysis, proving that it achieves a much better (almost) $(5/6 \sim .833)$-approximation. This significantly improves previous approximations both for general and bipartite graphs. We also prove that our analysis of Bernstein's protocol is tight.
Optimal Power Assignment for MIMO Channels Under Joint Total and Per-Group Power Constraints
Venue: arXiv cs.IT 2023
Year: 2023

In this paper we consider a communication system with one transmitter and one receiver. The transmit antennas are partitioned into disjoint groups, and each group must satisfy an average power constraint in addition to the standard overall one. The optimal power allocation (OPA) for the transmit antennas is obtained for the following cases: (i) fixed multiple-input multiple-output (MIMO) orthogonal channel, (ii) i.i.d. fading MIMO orthogonal channel, and (iii) i.i.d. Rayleigh fading multiple-input single-output (MISO) and MIMO channels. The channel orthogonality is encountered in the practical case of the massive MIMO channel under favorable propagation conditions. The closed-form solution to the OPA for a fixed channel is found using the Karush-Kuhn-Tucker (KKT) conditions and it is similar to the standard water-filling procedure while the effect of the per-group average power constraint is added. For a fading channel, an algorithm is proposed to give the OPA, and the algorithm's convergence is proved via a majorization inequality and a Schur-concavity property.
Analysis of different temporal graph neural network configurations on dynamic graphs
Venue: arXiv cs.LG 2023
Year: 2023

In recent years, there has been an increasing interest in the use of graph neural networks (GNNs) for analyzing dynamic graphs, which are graphs that evolve over time. However, there is still a lack of understanding of how different temporal graph neural network (TGNs) configurations can impact the accuracy of predictions on dynamic graphs. Moreover, the hunt for benchmark datasets for these TGNs models is still ongoing. Up until recently, Pytorch Geometric Temporal came up with a few benchmark datasets but most of these datasets have not been analyzed with different TGN models to establish the state-of-the-art. Therefore, this project aims to address this gap in the literature by performing a qualitative analysis of spatial-temporal dependence structure learning on dynamic graphs, as well as a comparative study of the effectiveness of selected TGNs on node and edge prediction tasks. Additionally, an extensive ablation study will be conducted on different variants of the best-performing TGN to identify the key factors contributing to its performance. By achieving these objectives, this project will provide valuable insights into the design and optimization of TGNs for dynamic graph analysis, with potential applications in areas such as disease spread prediction, social network analysis, traffic prediction, and more. Moreover, an attempt is made to convert snapshot-based data to the event-based dataset and make it compatible with the SOTA model namely TGN to perform node regression task.
Federated Neural Radiance Fields
Venue: arXiv cs.CV 2023
Year: 2023

The ability of neural radiance fields or NeRFs to conduct accurate 3D modelling has motivated application of the technique to scene representation. Previous approaches have mainly followed a centralised learning paradigm, which assumes that all training images are available on one compute node for training. In this paper, we consider training NeRFs in a federated manner, whereby multiple compute nodes, each having acquired a distinct set of observations of the overall scene, learn a common NeRF in parallel. This supports the scenario of cooperatively modelling a scene using multiple agents. Our contribution is the first federated learning algorithm for NeRF, which splits the training effort across multiple compute nodes and obviates the need to pool the images at a central node. A technique based on low-rank decomposition of NeRF layers is introduced to reduce bandwidth consumption to transmit the model parameters for aggregation. Transferring compressed models instead of the raw data also contributes to the privacy of the data collecting agents.
Faster OreFSDet : A Lightweight and Effective Few-shot Object Detector for Ore Images
Venue: arXiv cs.CV 2023
Year: 2023

For the ore particle size detection, obtaining a sizable amount of high-quality ore labeled data is time-consuming and expensive. General object detection methods often suffer from severe over-fitting with scarce labeled data. Despite their ability to eliminate over-fitting, existing few-shot object detectors encounter drawbacks such as slow detection speed and high memory requirements, making them difficult to implement in a real-world deployment scenario. To this end, we propose a lightweight and effective few-shot detector to achieve competitive performance with general object detection with only a few samples for ore images. First, the proposed support feature mining block characterizes the importance of location information in support features. Next, the relationship guidance block makes full use of support features to guide the generation of accurate candidate proposals. Finally, the dual-scale semantic aggregation module retrieves detailed features at different resolutions to contribute with the prediction process. Experimental results show that our method consistently exceeds the few-shot detectors with an excellent performance gap on all metrics. Moreover, our method achieves the smallest model size of 19MB as well as being competitive at 50 FPS detection speed compared with general object detectors. The source code is available at https://github.com/MVME-HBUT/Faster-OreFSDet.
Exploration of Unranked Items in Safe Online Learning to Re-Rank
Venue: arXiv cs.IR 2023
Year: 2023

Bandit algorithms for online learning to rank (OLTR) problems often aim to maximize long-term revenue by utilizing user feedback. From a practical point of view, however, such algorithms have a high risk of hurting user experience due to their aggressive exploration. Thus, there has been a rising demand for safe exploration in recent years. One approach to safe exploration is to gradually enhance the quality of an original ranking that is already guaranteed acceptable quality. In this paper, we propose a safe OLTR algorithm that efficiently exchanges one of the items in the current ranking with an item outside the ranking (i.e., an unranked item) to perform exploration. We select an unranked item optimistically to explore based on Kullback-Leibler upper confidence bounds (KL-UCB) and safely re-rank the items including the selected one. Through experiments, we demonstrate that the proposed algorithm improves long-term regret from baselines without any safety violation.
Integrated Sensing and Communication in Coordinated Cellular Networks
Venue: arXiv cs.IT 2023
Year: 2023

Integrated sensing and communication (ISAC) is a promising technique to provide sensing services in future wireless networks. Numerous existing works have adopted a monostatic radar architecture to realize ISAC, i.e., employing the same base station (BS) to transmit the ISAC signal and receive the echo. Yet, the concurrent information transmission causes unavoidable self-interference (SI) to the radar echo at the BS. To overcome this difficulty, we propose a coordinated cellular network-supported multistatic radar architecture to implement ISAC, which allows us to spatially separate the ISAC signal transmission and radar echo reception, intrinsically circumventing the problem of SI. To this end, we jointly optimize the transmit and receive beamforming policy to minimize the sensing beam pattern mismatch error subject to ISAC quality-of-service requirements. The resulting non-convex optimization problem is tackled by an alternating optimization-based suboptimal algorithm. Simulation results showed that the proposed scheme outperforms the two baseline schemes adopting conventional designs.