Reviewerly/exHarmony huggingface dataset:
Boon: A Neural Search Engine for Cross-Modal Information Retrieval

Visual-Semantic Embedding (VSE) networks can help search engines understand the meaning behind visual content and associate it with relevant textual information, leading to accurate search results. VSE networks can be used in cross-modal search engines to embed image and textual descriptions in a shared space, enabling image-to-text and text-to-image retrieval tasks. However, the full potential of VSE networks for search engines has yet to be fully explored. This paper presents Boon, a novel cross-modal search engine that combines two state-of-the-art networks: the GPT-3.5-turbo large language model, and the VSE network VITR (VIsion Transformers with Relation-focused learning) to enhance the engine's capabilities in extracting and reasoning with regional relationships in images. VITR employs encoders from CLIP that were trained with 400 million image-description pairs and it was fine-turned on the RefCOCOg dataset. Boon's neural-based components serve as its main functionalities: 1) a 'cross-modal search engine' that enables end-users to perform image-to-text and text-to-image retrieval. 2) a 'multi-lingual conversational AI' component that enables the end-user to converse about one or more images selected by the end-user. Such a feature makes the search engine accessible to a wide audience, including those with visual impairments. 3) Boon is multi-lingual and can take queries and handle conversations about images in multiple languages. Boon was implemented using the Django and PyTorch frameworks. The interface and capabilities of the Boon search engine are demonstrated using the RefCOCOg dataset, and the engine's ability to search for multimedia through the web is facilitated by Google's API.
Spatially Localised Immersive Contemporary and Historic Photo Presentation on Mobile Devices in Augmented Reality

These days, taking a photo is the most common way of capturing a moment. Some of these photos captured in the moment are never to be seen again. Others are almost immediately shared with the world. Yet, the context of the captured moment can only be shared to a limited extent. The continuous improvement of mobile devices has not only led to higher resolution cameras and, thus, visually more appealing pictures but also to a broader and more precise range of accompanying sensor metadata. Positional and bearing information can provide context for photos and is thus an integral aspect of the captured moment. However, it is commonly only used to sort photos by time and possibly group by place. Such more precise sensor metadata, combined with the increased computing power of mobile devices, can enable more and more powerful Augmented Reality (AR) capabilities, especially for communicating the context of a captured photo. Users can thereby witness the captured moment in its real location and also experience its spatial contextualization. With the help of a suitable data augmentation, such context-preserving presentation can be extended even to non-digitally born content, including historical images. This offers new immersive ways to experience the cultural history of one's current location. In this paper, we present an approach for location-based image presentation in AR on mobile devices. With this approach, users can experience captured moments in their physical context. We demonstrate the power of this approach based on a prototype implementation and evaluate it in a user study.
Experimental Simulation for Two Optically Filtered Modulation Weights in Laser Diode as a Self-Learning Layer

In this study, the response of a nonlinear laser medium is experimentally studied. In the study, a hybrid version of the input layer that multiplies optically and accumulates electrically is compared with a wholly optical version that multiplies and accumulates optically. This medium is subjected to two different paths of optically filtered and attenuated feedback. With such a system, the variation of feedback weight in one of them is tested in correspondence to the second one. Observations for frequency spectra are carried out to simulate the resultant response with an input layer for a neural network based on chaotic carriers. Chaotic laser emission was observed as a function of several control parameters, which are D.C. bias voltage, branch optical attenuation, and feedback strengths based on filtration with fiber Bragg grating. This learning rule is linear in the difference between each input and output of a neuron. This is an enhancing/inhibiting rule. The thresholds are adjusted in such a way that the output of the neuron is either pushed in the same direction as the input (enhancing) or pushed in the opposite direction (inhibiting).&#x0D;
Treehouse: A Case For Carbon-Aware Datacenter Software

The end of Dennard scaling and the slowing of Moore's Law has put the energy use of datacenters on an unsustainable path. Datacenters are already a significant fraction of worldwide electricity use, with application demand scaling at a rapid rate. We argue that substantial reductions in the carbon intensity of datacenter computing are possible with a software-centric approach: by making energy and carbon visible to application developers on a fine-grained basis, by modifying system APIs to make it possible to make informed trade offs between performance and carbon emissions, and by raising the level of application programming to allow for flexible use of more energy efficient means of compute and storage. We also lay out a research agenda for systems software to reduce the carbon footprint of datacenter computing.
The Odd One Out: Energy is Not Like Other Metrics

Energy requirements for datacenters are growing at a fast pace. Existing techniques for making datacenters efficient focus on hardware. However, the gain in energy efficiency that can be achieved without making the applications energy-aware is limited. To overcome this limitation, recent work has proposed making the software running in datacenters energy aware. To do so, we must be able to track energy consumption at various granularities at the software level - (i) process level; (ii) application level; (iii) end-to-end request level. Currently, existing software energy-tracking techniques primarily focus on tracking energy at the process or application level; only a few techniques track energy at an end-to-end request level. However, not tracking energy at an end-to-end request level can lead to false software optimizations and cause a decrease in energy efficiency. To track energy at an end-to-end request level, we can leverage end-to-end tracking techniques for other metrics such as distributed tracing. However, we posit that energy cannot be treated as just another metric and that we cannot use existing frameworks without modifications. In this paper, we discuss how energy is different from other metrics and describe an energy-tracking workflow that leverages these differences and tracing techniques in order to track energy consumption of end-to-end requests.
Advancing Pandemic Preparedness in Healthcare 5.0: A Survey of Federated Learning Applications

The intersection of Federated Learning (FL) and Healthcare 5.0 promises a transformative shift towards a more resilient future, particularly concerning pandemic preparedness. Within this context, Healthcare 5.0 signifies a holistic approach to healthcare delivery, where interconnected technologies enable data-driven decision-making, patient-centric care, and enhanced efficiency. This paper provides an in-depth exploration of FL’s role within the framework of Healthcare 5.0 and its implications for the pandemic response. Specifically, FL offers the potential to revolutionize pandemic preparedness within Healthcare 5.0 in several vital ways: it enables collaborative learning from distributed data sources without compromising individual data privacy, facilitates decentralized decision-making by empowering local healthcare institutions to contribute to a collective knowledge pool, and enhances real-time surveillance, enabling early detection of outbreaks and informed responses. We start by laying out the concepts of FL and Healthcare 5.0, followed by an analysis of current pandemic preparedness and response mechanisms. We delve into FL’s applications and case studies in healthcare, highlighting its potential benefits, including privacy protection, decentralized decision-making, and implementation challenges. By articulating how FL fits into Healthcare 5.0, we envisage future applications in a technologically integrated health system. By examining current applications and case studies of FL in healthcare, we highlight its potential benefits, including enhanced privacy protection and more effective decision support systems. Our findings demonstrate that FL can significantly improve pandemic response times and accuracy. Moreover, we speculate on the potential scenarios where FL could enhance pandemic preparedness and make healthcare more resilient. Finally, we recommend that policymakers, technologists, and educators address potential challenges and maximize the benefits of FL in Healthcare 5.0. This paper aims to contribute to the discourse on next-generation healthcare technologies, emphasizing FL’s potential to shape a more resilient healthcare future.
RAVA: An Open Hardware True Random Number Generator Based on Avalanche Noise

Entropy is a crucial resource in the domains of cryptography, artificial intelligence, and science. This paper introduces RAVA, a true random number generator based on avalanche noise. RAVA is an open-source device designed to offer a transparent and customizable platform, making auditable and high-quality entropy accessible to a wider audience. The device employs a differential design, which involves comparing two similar noise sources to mitigate the impact of environmental factors. Furthermore, RAVA incorporates a dual entropy core architecture featuring two independent entropy channels that generate random bytes simultaneously. A stochastic model is theoretically derived and empirically confirmed, offering valuable insights into the entropy extraction mechanism and allowing the estimation of the minimum bias attainable. An implementation is presented as a discrete circuit with an ATmega32U4 microcontroller including a USB interface, achieving an unbiased throughput of 136.0 Kbit/s without the necessity of post-processing algorithms. The generated random bytes are evaluated for bias and serial correlation, their entropy is assessed using NIST SP 800-90B estimators, and the randomness quality is verified using the NIST 800-22R1a test suit. For comparison, the same tests are applied to a commercial device based on quantum optical phenomena, revealing similar distributions for both devices across the studied metrics.
Supply Chain Management Serious Game Using Blockchain Smart Contract

Smart contracts and Blockchain are necessary for transactions in supply chain management. A smart contract is an agreement in the form of a cryptographic code digitally signed by the actors. This technology holds the data transaction transparently and securely between the actors. The actors may benefit without intermediaries involved in the transaction. Supply chain management will be easily tracked by decentralized data sharing. Supply chain management involves transactions in agricultural sectors such as farmers, Village Cooperatives units, distributors, and consumers. The Cooperative is a party that buys goods from farmers and sells them to consumers. A smart contract is a digital signature that can access the system in transactions. The system records log activities that show the origin of transactions. All actors can see the transaction of agricultural products stored in the blockchain ledger in the Blockchain. This paper discusses blockchain-based smart contracts scenario for supply chain management that is simulated and implemented in the Serious Game. The Game is developed using Unity 3D Development Platform, programming language contract using Solidity, Moralis as Web3 Development Platform and dApp as backend blockchain, Network Environment Polygon Ethereum, and Database that is Redis Enterprise, MongoDB Compass, and Mumbai Testnet.
Image Compression for Wireless Sensor Network: A Model Segmentation-Based Compressive Autoencoder

Aiming at the problems of image quality, compression performance, and transmission efficiency of image compression in wireless sensor networks (WSN), a model segmentation-based compressive autoencoder (MS-CAE) is proposed. In the proposed algorithm, we first divide each image in the dataset into pixel blocks and design a novel deep image compression network with a compressive autoencoder to form a compressed feature map by encoding pixel blocks. Then, the reconstructed image is obtained by using the quantized coefficients of the quantizer and splicing the decoded feature maps in order. Finally, the deep network model is segmented into two parts: the encoding network and the decoding network. The weight parameters of the encoding network are deployed to the edge device for the compressed image in the sensor network. For high-quality reconstructed images, the weight parameters of the decoding network are deployed to the cloud system. Experimental results demonstrate that the proposed MS-CAE obtains a high signal-to-noise ratio (PSNR) for the details of the image, and the compression ratio at the same bit per pixel (bpp) is significantly higher than that of the compared image compression algorithms. It also indicates that the MS-CAE not only greatly relieves the pressure of the hardware system in sensor network but also effectively improves image transmission efficiency and solves the deployment problem of image monitoring in remote and energy-poor areas.
The Role of Immersive Arts in Interpreting Traditional Cultural Phenomena

Immersive art is a new form of media art that relies on the theory of stream of consciousness, on the stimulation of various human senses, often using light to achieve a deep psychological and physiological immersion of the audience in a trance-like empathy with the art. From the point of view of technical means it can be divided into virtual reality, augmented reality, mixed reality and expanded reality. The article analyses the problems of using technical means of immersive art, considers the advantages and disadvantages of its application in various cultural expositions. The questions arising when applying the means of immersive art to traditional culture (in this case to shamanic culture) are considered. The study finds that works of immersive art demonstrate contemporary culture much more often than traditional culture. Using the example of four technical tools in analysing works of immersive art, the author describes the problems that exist at this stage of its application. The aim of the work is to identify and consider the advantages and disadvantages of incorporating traditional meanings into virtual reality. The tasks of the work include: analysing technical means of immersive art, peculiarities of embodiment of traditional meanings in immersive art (on the example of shamanism). The author argues that virtual spaces created by immersive art are currently gravitating towards purely sensory experience and are not able to carry information about the connotation of meanings. In the process of creating works of immersive art, artists have their own understanding of shamanic culture and cannot reproduce shamanic rituals, so there is a problem of inability to interpret the cultural connotation and show the depth of the rich traditional shamanic culture.
Detection of typical bipartite entanglement by local generalized measurements

Motivated by the need for efficient local entanglement detection for applications in quantum information processing, sufficient conditions for arbitrary-dimensional local bipartite entanglement detection based on correlation matrices and joint probability distributions are investigated. In particular, their dependence on the nature of different classes of local measurements is explored for generalized measurements based on informationally complete $(N,M)$ positive-operator-valued measures (POVMs) [Siudzinska, Phys. Rev. A 105, 042209 (2022)]. It is shown that symmetry properties of $(N,M)$ POVMs necessarily imply that these sufficient conditions for bipartite entanglement detection exhibit characteristic scaling properties relating equivalent sufficient conditions. Based on these general scaling properties, the efficiency of different classes of local quantum measurement detecting typical bipartite entanglement is investigated quantitatively. For this purpose Euclidean volume ratios between locally detectable bipartite entangled states and all bipartite quantum states are determined numerically with the help of a Monte Carlo algorithm. Our results demonstrate that physically realizable $(N,M)$ POVMs are sufficient for optimal local entanglement detection. In particular, this implies that for this purpose the construction of optimal $(N,M)$ POVMs is not necessary. As questions concerning the existence and construction of optimal $(N,M)$ POVMs are still largely open, this may offer interesting perspectives for practical applications in quantum information processing.
Improving Clothing Product Quality and Reducing Waste Based on Consumer Review Using RoBERTa and BERTopic Language Model

The disposability of clothing has emerged as a critical concern, precipitating waste accumulation due to product quality degradation. Such consequences exert significant pressure on resources and challenge sustainability efforts. In response, this research focuses on empowering clothing companies to elevate product excellence by harnessing consumer feedback. Beyond insights, this research extends to sustainability by providing suggestions on refining product quality by improving material handling, gradually mitigating waste production, and cultivating longevity, therefore decreasing discarded clothes. Managing a vast influx of diverse reviews necessitates sophisticated natural language processing (NLP) techniques. Our study introduces a Robustly optimized BERT Pretraining Approach (RoBERTa) model calibrated for multilabel classification and BERTopic for topic modeling. The model adeptly distills vital themes from consumer reviews, exhibiting astounding accuracy in projecting concerns across various dimensions of clothing quality. NLP’s potential lies in endowing companies with insights into consumer review, augmented by the BERTopic to facilitate immersive exploration of harvested review topics. This research presents a thorough case for integrating machine learning to foster sustainability and waste reduction. The contribution of this research is notable for its integration of RoBERTa and BERTopic in multilabel classification tasks and topic modeling in the fashion industry. The results indicate that the RoBERTa model exhibits remarkable performance, as demonstrated by its macro-averaged F1 score of 0.87 and micro-averaged F1 score of 0.87. Likewise, BERTopic achieves a coherence score of 0.67, meaning the model can form an insightful topic.
Effect of Augmented Reality Support on Quality Inspection of Welded Structures

With the increasing complexity of production systems and manufactured products, operators face high demands for professional expertise and long-term concentration. Augmented reality (AR) can support users in their work by displaying relevant virtual data in their field of view. In contrast to the extensive research on AR assistance in assembly, maintenance, and training, AR support in quality inspection has received less attention in the industrial context. Quality inspection is an essential part of industrial processes; thus, it is important to verify whether AR assistance can support users in these tasks. This work proposes an AR-based approach for quality inspection. For this, pilot AR software was designed and developed. The proposed AR approach was tested with end users. The task efficiency, the error rate, the perceived mental workload, and the usability of the AR approach were analysed and compared to the conventional paper-based support. The field research confirmed the positive effect on user efficiency during quality inspection while decreasing the perceived mental workload. This work extends the research on the potential of AR assistance in industrial applications and provides experimental validation to confirm the benefits of AR support on user performance during quality inspection tasks.
The metaverse—Not a new frontier for crime

Abstract Law enforcement co‐ordination agencies have recently issued position/guidance documents relating to the potential for VR environments (the “Metaverse”) to become new environment for criminal activity, and calling for additional work to enhance investigative capability. By reviewing the historic development of VR and comparing it with the appearance of the WWW, the authors propose that the situation is not as dire as the issued documents may suggest, but represents an evolutionary rather than revolutionary step in online experiences. They conclude, therefore, that while ability to examine VR presentation/interaction devices may be useful, continued development of ability to examine online systems remains essential. This article is categorized under: Digital and Multimedia Science &gt; Multimedia Forensics Digital and Multimedia Science &gt; Cybercrime Investigation Digital and Multimedia Science &gt; Artificial Intelligence
Full‐Fourier‐Component Tailorable Optical Neural Meta‐Transformer

Abstract Brain‐inspired optical neural computing (ONC) is the state‐of‐the‐art scheme in modern computing, offering robust strategies to execute advanced inference with a high throughput and large‐scale parallelism. However, the hitherto prevalent diffractive ONC networks have watered‐down competence, which is mostly a phase‐only methodology but fails to precisely handle the Fourier transform of complex fields, thus forfeiting the integrity of the architecture and half the volume of available training weights. Here, a novel neural meta‐transformer (ONM) enabled by an optical rotation‐isolator‐assisted paradigm is proposed, whose meta‐neurons utilize structural birefringence and polarization rotation to achieve independently arbitrary tailoring of full Fourier components, that is the complete learnable parameters of diffractive ONC. The full‐Fourier‐component ONM has great merits over phase‐only counterparts in all representative cases: being a classifier, it improves the recognition accuracy, especially for input with more high‐frequency features; acting as an imager, the background noise of output is effectively diminished; and when engineering as an encoder, both near‐field grayscale nanoprinting and neural meta‐holography are yielded. The mechanism is minimalist, compact, and compatible with nonlinear activation, opening the route to fully parametric intelligent meta‐devices, with far‐reaching implications for optical computing, display, encryption, etc.
Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms

The scattering transform is a multilayered wavelet-based architecture that acts as a model of convolutional neural networks. Recently, several works have generalized the scattering transform to graph-structured data. Our work builds on these constructions by introducing windowed and nonwindowed geometric scattering transforms for graphs based on two very general classes wavelets, which are in most cases based on asymmetric matrices. We show that these transforms have many of the same theoretical guarantees as their symmetric counterparts. As a result, the proposed construction unifies and extends known theoretical results for many of the existing graph scattering architectures. Therefore, it helps bridge the gap between geometric scattering and other graph neural networks by introducing a large family of networks with provable stability and invariance guarantees. These results lay the groundwork for future deep learning architectures for graph-structured data that have learned filters and also provably have desirable theoretical properties.