Reviewerly/exHarmony huggingface dataset:
Optimal rendition resolution selection algorithm for web streaming players

In web streaming, the size of video rendered on screen may be influenced by a number of factors, such as the layout of a web page embedding the video, the position and size of the web browser window, and the resolution of the screen. During the playback, the adaptive streaming players, usually select one of the available encoded streams (renditions) to pull and render on the screen. Such selection is typically done based on the available network bandwidth, and also based on the size of the player window. Typically, the logic of matching video stream to be played to the size of the window is very simplistic, considering only pixel dimensions of the video. However, with vastly different video playback devices, their pixel densities and other parameters influencing the Quality of Experience (QoE), the reliance of pixel matching is bound to be suboptimal. A better approach must use a proper QoE model, considering parameters of viewing setup on each device, and then predicting which encoded resolution, given player window and other constraints would achieve best quality. In this paper, we adopt such a model and develop an optimal rendition selection algorithm based on it. We report results by considering several different categories of receiving devices (HDTV, PCs, tablets and mobile) and show that optimal selections in all those cases will be considerably different.
Exploring the importance of pooling schemes for convolutional neural networks in hiragana character classification

This paper presents a comparison between the implementation of different convolutional neural network models varying the usage of pooling layers to address the problem of hiragana character classification. This study is focused on understanding how the selection and usage of different pooling layers affect the accuracy convergence in a model. To assess this situation eight models were tested with different configurations and using minimum pooling, average pooling, and max pooling schemes. Experimental results to validate the analysis and implementation are provided.
Private key and password protection by steganographic image encryption

We propose a technique to protect and preserve a private key or a passcode in an encrypted two-dimensional graphical image. The plaintext private key or the passcode is converted into an encrypted QR code and embedded into a real-life color image with a steganographic scheme. The private key or the passcode is recovered from the stego color image by first extracting the encrypted QR code from the color image, followed by decryption of the QR code. The cryptographic key for encryption of the QR code is generated from the output of a Linear Feedback Shift Register (LFSR), initialized by a seed image chosen by the user. The user can store the seed image securely, without the knowledge of an attacker. Even if an active attacker modifies the seed image (without knowledge of the fact that it is the seed image), the user can easily restore it if he/she keeps multiple copies of it, so that the encryption key can be regenerated easily. Our experiments prove the feasibility of the technique using sample private key data and real-life color images.
CLASSROOM: synthetic high dynamic range light field dataset

Light field images provide tremendous amounts of visual information regarding the represented scenes, as they describe the light traversing in all directions for all the points of 3D space. Due to the recent technological advancements of light field visualization and its increasing relevance in research, the need for light field image datasets has risen significantly. Among the applications for which light field datasets are considered, high dynamic range light field image reconstruction has gained notable attention in the past years. When capturing a scene, either a single camera with a 2D microlense array or a 2D array of cameras is used to produce narrow- and wide-baseline light field images, respectively. Additionally, the turn-table methodology may be used as well for narrow-baseline light fields. While the majority of these methods enables the creation of plausible and reliable light field image datasets, such baseline-specific setups can be extremely expensive and may require immense computing resources for proper calibration. Furthermore, the resulting light field is commonly limited with regard to angular resolution. A suitable alternative to produce a light field dataset is to do it synthetically by rendering light field images, which may easily overcome the aforementioned issues. In this paper, we discuss our work on creating the “CLASSROOM” light field image dataset, depicting a classroom scene. The content is rendered in horizontal-only parallax and full parallax as well. The scene contains a high variety of light distribution, particularly involving under-exposed and over-exposed regions, which are essential to HDR image applications.
Fast emulation of density functional theory simulations using approximate Gaussian processes

Fitting a theoretical model to experimental data in a Bayesian manner using Markov chain Monte Carlo typically requires one to evaluate the model thousands (or millions) of times. When the model is a slow-to-compute physics simulation, Bayesian model fitting becomes infeasible. To remedy this, a second statistical model that predicts the simulation output – an “emulator” – can be used in lieu of the full simulation during model fitting. A typical emulator of choice is the Gaussian process (GP), a flexible, non-linear model that provides both a predictive mean and variance at each input point. Gaussian process regression works well for small amounts of training data (n &lt; 10<sup>3</sup>), but becomes slow to train and use for prediction when the data set size becomes large. Various methods can be used to speed up the Gaussian process in the medium-to-large data set regime (n &lt; 10<sup>5</sup>), trading away predictive accuracy for drastically reduced runtime. This work examines the accuracy-runtime trade-off of several approximate Gaussian process models – the sparse variational GP, stochastic variational GP, and deep kernel learned GP – when emulating the predictions of density functional theory (DFT) models. Additionally, we use the emulators to calibrate, in a Bayesian manner, the DFT model parameters using observed data, resolving the computational barrier imposed by the data set size, and compare calibration results to previous work. The utility of these calibrated DFT models is to make predictions, based on observed data, about the properties of experimentally unobserved nuclides of interest e.g. super-heavy nuclei.
Unsupervised domain adaptation using feature aligned maximum classifier discrepancy

The maximum classifier discrepancy method has achieved great success in solving unsupervised domain adaptation tasks for image classification in recent years. Its basic structure consists of a feature generator and two classifiers that aim to maximize the classifier discrepancy while minimizing the generator discrepancy of the target samples. This method improves the performance of the existing adversarial training methods by employing task-specific classifiers that remove the ambiguity in classifying the target samples near the class boundaries. In this paper, we propose a modified network architecture and two training objectives to further boost the performance of the maximum classifier discrepancy method. The first training objective minimizes the feature level discrepancy and forces the generator to generate domain invariant features. This training objective is particularly beneficial when the source and the target domain distributions are vastly different. The second training objective that works at the mini-batch level aims at creating a uniform distribution of the target class predictions by maximizing the entropy of the expectation of the target class predictions. We show through extensive empirical evaluations that the proposed architecture and training objectives significantly improve the performance of the original algorithm. Furthermore, this method also outperforms the state-of-the-art techniques in most unsupervised domain adaptation tasks.
A beta-variational autoencoder (beta-VAE) to retrieve relevant degrees of freedom in a physical system

A popular framework to design optical components is inverse design. Here, one starts from a desired optical response and tries to find an optical design with exactly this response. In recent years, Machine Learning and in particular Deep Learning have been used in this regard. Many successes have been reported in the literature. What often arises in practice is the non-uniqueness problem. Multiple designs can lead to similar or even exactly the same optical responses. This is a complication, because most methods can only propose one design. The non-uniqueness problem can be mitigated by considering the degrees of freedom of the system. While multiple geometric parameters can correspond to the same response, the degrees of freedom define them uniquely. When working with these parameters, there is a one-to-one mapping between design and response. In order to find the degrees of freedom in an optical system, we propose a neural network architecture called a beta-VAE. We study its behaviour for analytical datasets of sine waves and damped harmonic oscillators. This gives us insight into the working of the neural network. We find that degrees of freedom can be retrieved for physical systems relevant to Optics.
A Person Re-identification Approach Focusing on the Occlusion Problem and Ranking Optimization

Person re-identification (Re-ID) aims to re-identify people across multiple video frames captured at various time instants. With the advancement of deep neural networks and increasing demand of intelligent video surveillance, recent works have proposed many deep learning based approaches to address the task. However, many state-of-the-arts Re-ID methods are not robust enough and do not perform well on the Synergy Re-Identification dataset provided by the DeepSportRadar Player Re-Identification Challenge due to the severe occlusion problem. To better re-identify basketball players, we propose a person Re-ID approach that focuses on the occlusion problem and ranking optimization. Specifically, our proposed approach consists of two stages. In the first stage we extract the global features and local features of the input image by two branches respectively. In the second stage, we propose a ranking optimization method that consists of three steps: k-reciprocal re-ranking, metric fusion, and distance mapping. We conduct extensive experiments to show that our proposed approach can achieve superior performance compared with the state-of-the-art methods and we also experimentally evaluate the training tricks and ranking optimization methods of existing Re-ID methods. Our proposed method achieves $98.38%$ mAP and $99.57%$ rank-1 on the challenge set of the Synergy Re-Identification dataset, and with this method our team, Fiery Tyrannosaurus Warrior, won the second place in the DeepSportRadar Player Re-Identification Challenge. Furthermore, we provide some ideas for potentially achieving better performance on the Synergy Re-Identification dataset.
Attention-Aware Multiple Granularities Network for Player Re-Identification

With the development of deep learning technologies, the performance of person re-identification (ReID) has been greatly improved. However, as a subdomain of person ReID, the research for player ReID is important for the sports field yet lacks sufficient effort so far. Player ReID aims to retrieve a specified player from a gallery of players' images captured by different cameras at various time steps. Compared with the traditional person ReID, player ReID suffers from various difficult problems, e.g., the high similarity of players' appearance, limited-scale datasets, variable and low image resolution, and severe occlusion. To solve such a challenging task, we propose a method named as Attention-Aware Multiple Granularities Network (A$^2$MGN), which consists of multiple branches to capture discriminative features of players from different granularities. Through the criterion of triplet loss and cross-entropy loss, the model can localize different parts of the player and make a comprehensive comparison between each pair of images. Extensive experiments demonstrate the effectiveness and superiority of our method, and our team (MM2022-bupt) achieves the top-3 in the challenge of ACM MMSports 2022 with mAP of 0.96, Rank-1 of 0.99, and Rank-5 of 1.00.
Quasi-continuously tunable non-volatile photonic switches

Phase-change materials (PCMs) integrated photonics are generating new paradigms, thanks to their zero static energy consumption, capability of switching reversibly, and drastic optical property contrast. However, electrically switching PCMs is inherently stochastic, hindering reliable multi-level or quasi-continuous operations. Here we propose that reliable quasi-continuous operation can be achieved in GST optical switches, where several GST patches are controlled in a binary fashion by interleaved PIN-diode doped silicon microheaters. We further experimentally demonstrate that the idea works in a tunable attenuator.
Topological quantum photonics

The field of topological photonics leverages concepts from geometry and topology – the branch of mathematics that deals with global properties that cannot be changed under continuous deformation – to produce electromagnetic modes that propagate with immunity to certain types of disorder and imperfections. In the last few years, several experiments have achieved the generation and propagation of quantum states of light in these so-called topological modes, demonstrating enhanced robustness of such inherently fragile quantum states. In this talk we will focus on our recent results in this area including topological protection of spatial entanglement and the demonstration of topology as an additional degree of freedom for entanglement using CMOS-compatible silicon photonics platforms.
Photometrically consistent projection experience for Metaverse screens everywhere

This paper presents a technical ensemble between a mobile projector and smartphone camera for providing colorimetrically calibrated realistic experience everywhere. Since portable projectors can be used in any places without a dedicated white screen, it is very essential to calibrate color of output videos no matter what a colored surface is used for a screen. That is, a calibration process is strongly required to realize the constant intended color experience while using mobile projectors. For that, we build an easy calibration process as follows. First, the (Android or iOS) mobile and projector are connected in a common local common network, and then a mobile application generates a specific electro-optical (RGB-XYZ) conversion function by capturing re ected light from a given specific white plate. The smartphone that plays the role of an optical measurement device transmits measured light information and calibration instruction to the display system. Then the projector adjusts its light outputs and notify the status to the mobile. This `measurement-adjustment' process is recursively conducted within few seconds. The process is completed when the adjustment result meets a stop condition for a calibration target on given colored wall screen. Based on our experimental results of photometric calibration using many kinds of recently released Android and iOS smartphones, the e effectiveness of the methods is proven.
Analysis of high dynamic range light field images in practical utilization contexts

High Dynamic Range (HDR) imaging has proven its importance in numerous applications over the recent years. In the industry, there is an evident strive to further enhance the quality of the captured contents. Accordingly, legacy Low Dynamic Range (LDR) images are currently being reconstructed as HDR images due to their importance and usage in so many use case contexts. Analogous to conventional LDR and HDR images, the HDR reconstruction of light field images has also become relevant within the scientific community. Although the procedure of reconstruction is similar in several ways, the creation of HDR light field contents poses a significantly greater challenge and suffers from obstacles that are not present in case of the 2D counterpart. In this paper, we provide a context-dependent analysis of HDR light field imaging. The investigated use cases include, but are not limited to industrial prototyping, medical applications, control systems, digital signage, exhibitions of cultural heritage, education, cinematography and communication. The work takes into consideration global illumination and rendering challenges. The topics of real-time systems and services, cost-efficient practices, content availability, baseline-specific considerations, apparatus-specific optical limitations, user interaction and general plausibility are additionally emphasized in the analysis. The paper also provides a set of recommendations regarding the use-case-specific requirements of the investigated practical contexts.
DATa: Domain Adaptation-aided deep Table detection using visual–lexical representations

Considerable research attention has been paid to table detection by developing not only rule-based approaches reliant on hand-crafted heuristics but also deep learning approaches. Although recent studies successfully perform table detection with enhanced results, they often experience performance degradation when they are used for transferred domains whose table layout features might differ from the source domain in which the underlying model has been trained. To overcome this problem, we present DATa, a novel Domain Adaptation-aided deep Table detection method that guarantees satisfactory performance in a specific target domain where few trusted labels are available. To this end, we newly design lexical features and an augmented model used for re-training. More specifically, after pre-training one of state-of-the-art vision-based models as our backbone network, we re-train our augmented model, consisting of the vision-based model and the multilayer perceptron (MLP) architecture. Using new confidence scores acquired based on the trained MLP architecture as well as an initial prediction of bounding boxes and their confidence scores, we calculate each confidence score more accurately. To validate the superiority of DATa, we perform experimental evaluations by adopting a real-world benchmark dataset in a source domain and another dataset in our target domain consisting of materials science articles. Experimental results demonstrate that the proposed DATa method substantially outperforms competing methods that only utilize visual representations in the target domain. Such gains are possible owing to the capability of eliminating high false positives or false negatives according to the setting of a confidence score threshold.
Replication of first-click eye tracking A/B test of webpage interactive elements

In this paper, we present a replication of our experiments of first click eye tracking A/B test of interactive website elements [13].The main difference between these two experiments is the equipment used; the first study was done using Tobi X-60 eye-tracker, while the replication described in this paper was conducted using Tobii TX 300, a higher frequency eye tracker.Eye tracking metrics used to evaluate the user experience of websites were almost identical.This paper presents the results of an experiment in which seven commercial websites were tested with the A/B first click test.This work examines the validity of a specific set of eye tracking metrics for their broad application in user experience research on websites.
An Integrated Checklist for Architecture Design of Critical Software Systems

With the advancement of digitalization, critical information infrastructures, such as intelligent energy distribution, transportation, or healthcare, have opened themselves towards intelligent technological opportunities, including automation of previously manual decision making.As a side effect, the digitalization of these infrastructures gives rise to new challenges, especially linked to the complexity of architecture design of these infrastructures, to later support necessary software quality and safeguard the systems against attacks and other harm.To support software architects in the design of these critical software systems, well structure architectural knowledge would be of great help to prevent the architects from missing some of the crucial concerns that need to be reflected with built-in architectural mechanisms, early during architecture design.Given the narrow scope of existing guidelines, with the need of browsing and combining multiple sources, this paper proposes an integrated checklist to cover the breath of architectural concerns for the design of critical software systems, covering the need for built-in mechanisms to prevent, detect, stop, recover from and analyse intentional as well as unintentional threats to system dependability.Contrary to existing guidelines that typically focus on runtime incident handling, our checklist is to be used during architecture design to ensure that the system has built-in mechanisms to either handle the incidents automatically or include the right mechanisms to support the runtime incident handling.
A matrix factorization recommendation model for tourism points of interest based on interest shift and differential privacy

Adding noise to user history data helps to protect user privacy in recommendation systems but affects the recommendation performance. To solve this problem, a matrix factorization tourism point of interest recommendation model based on interest offset and differential privacy is proposed in this paper. The recommendation performance of the model is improved by analyzing user interest preferences. Specifically, user interest offsets are extracted from user tags and user ratings under time-series factors to calculate user interest drift. Then, similar neighbors are found to train user feature preferences which are integrated into the matrix model in the form of regular terms. Meanwhile, based on the differential privacy theory, a privacy neighbor selection algorithm combining the K-Medoides clustering algorithm and index mechanism is designed to effectively protect the identity of neighbors and prevent KNN attacks. Besides, the Laplace mechanism is used to implement differential privacy protection for the model’s gradient descent process. Finally, the feasibility of the proposed recommendation model is verified through experiments, and the experimental results indicate that this model has advantages in recommendation accuracy and privacy protection.
COMPARATIVE INVESTIGATION OF NEURAL AND LOCALLY BINARY ALGORITHMS FOR IMAGE IDENTIFICATION OF PLANKTON POPULATIONS

The work is devoted to the «neural network-lbp» method of processing satellite images of multispectral water coastal systems for identification of phytoplankton populations of spotted structure: determination of their boundaries, distribution of color gradations and, based on this, determination of the distribution of phytoplankton concentrations inside the spots and the location of the center of mass. The efficiency and effectiveness of the proposed neural network-lbp method is investigated in comparison with the method based on the use of a three-layer neural network. For the analysis, a test set of images is involved – flat shapes with rather complex boundaries, which will allow us to quantify the quality of the algorithms being compared. The results of the work show an increase in recognition accuracy by 1.5-3% when using the proposed method.