Miracle-dz/UniFAR-FTUs huggingface dataset:
Location Estimation of an Urban Scene using Computer Vision Techniques

: The process of adding the geographical identiﬁcation data to an image is called geotagging and is important for a range of applications starting from tourism to law enforcement agencies. The most convenient way of adding location metadata to an image is GPS geotagging. This article presents an alternative way of adding the approximate location metadata to an urban scene image by ﬁnding similar images in a dataset of geotagged images. The matching is done by extracting the image features and descriptors and matching them. The dataset consists in geotagged 360 ◦ panoramic images. We explored three methods of matching the images, each one being an iteration of the previous method. The ﬁrst method used only feature detection and matching using AKAZE and FLANN, the second method performed image segmentation to provide a mask for extracting features and descriptors only from buildings and the third method preprocessed the dataset to obtain better accuracy. We managed to improve the accuracy of the system by 25%. Following the in-depth analysis of the results we will present the results as well as future improvements.
Improving Neural Network-based Multidimensional Projections

Dimensionality reduction methods are often used to explore multidimensional data in data science and information visualization. Techniques of the SNE-class, such as t-SNE, have become the standard for data exploration due to their good visual cluster separation, but are computationally expensive and don’t have out-of-sample capability by default. Recently, a neural network-based technique was proposed, which adds out-of-sample capability to t-SNE with good results, but with the disavantage of introducing some diffusion of the points in the result. In this paper we evaluate many neural network-tuning strategies to improve the results of this technique. We show that a careful selection of network architecture, loss function and data augmentation strategy can improve results.
Self-supervised Depth Estimation based on Feature Sharing and Consistency Constraints

: In this work, we propose a self-supervised approach to depth estimation. Our method uses depth consistency to generate soft visibility mask that reduces the error contribution of inconsistent regions produced by occlusions. In addition, we allow the pose network to take advantage of the depth network representations to produce more accurate results. The experiments are conducted on the KITTI 2015 dataset. We analyze the effect of each component in the performance of the model and demonstrate that the consistency constraint and feature sharing can effectively improve our results. We show that our method is competitive when compared to the state of the art.
Affine Transformation from Fundamental Matrix and Two Directions

: Researchers have recently shown that afﬁne transformations between corresponding patches of two images can be applied for 3D reconstruction, including the reconstruction of surface normals. However, the accurate estimation of afﬁne transformations between image patches is very challenging. This paper mainly proposes a novel method to estimate afﬁne transformations from two directions if epipolar geometry of the image pair is known. A reconstruction pipeline is also proposed here in short. As side effects, two proofs are also given. The ﬁrst one is to determine the relationship between afﬁne transformations and the fundamental matrix, the second one shows how optimal surface normal estimation can be obtained via the roots of a cubic polynomial. A visual debugger is also proposed to validate the estimated surface normals in real images.
Time-series Visualization of Twitter Trends

: Twitter provides a function called “trend” that presents popular words and hashtags. Typically, one trend word or hashtag is related to thousands of tweets. It is difficult to understand such thousands of tweets in a short time by using the standard sort methods and the standard display method provided by Twitter. Most of previous studies analyzed and visualized tweets by using text-based clustering methods. However, these methods suffer from the accuracy of clustering results, because a typical tweet has only poor textual information. This paper presents a Twitter trend analysis system that combines retweet clustering and time-series visualization to allow users to understand flows of topics in a Twitter trend in a short time. This system also provides a list of effective legends and a display of individual tweets with photos in order for users to further understand topics in a trend. To illustrate the effectiveness of this system, this paper presents the results of experiments on the analysis of Twitter trends related to a popular sport event and a popular music program.
Superpoints in RANSAC Planes: A New Approach for Ground Surface Extraction Exemplified on Point Classification and Context-aware Reconstruction

: In point clouds obtained from airborne data, the ground points have traditionally been identiﬁed as local minima of the altitude. Subsequently, the 2.5D digital terrain models have been computed by approximation of a smooth surfaces from the ground points. But how can we handle purely 3D surfaces of cultural heritage monuments covered by vegetation or Alpine overhangs, where trees are not necessarily growing in bottom-to-top direction? We suggest a new approach based on a combination of superpoints and RANSAC implemented as a ﬁltering procedure, which allows efﬁcient handling of large, challenging point clouds without necessity of training data. If training data is available, covariance-based features, point histogram features, and dataset-dependent features as well as combinations thereof are applied to classify points. Results achieved with a Random Forest classiﬁer and non-local optimization using Markov Random Fields are analyzed for two challenging datasets: an airborne laser scan and a photogrammetrically reconstructed point cloud. As an application, surface reconstruction from the thus cleaned point sets is demonstrated.
Towards Automatic CAD Modeling from 3D Scan Sketch based Representation

: This paper proposes a novel approach to convert a 3D scan to its CAD counterpart. The objective is to extract intermediate sketch planes that well represent the input scan and are close enough to the original design intent. These sketches can then be easily converted into CAD models automatically due to their faithful representation of the input geometry. One objective is to avoid incorporating user/company dependent content in the CAD reconstruction process. The intermediate representation shall be directly supported in any CAD envi-ronment to boost the designer’s work without the need of supplementary (model conversion, automatic feature recognition) steps. Nowadays, it is common to digitize an object and reconstruct its geometric primitives. However, this reconstruction contains only geometry. In literature, the ﬁnal goal might be met by recovering the modeling tree itself, by means of automatic feature recognition, and converting to the proper format of a speciﬁc CAD software package. However, the constructed tree and its conversion introduce issues in the reconstruction process. The deﬁnition of an exact modeling tree, and the production of a meaningful ﬁnal CAD model are rather hard to obtain. This imposes a rather inefﬁcient working method, thereby heavily impacting the designer’s modeling skills.
Distributed Information Integration in Convolutional Neural Networks

: A large body of physiological ﬁndings has suggested the vision system understands a scene in terms of its local features such as lines and curves. A highly notable computer algorithm developed that models such behaviour is the Convolutional Neural Network (CNN). Whilst recognising an object in various scales remains trivial for the human vision system, CNNs struggle to achieve the same behaviour. Recent physiological ﬁndings are suggesting two new paradigms. Firstly, the visual system uses both local and global features in its recognition function. Secondly, the brain uses a distributed processing architecture to learn information from multiple modalities. In this paper we combine these paradigms and propose a distributed information integration model called D-Net to improve scale-invariant classiﬁcation of images. We use a CNN to extract local features and, inspired by Google’s INCEPTION model, develop a trainable method using ﬁlter pyramids to extract global features called Filter Pyramid Convolutions (FPC). D-Net locally processes CNN and FPC features, fuses the outcomes and obtains a global estimate via the central processor. We test D-Net on classiﬁcation of scaled images on benchmark datasets. Our results show D-Net’s potential effectiveness towards classiﬁcation of scaled images.
Cooperative Stereo-Zoom Matching for Disparity Computation

: This paper investigates a stereo matching approach incorporated with the zooming information. Conventional stereo vision algorithms take one pair of images for correspondence matching, while our proposed method adopts two zoom lens cameras to acquire multiple stereo image pairs with zoom changes. These image sequences are able to provide more accurate results for stereo matching algorithms. The new framework makes the rectiﬁed images compliant with the zoom characteristics by the deﬁnition of the relationship between the left and right images. Our approach can be integrated with existing stereo matching algorithms under some requirements and adjustments. In the experiments, we test the proposed framework on 2014 Middle-bury benchmark dataset and our own zoom image dataset. The results have demonstrated the improvement of disparity computation of the our technique.
On the Preference for Travel by Steering in a Virtual Reality Game

: Travel is one of the most important tasks in virtual reality (VR) experiences. Paradoxically, the most popular travel techniques in virtual reality games are known to be more likely to cause cybersickness than some of the less popular travel techniques. Recently, at least one VR gaming company shared quantitative data on this issue. In an attempt to explain this data, this work argues that steering techniques might result in stronger immersion, better physical ergonomics, and more pleasure than offered by teleportation techniques. Furthermore, trends are identiﬁed that might reduce the preference for steering techniques in the future. The presented discussion of current and future preferences for steering techniques in VR games might help to better understand and design for the needs of VR players.
Irosashi: Visualization of the Colors of a Building Which Leave an Impression to Identify Characteristics of an Urban Environment

: The purpose of this research is to propose a GUI for visualizing data showing the characteristics of the city and evaluate its effect. The author was interested in visual impressions of the urban environment and wanted to share these impressions with others. First, I made a prototype of "Irosashi Treemap", which plots all colors used for buildings in the western part of Hakodate city. However, the colors displayed by the Irosashi Treemap differed from the impressions of the subjects who knew the western area due to the many achromatic colors. Therefore, I collected color from the triangular roof and walls on the first floor of the building I felt frequent. In addition, it was suggested that the "Irosashi Treemap" plotted all colors, making it difficult to understand the characteristics of the colors. Therefore, a prototype of "Irosashi Impression", which plots representative colors, was produced. In evaluation experiment, three subjects answered that the impression of the city was close to the impression of the color taken from the triangular roof and walls. In the future, we will ask many people to collect impressive colors of the city. And evaluate the effect of "Irosashi Impression".
Detecting Anomalous Regions from an Image based on Deep Captioning

: In this paper we propose a one-class anomalous region detection method from an image based on deep captioning. Such a method can be installed on an autonomous mobile robot, which reports anomalies from observation without any human supervision and would interest a wide range of researchers, practitioners, and users. In addition to image features, which were used by conventional methods, our method exploits recent advances in deep captioning, which is based on deep neural networks trained on a large-scale data on image - caption pairs, enabling anomaly detection in the semantic level. Incremental clustering is adopted so that the robot is able to model its observation into a set of clusters and report substantially new observations as anomalies. Extensive experiments using two real-world data demonstrate the superiority of our method in terms of recall, precision, F measure, and AUC over the traditional approach. The experiments also show that our method exhibits excellent learning curve and low threshold dependency.
Classification of Histopathological Images using Scale-Invariant Feature Transform

: Throughout the years, Scale-Invariant Feature Transform (SIFT) was a widely adopted method in the image matching and classiﬁcation tasks. However, due to the recent advances in convolutional neural networks, the popularity of SIFT and other similar feature descriptors signiﬁcantly decreased, leaving SIFT underresearched in some of the emerging applications. In this paper we examine the suitability of SIFT feature descriptors in one such task, the histopathological image classiﬁcation. In the conducted experimental study we investigate the usefulness of various variants of SIFT on the BreakHis Breast Cancer Histopathological Database. While colour is known to be signiﬁcant in case of human performed analysis of histopathological images, SIFT variants using different colour spaces have not been thoroughly examined on this type of data before. Observed results indicate the effectiveness of selected SIFT variants, particularly Hue-SIFT, which outperformed the reference convolutional neural network ensemble on some of the considered magniﬁcations, simultaneously achieving lower variance. This proves the importance of using different colour spaces in classiﬁcation tasks with histopathological data and shows promise to ﬁnd its use in diversifying classiﬁer ensembles
Graph based Method for Online Handwritten Character Recognition

: In this research, we attempt to propose a novel graph-based approach for online handwritten character recognition. Unlike the most well-known online handwritten recognition methods, which are based on statistical representations, we set forward a new approach based on structural representation to overcome the inherent deformations of handwritten characters. An Attributed Relational Graph (ARG) is dedicated to allowing the direct labeling of nodes (strokes) and edges (relationships) of a graph to model the input character. Each node is characterized by a set of fuzzy membership degrees describing their properties (type, size). Fuzzy description is invested in order to guarantee more robustness against uncertainty, ambiguity and vagueness. ARGs edges stand for spatial relationships between different strokes. At a subsequent stage, a tree-search based optimal matching algorithm is explored, which allows the search for character structures i.e the minimum cost of nodes. Experiments performed on ADAB and IRONOFF datasets, reveal promising results. In particular, the comparison with the state of the art demonstrates the signiﬁcance of the proposed system.
Proxy Embeddings for Face Identification among Multi-Pose Templates

: Many of a large scale face identiﬁcation systems operates on databases containing images showing heads in multiple poses (from frontal to full proﬁles). However, as it was shown in the paper, off-the-shelf methods are not able to take advantage of this particular data structure. The main idea behind our work was to adapt the methods proposed for multi-view and semi-3D objects classiﬁcation to the multi-pose face recognition problem. The proposed approach involves neural network training with proxy embeddings and building the gallery templates out of aggregated samples. A benchmark testing scenario is proposed for the purpose of the problem, which is based on the linked gallery and probes databases. The gallery database consists of multi-pose face images taken under controlled conditions, and the probes database contains samples of in-the-wild type. Both databases must be linked, having at least partially common labels. Two variants of the proposed training procedures were tested, namely, the neighbourhood component analysis with proxies (NCA-proxies) and the triplet margin loss with proxies (triplet-proxies). It is shown that the proposed methods perform better than models trained with cross-entropy loss and than off-the-shelf methods. Rank-1 accuracy was improved from 48.82% for off-the-shelf baseline to 86.86% for NCA-proxies. In addition, transfer of proxy points between two independently trained models was discussed, similarly to hyper-parameters transfer methodology. Proxy embeddings transfer opens a possibility of training two domain-speciﬁc networks with respect to two datasets identiﬁcation schema.
Semantic Segmentation using Light Attention Mechanism

: Semantic segmentation using convolutional neural networks (CNN) can be applied to various fields such as automatic driving. Semantic segmentation is pixel-wise class classification, and various methods using CNN have been proposed. We introduce a light attention mechanism to the encoder-decoder network. The network that introduced a light attention mechanism pays attention to features extracted during training, emphasizes the features judged to be effective for training and suppresses the features judged to be irrelevant for each pixel. As a result, training can be performed by focusing on only necessary features. We evaluated the proposed method using the CamVid dataset and obtained higher accuracy than conventional segmentation methods.
Theorems for free for free: parametricity, with and without types

The polymorphic blame calculus integrates static typing, including universal types, with dynamic typing. The primary challenge with this integration is preserving parametricity: even dynamically-typed code should satisfy it once it has been cast to a universal type. Ahmed et al. (2011) employ runtime type generation in the polymorphic blame calculus to preserve parametricity, but a proof that it does so has been elusive. Matthews and Ahmed (2008) gave a proof of parametricity for a closely related system that combines ML and Scheme, but later found a flaw in their proof. In this paper we present an improved version of the polymorphic blame calculus and we prove that it satisfies relational parametricity. The proof relies on a step-indexed Kripke logical relation. The step-indexing is required to make the logical relation well-defined in the case for the dynamic type. The possible worlds include the mapping of generated type names to their types and the mapping of type names to relations. We prove the Fundamental Property of this logical relation and that it is sound with respect to contextual equivalence. To demonstrate the utility of parametricity in the polymorphic blame calculus, we derive two free theorems.
Towards Keypoint Guided Self-Supervised Depth Estimation

This paper proposes to use keypoints as a self-supervision clue for learning depth map estimation from a collection of input images. As ground truth depth from real images is difficult to obtain, there are many unsupervised and self-supervised approaches to depth estimation that have been proposed. Most of these unsupervised approaches use depth map and ego-motion estimations to reproject the pixels from the current image into the adjacent image from the image collection. Depth and ego-motion estimations are evaluated based on pixel intensity differences between the correspondent original and reprojected pixels. Instead of reprojecting the individual pixels, we propose to first select image keypoints in both images and then reproject and compare the correspondent keypoints of the two images. The keypoints should describe the distinctive image features well. By learning a deep model with and without the keypoint extraction technique, we show that using the keypoints improve the depth estimation learning. We also propose some future directions for keypoint-guided learning of structure-from-motion problems.
Fast Local LUT Upsampling

: Edge-preserving ﬁlters have been used in various applications in image processing. As the number of pixels of digital cameras has been increasing, the computational cost becomes higher, since the order of the ﬁlters depends on the image size. There are several acceleration approaches for the edge-preserving ﬁltering; how-ever, most approaches reduce the dependency of ﬁltering kernel size to the processing time. In this paper, we propose a method to accelerate the edge-preserving ﬁlters for high-resolution images. The method subsam-ples an input image and then performs the edge-preserving ﬁltering on the subsampled image. Our method then upsamples the subsampled image with the guidance, which is the high-resolution input images. For this upsampling, we generate per-pixel LUTs for high-precision upsampling. Experimental results show that the proposed method has higher performance than the conventional approaches.
Image-based Classification of Swiss Traditional Costumes using Contextual Features

: In this work we propose a method for feature-based clothing recognition, and prove its applicability by performing image-based recognition of Swiss traditional costumes. We employ an estimation of a simpliﬁed human skeleton (a poselet) to extract visually indistinguishable but reproducible features. The descriptors of those features are constructed, while accounting for possible displacement of clothes along human body. The similarity metrics mean squared error and correlation coefﬁcient are surveyed, and color spaces YIQ and CIELAB are investigated for their ability to isolate scene brightness in a separate channel. We show that the model trained with mean squared error performs best in the CIELAB color space and achieves an F 0 . 5 -score of 0.77. Furthermore, we show that omission of the brightness channel produces less biased, but overall poorer descriptors.