OpenRaiser/Intern-Atlas huggingface dataset:
Multiscale Governance
Venue: arXiv cs.CY 2021
Year: 2021

Future societal systems will be characterized by heterogeneous human behaviors and also collective action. The interaction between local systems and global systems will be complex. Humandemics will propagate because of the pathways that connect the different systems and several invariant behaviors and patterns that have emerged globally. On the contrary, infodemics of misinformation can be a risk as it has occurred in the COVID-19 pandemic. The emerging fragility or robustness of the system will depend on how this complex network of systems is governed. Future societal systems will not be only multiscale in terms of the social dimension, but also in the temporality. Necessary and proper prevention and response systems based on complexity, ethic and multi-scale governance will be required. Real-time response systems are the basis for resilience to be the foundation of robust societies. A top-down approach led by Governmental organs for managing humandemics is not sufficient and may be only effective if policies are very restrictive and their efficacy depends not only in the measures implemented but also on the dynamics of the policies and the population perception and compliance. This top-down approach is even weaker if there is not national and international coordination. Coordinating top-down agencies with bottom-up constructs will be the design principle. Multi-scale governance integrates decision-making processes with signaling, sensing and leadership mechanisms to drive thriving societal systems with real-time sensitivity.
A fully automated end-to-end process for fluorescence microscopy images of yeast cells: From segmentation to detection and classification
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, an enormous amount of fluorescence microscopy images were collected in high-throughput lab settings. Analyzing and extracting relevant information from all images in a short time is almost impossible. Detecting tiny individual cell compartments is one of many challenges faced by biologists. This paper aims at solving this problem by building an end-to-end process that employs methods from the deep learning field to automatically segment, detect and classify cell compartments of fluorescence microscopy images of yeast cells. With this intention we used Mask R-CNN to automatically segment and label a large amount of yeast cell data, and YOLOv4 to automatically detect and classify individual yeast cell compartments from these images. This fully automated end-to-end process is intended to be integrated into an interactive e-Science server in the PerICo1 project, which can be used by biologists with minimized human effort in training and operation to complete their various classification tasks. In addition, we evaluated the detection and classification performance of state-of-the-art YOLOv4 on data from the NOP1pr-GFP-SWAT yeast-cell data library. Experimental results show that by dividing original images into 4 quadrants YOLOv4 outputs good detection and classification results with an F1-score of 98% in terms of accuracy and speed, which is optimally suited for the native resolution of the microscope and current GPU memory sizes. Although the application domain is optical microscopy in yeast cells, the method is also applicable to multiple-cell images in medical applications
C2CL: Contact to Contactless Fingerprint Matching
Venue: arXiv cs.CV 2021
Year: 2021

Matching contactless fingerprints or finger photos to contact-based fingerprint impressions has received increased attention in the wake of COVID-19 due to the superior hygiene of the contactless acquisition and the widespread availability of low cost mobile phones capable of capturing photos of fingerprints with sufficient resolution for verification purposes. This paper presents an end-to-end automated system, called C2CL, comprised of a mobile finger photo capture app, preprocessing, and matching algorithms to handle the challenges inhibiting previous cross-matching methods; namely i) low ridge-valley contrast of contactless fingerprints, ii) varying roll, pitch, yaw, and distance of the finger to the camera, iii) non-linear distortion of contact-based fingerprints, and vi) different image qualities of smartphone cameras. Our preprocessing algorithm segments, enhances, scales, and unwarps contactless fingerprints, while our matching algorithm extracts both minutiae and texture representations. A sequestered dataset of 9,888 contactless 2D fingerprints and corresponding contact-based fingerprints from 206 subjects (2 thumbs and 2 index fingers for each subject) acquired using our mobile capture app is used to evaluate the cross-database performance of our proposed algorithm. Furthermore, additional experimental results on 3 publicly available datasets show substantial improvement in the state-of-the-art for contact to contactless fingerprint matching (TAR in the range of 96.67% to 98.30% at FAR=0.01%).
LI-Net: Large-Pose Identity-Preserving Face Reenactment Network
Venue: arXiv cs.CV 2021
Year: 2021

Face reenactment is a challenging task, as it is difficult to maintain accurate expression, pose and identity simultaneously. Most existing methods directly apply driving facial landmarks to reenact source faces and ignore the intrinsic gap between two identities, resulting in the identity mismatch issue. Besides, they neglect the entanglement of expression and pose features when encoding driving faces, leading to inaccurate expressions and visual artifacts on large-pose reenacted faces. To address these problems, we propose a Large-pose Identity-preserving face reenactment network, LI-Net. Specifically, the Landmark Transformer is adopted to adjust driving landmark images, which aims to narrow the identity gap between driving and source landmark images. Then the Face Rotation Module and the Expression Enhancing Generator decouple the transformed landmark image into pose and expression features, and reenact those attributes separately to generate identity-preserving faces with accurate expressions and poses. Both qualitative and quantitative experimental results demonstrate the superiority of our method.
Facial Attribute Transformers for Precise and Robust Makeup Transfer
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we address the problem of makeup transfer, which aims at transplanting the makeup from the reference face to the source face while preserving the identity of the source. Existing makeup transfer methods have made notable progress in generating realistic makeup faces, but do not perform well in terms of color fidelity and spatial transformation. To tackle these issues, we propose a novel Facial Attribute Transformer (FAT) and its variant Spatial FAT for high-quality makeup transfer. Drawing inspirations from the Transformer in NLP, FAT is able to model the semantic correspondences and interactions between the source face and reference face, and then precisely estimate and transfer the facial attributes. To further facilitate shape deformation and transformation of facial parts, we also integrate thin plate splines (TPS) into FAT, thus creating Spatial FAT, which is the first method that can transfer geometric attributes in addition to color and texture. Extensive qualitative and quantitative experiments demonstrate the effectiveness and superiority of our proposed FATs in the following aspects: (1) ensuring high-fidelity color transfer; (2) allowing for geometric transformation of facial parts; (3) handling facial variations (such as poses and shadows) and (4) supporting high-resolution face generation.
Intelligent Reflecting Surface Assisted Terahertz Communications toward 6G
Venue: arXiv cs.IT 2021
Year: 2021

Terahertz (THz) communications have emerged as a promising candidate to support the heavy data traffic and exploding network capacity in the future 6G wireless networks. However, THz communications are facing many challenges for practical implementation, such as propagation loss, signal blockage, and hardware cost. In this article, an emerging paradigm of intelligent reflecting surface (IRS) assisted THz communications is analyzed, to address the above issues, by leveraging the joint active and passive beamforming to enhance the communication quality and reduce overheads. Aiming at practical implementation, an overview of the currently available approaches of realizing THz active/passive beam steering at transmitter and IRS is presented. Based on these approaches, a beam training strategy for establishing joint beamforming is then investigated in THz communications. Moreover, various emerging and appealing 6G scenarios that integrate IRS into THz communications are envisioned. Open challenges and future research directions for this new paradigm are finally highlighted.
Ethical User Interfaces: Exploring the Effects of Dark Patterns on Facebook
Venue: arXiv cs.HC 2021
Year: 2021

Many researchers have been concerned with whether social media has a negative impact on the well-being of their audience. With the popularity of social networking sites (SNS) steadily increasing, psychological and social sciences have shown great interest in their effects and consequences on humans. In this work, we investigate Facebook using the tools of HCI to find connections between interface features and the concerns raised by these domains. Using an empirical design analysis, we identify interface interferences impacting users' online privacy. Through a subsequent survey (n=116), we find usage behaviour changes due to increased privacy concerns and report individual cases of addiction and mental health issues. These observations are the results of a rapidly changing SNS creating a gap of understanding between users' interactions with the platform and future consequences. We explore how HCI can help close this gap and work towards more ethical user interfaces in the future.
RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we study the compositional learning of images and texts for image retrieval. The query is given in the form of an image and text that describes the desired modifications to the image; the goal is to retrieve the target image that satisfies the given modifications and resembles the query by composing information in both the text and image modalities. To remedy this, we propose a novel architecture designed for the image-text composition task and show that the proposed structure can effectively encode the differences between the source and target images conditioned on the text. Furthermore, we introduce a new joint training technique based on the graph convolutional network that is generally applicable for any existing composition methods in a plug-and-play manner. We found that the proposed technique consistently improves performance and achieves state-of-the-art scores on various benchmarks. To avoid misleading experimental results caused by trivial training hyper-parameters, we reproduce all individual baselines and train models with a unified training environment. We expect this approach to suppress undesirable effects from irrelevant components and emphasize the image-text composition module's ability. Also, we achieve the state-of-the-art score without restricting the training environment, which implies the superiority of our method considering the gains from hyper-parameter tuning. The code, including all the baseline methods, are released https://github.com/nashory/rtic-gcn-pytorch.
Scaling Scaling Laws with Board Games
Venue: arXiv cs.LG 2021
Year: 2021

The largest experiments in machine learning now require resources far beyond the budget of all but a few institutions. Fortunately, it has recently been shown that the results of these huge experiments can often be extrapolated from the results of a sequence of far smaller, cheaper experiments. In this work, we show that not only can the extrapolation be done based on the size of the model, but on the size of the problem as well. By conducting a sequence of experiments using AlphaZero and Hex, we show that the performance achievable with a fixed amount of compute degrades predictably as the game gets larger and harder. Along with our main result, we further show that the test-time and train-time compute available to an agent can be traded off while maintaining performance.
AI perspectives in Smart Cities and Communities to enable road vehicle automation and smart traffic control
Venue: arXiv cs.AI 2021
Year: 2021

Smart Cities and Communities (SCC) constitute a new paradigm in urban development. SCC ideates on a data-centered society aiming at improving efficiency by automating and optimizing activities and utilities. Information and communication technology along with internet of things enables data collection and with the help of artificial intelligence (AI) situation awareness can be obtained to feed the SCC actors with enriched knowledge. This paper describes AI perspectives in SCC and gives an overview of AI-based technologies used in traffic to enable road vehicle automation and smart traffic control. Perception, Smart Traffic Control and Driver Modelling are described along with open research challenges and standardization to help introduce advanced driver assistance systems and automated vehicle functionality in traffic. To fully realize the potential of SCC, to create a holistic view on a city level, the availability of data from different stakeholders is need. Further, though AI technologies provide accurate predictions and classifications there is an ambiguity regarding the correctness of their outputs. This can make it difficult for the human operator to trust the system. Today there are no methods that can be used to match function requirements with the level of detail in data annotation in order to train an accurate model. Another challenge related to trust is explainability, while the models have difficulties explaining how they come to a certain conclusion it is difficult for humans to trust it.
The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods
Venue: arXiv cs.CV 2021
Year: 2021

For an autonomous robotic system, monitoring surgeon actions and assisting the main surgeon during a procedure can be very challenging. The challenges come from the peculiar structure of the surgical scene, the greater similarity in appearance of actions performed via tools in a cavity compared to, say, human actions in unconstrained environments, as well as from the motion of the endoscopic camera. This paper presents ESAD, the first large-scale dataset designed to tackle the problem of surgeon action detection in endoscopic minimally invasive surgery. ESAD aims at contributing to increase the effectiveness and reliability of surgical assistant robots by realistically testing their awareness of the actions performed by a surgeon. The dataset provides bounding box annotation for 21 action classes on real endoscopic video frames captured during prostatectomy, and was used as the basis of a recent MIDL 2020 challenge. We also present an analysis of the dataset conducted using the baseline model which was released as part of the challenge, and a description of the top performing models submitted to the challenge together with the results they obtained. This study provides significant insight into what approaches can be effective and can be extended further. We believe that ESAD will serve in the future as a useful benchmark for all researchers active in surgeon action detection and assistive robotics at large.
A Preliminary Proposal for an Analytical Model for Evaluating the Impact on Performance of Data Access Patterns in Transaction Execution
Venue: arXiv cs.PF 2021
Year: 2021

We present a preliminary proposal for an analytical model for evaluating the impact on performance of data access patterns in concurrent transaction execution. We consider the case of concurrency control protocols that use locking to ensure isolation in the execution of transactions. We analyse scenarios where transactions access one or more sets of data items in the same order or in different order.
Analysis of Twitter Users' Lifestyle Choices using Joint Embedding Model
Venue: arXiv cs.CL 2021
Year: 2021

Multiview representation learning of data can help construct coherent and contextualized users' representations on social media. This paper suggests a joint embedding model, incorporating users' social and textual information to learn contextualized user representations used for understanding their lifestyle choices. We apply our model to tweets related to two lifestyle activities, `Yoga' and `Keto diet' and use it to analyze users' activity type and motivation. We explain the data collection and annotation process in detail and provide an in-depth analysis of users from different classes based on their Twitter content. Our experiments show that our model results in performance improvements in both domains.
Learned transform compression with optimized entropy encoding
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of learned transform compression where we learn both, the transform as well as the probability distribution over the discrete codes. We utilize a soft relaxation of the quantization operation to allow for back-propagation of gradients and employ vector (rather than scalar) quantization of the latent codes. Furthermore, we apply similar relaxation in the code probability assignments enabling direct optimization of the code entropy. To the best of our knowledge, this approach is completely novel. We conduct a set of proof-of concept experiments confirming the potency of our approaches.
Streaming Self-Training via Domain-Agnostic Unlabeled Images
Venue: arXiv cs.CV 2021
Year: 2021

We present streaming self-training (SST) that aims to democratize the process of learning visual recognition models such that a non-expert user can define a new task depending on their needs via a few labeled examples and minimal domain knowledge. Key to SST are two crucial observations: (1) domain-agnostic unlabeled images enable us to learn better models with a few labeled examples without any additional knowledge or supervision; and (2) learning is a continuous process and can be done by constructing a schedule of learning updates that iterates between pre-training on novel segments of the streams of unlabeled data, and fine-tuning on the small and fixed labeled dataset. This allows SST to overcome the need for a large number of domain-specific labeled and unlabeled examples, exorbitant computational resources, and domain/task-specific knowledge. In this setting, classical semi-supervised approaches require a large amount of domain-specific labeled and unlabeled examples, immense resources to process data, and expert knowledge of a particular task. Due to these reasons, semi-supervised learning has been restricted to a few places that can house required computational and human resources. In this work, we overcome these challenges and demonstrate our findings for a wide range of visual recognition tasks including fine-grained image classification, surface normal estimation, and semantic segmentation. We also demonstrate our findings for diverse domains including medical, satellite, and agricultural imagery, where there does not exist a large amount of labeled or unlabeled data.
Spotify at TREC 2020: Genre-Aware Abstractive Podcast Summarization
Venue: arXiv cs.CL 2021
Year: 2021

This paper contains the description of our submissions to the summarization task of the Podcast Track in TREC (the Text REtrieval Conference) 2020. The goal of this challenge was to generate short, informative summaries that contain the key information present in a podcast episode using automatically generated transcripts of the podcast audio. Since podcasts vary with respect to their genre, topic, and granularity of information, we propose two summarization models that explicitly take genre and named entities into consideration in order to generate summaries appropriate to the style of the podcasts. Our models are abstractive, and supervised using creator-provided descriptions as ground truth summaries. The results of the submitted summaries show that our best model achieves an aggregate quality score of 1.58 in comparison to the creator descriptions and a baseline abstractive system which both score 1.49 (an improvement of 9%) as assessed by human evaluators.
Optimal Resource Allocation Design for Large IRS-Assisted SWIPT Systems: A Scalable Optimization Framework
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we study the optimal resource allocation algorithm design for large intelligent reflecting surface (IRS)-assisted simultaneous wireless information and power transfer (SWIPT) systems. To facilitate efficient system design for large IRSs, instead of jointly optimizing all the IRS elements, we partition the IRS into several tiles and employ a scalable optimization framework comprising an offline design stage and an online optimization stage. In the offline stage, the IRS elements of each tile are jointly designed to support a set of different phase shift configurations, referred to as transmission modes, while the best transmission mode is selected from the set for each tile in the online stage. Given a transmission mode set, we aim to minimize the total base station (BS) transmit power by jointly optimizing the beamforming and the transmission mode selection policy taking into account the quality-of-service requirements of information decoding and non-linear energy harvesting receivers, respectively. Although the resource allocation algorithm design is formulated as a non-convex combinatorial optimization problem, we solve it optimally by applying the branch-and-bound (BnB) approach which entails a high computational complexity. To strike a balance between optimality and computational complexity, we also develop an efficient suboptimal algorithm capitalizing on the penalty method and successive convex approximation. Our simulation results show that the proposed designs enable considerable power savings compared to several baseline schemes. Moreover, our results reveal that by properly adjusting the numbers of tiles and transmission modes, the proposed scalable optimization framework indeed facilitates online design for large IRSs.
Monitoring Social-distance in Wide Areas during Pandemics: a Density Map and Segmentation Approach
Venue: arXiv cs.CV 2021
Year: 2021

With the relaxation of the containment measurements around the globe, monitoring the social distancing in crowded public places is of grate importance to prevent a new massive wave of COVID-19 infections. Recent works in that matter have limited themselves by detecting social distancing in corridors up to small crowds by detecting each person individually considering the full body in the image. In this work, we propose a new framework for monitoring the social-distance using end-to-end Deep Learning, to detect crowds violating the social-distance in wide areas where important occlusions may be present. Our framework consists in the creation of a new ground truth based on the ground truth density maps and the proposal of two different solutions, a density-map-based and a segmentation-based, to detect the crowds violating the social-distance constrain. We assess the results of both approaches by using the generated ground truth from the PET2009 and CityStreet datasets. We show that our framework performs well at providing the zones where people are not following the social-distance even when heavily occluded or far away from one camera.