Reviewerly/exHarmony huggingface dataset:
Critical Digital: A Taxonomy to Classify Digital Integration in the Museum Domain

The evolution of the information and communication ecosystem has led to significant changes in the museum visitor’s behavior and expectations, as they now manifest a desire for participation, collaboration, and sharing – suggesting the corresponding need to readjust the institution’s discourse and communication approaches through digital integration. Despite the inherent qualities of digital media, they can be presented in various platforms, interfaces, and devices, contributing to the lack of consensus and consistency in the terminology used in academia and among designers, technologists, programmers, and developers, resulting in numerous classification proposals. This condition tends to hinder a practical and, most importantly, insightful and critical appropriation of digital technology among museum professionals. Therefore, a three-dimensional taxonomy is proposed. The first dimension – virtualization – is created based on the reality-virtuality continuum and allows assessing possible variations and compositions of real and virtual objects. The second dimension – observation distance – is focused on the formal design contours of video displays, such as the dimension and positioning related to the user. The third dimension – interactivity – classifies, based on types of interaction, the extent to which users actively modify the content in a mediated environment in real-time. Each interdependent dimension is divided into six discrete classes. This tripartite classification system streamlines the main qualitative characteristics within different digital mediation systems, serving as a theoretical framework for digital integration centered on sustainability, accessibility, inclusivity, critical thinking, and individual subjectivity during the museum experience.
LMFD: lightweight multi-feature descriptors for image stitching

Image stitching is a fundamental pillar of computer vision, and its effectiveness hinges significantly on the quality of the feature descriptors. However, the existing feature descriptors face several challenges, including inadequate robustness to noise or rotational transformations and limited adaptability during hardware deployment. To address these limitations, this paper proposes a set of feature descriptors for image stitching named Lightweight Multi-Feature Descriptors (LMFD). Based on the extensive extraction of gradients, means, and global information surrounding the feature points, feature descriptors are generated through various combinations to enhance the image stitching process. This endows the algorithm with formidable rotational invariance and noise resistance, thereby improving its accuracy and reliability. Furthermore, the feature descriptors take the form of binary matrices consisting of 0s and 1s, not only facilitating more efficient hardware deployment but also enhancing computational efficiency. The utilization of binary matrices significantly reduces the computational complexity of the algorithm while preserving its efficacy. To validate the effectiveness of LMFD, rigorous experimentation was conducted on the Hpatches and 2D-HeLa datasets. The results demonstrate that LMFD outperforms state-of-the-art image matching algorithms in terms of accuracy. This empirical evidence solidifies the superiority of LMFD and substantiates its potential for practical applications in various domains.
Toward Artificial Empathy for Human-Centered Design

Abstract In the early stages of the design process, designers explore opportunities by discovering unmet needs and developing innovative concepts as potential solutions. From a human-centered design perspective, designers must develop empathy with people to truly understand their experiences and needs. However, developing empathy is a complex and subjective process that relies heavily on the designer's empathic capability, and is often subject to the experiences of a small group of people. Therefore, the development of empathic understanding is intuitive, and the discovery of underlying needs can be serendipitous and unrepresentative. This paper aims to provide insights from artificial intelligence research to indicate the future direction of AI-driven human-centered design, considering the essential role of empathy. Specifically, we conduct an interdisciplinary investigation of research areas such as data-driven user research, empathic design, and artificial empathy. Based on this foundation, we discuss the role that artificial empathy can play in human-centered design and propose an artificial empathy framework for human-centered design. Building on the mechanisms behind empathy and insights from empathic design research, the framework aims to break down the rather complex and subjective process of developing empathic understanding into modules and components that can potentially be modeled computationally. Furthermore, we discuss the expected benefits of developing such systems and identify research opportunities to suggest future research efforts.
Protecting Personal Data in Surveillance Society: A Designerly Approach to Privacy in Service Design

Service design emerged in the last decades as the design practice that creates the conditions of interactions in the context of services. Contemporary business models involve interactions between users and the service and between end-users, where personal information is exchanged and processed. Hence, in a world labelable as ‘surveillance society’ or ‘surveillance capitalism’, the privacy and confidentiality of users’ information, image, and identity face serious threats. Building upon the human-centered design paradigm, service design is called to take an ethical stance to design services that do not jeopardize users’ privacy. First, however, service design should build its own comprehensive designerly understanding of privacy, based on input from other disciplines and on research, through design ethnography, to meaningfully tackle privacy challenges. Then, service designers should develop a designerly, project-based approach to protect informational privacy. Through tests and the solution to speculative and thought problems, service designers develop a service design thinking for privacy to design privacy-oriented services. We argue that a culture of designerly understanding of privacy shared across the service design community leads to a better service design.
Educating the Future Digital Maturity Enablers. Learning from the Experience of the DC4DM LLabs

The ever-changing digitally enabled world we live in requires preparing young creatives and entrepreneurs with more suitable abilities and set of skills to face the challenges of ongoing digital transformations. The Erasmus + funded project DC4DM aims to implement a human-centered educational model to train Digital Maturity Enablers. This new type of professionals would be able to drive small and medium organizations towards their digital maturity: owning specific creative digital skills, they would be able to extract value from what the technological landscape offers and respond to the human needs through the principles of ethics and sustainability. This paper aims to describe the DC4DM educational model and reflect on the experience of three testing sessions, called DM Learning Labs, 10-days design-led workshops involving a great diversity of university students, diverse for country of origin and study background, start-ups, and several local mentors and stakeholders to co-design compelling future tech-scenarios. The paper will present the results from a preliminary study which aims to understand how attending a DM Learning Lab has impacted on the participants. Particularly, to understand which Digital Creative Ability had developed thanks to the LLab experience and which learnings were consequently applied in new creative and co-design contexts. The paper tries to highlight how much the DC4DM methodology has the potential to change the way to educate future professionals through cross-disciplinary and collaborative learning opportunities.
Usability Requirements for Visualization Technologies in Industrial Field

Data communication and visualization play a significant role in contemporary society in which the amount of information available is constantly growing. The paper focuses on the importance of communicating and visualizing data clearly and effectively, with a focus on industrial manufacturing contexts. Data visualization is a crucial element in enabling users to understand complex information sets through visual representations that make them easily accessible and interpretable. It is the process by which objects, numbers and concepts are transformed into an easily understood form. Within production processes, where information flows are numerous and complex, dashboards are used for rapid monitoring of metrics and performance indicators. To ensure the effectiveness of a dashboard, it must be adapted to the specific needs of the intended user and be consistent with its original purpose. The paper proposes an iterative process for designing the interaction between the user and the dashboard, based on the analysis of theoretical models related to visual perception, visual communication, and information visualization. From the collection of heuristics in the literature, requirements for evaluating the usability of information visualization systems were identified. The set of heuristics has been and are going to be a valuable resource for defining and implementing evaluation tests useful in designing new dashboard systems that make information easily accessible.
The Expanded Book - Various Perspectives in Augmented Reality

This research aims to understand how Augmented Reality (AR) can enhance and enrich the traditional experience of reading an illustrated children’s book. In this context, a children’s book has been developed that explores the digital component of AR, along with the characteristics of a physical book, resulting in a synergy with the printed narrative. The narrative is presented in two layers, a printed one and a digital one, the latter reveals characters that are not seen in the page layout. Thus, after the interactive book prototype was created, tests were conducted with a sample of the target audience. These tests helped to understand that this combination resulted in a captivating experience for the user who demonstrated a positive reaction to the book and the activity. The project revealed that it can bring the user and reading closer together through the use of technology, which has proven to be a relevant and engaging factor among young audiences. These tests also revealed that participants did not have difficulty handling both dimensions, the book and the mobile device, simultaneously. Similarly, the test also focused on information retention about the narrative to assess the ability of AR to improve the understanding of illustrated story reading.
Using Augmented Reality for Training Computer Workstation Ergonomic Issues: An Exploratory Project

The number of hours dedicated to working at a desk and in front of a computer is gradually taking its toll on millions of people. It has drastically increased the number of health issues associated with this study and work habits. The ATOMIC project, presented in this paper, explores the use of Virtual Reality and Augmented Reality solutions for training and teaching activities in a diverse array of contexts, such as creating awareness of computer workstation inherent ergonomic issues. The prototype presented is closely concerned with training its users to recognize their poor decisions regarding their desk workstation organization and helping to improve their decisions and behaviour in this context. It includes an augmented reality solution projected onto the desktop, capturing information with a standard webcam, a computer vision solution using OpenCV and Python, or a QR code solution through Unity and Vuforia. The prototype was evaluated during a summer school with the participation of 20 higher education students during a 4-stage test protocol (profiling, pre-test, AR experience, post-test) during which data regarding the user experience and interface usability was collected through 3 questionnaires all answered through an app on the users’ smartphone. The results attained were, in general, positive and showed that the users comprehended the purpose of the application and regarded it as useful for further understanding the benefits of small adjustments in their workstation layouts. Although they did not feel immersed in the experience, the users did see using the application as a better method for learning about ergonomic issues as opposed to more traditional pen-and-paper approaches. Overall, the tests conducted provided a detailed insight into the pertinence of the prototype and its context, as well as a list of suggestions for further improvements regarding its UX strengths and weaknesses.
INR-Arch: A Dataflow Architecture and Compiler for Arbitrary-Order Gradient Computations in Implicit Neural Representation Processing

An increasing number of researchers are finding use for nth-order gradient computations for a wide variety of applications, including graphics, meta-learning (MAML), scientific computing, and most recently, implicit neural representations (INRs). Recent work shows that the gradient of an INR can be used to edit the data it represents directly without needing to convert it back to a discrete representation. However, given a function represented as a computation graph, traditional architectures face challenges in efficiently computing its nth-order gradient due to the higher demand for computing power and higher complexity in data movement. This makes it a promising target for FPGA acceleration. In this work, we introduce INR-Arch, a framework that transforms the computation graph of an nth-order gradient into a hardware-optimized dataflow architecture. We address this problem in two phases. First, we design a dataflow architecture that uses FIFO streams and an optimized computation kernel library, ensuring high memory efficiency and parallel computation. Second, we propose a compiler that extracts and optimizes computation graphs, automatically configures hardware parameters such as latency and stream depths to optimize throughput, while ensuring deadlock-free operation, and outputs High-Level Synthesis (HLS) code for FPGA implementation. We utilize INR editing as our benchmark, presenting results that demonstrate 1.8-4.8x and 1.5-3.6x speedup compared to CPU and GPU baselines respectively. Furthermore, we obtain 3.1-8.9x and 1.7-4.3x lower memory usage, and 1.7-11.3x and 5.5-32.8x lower energy-delay product. Our framework will be made open-source and available on GitHub.****https://github.com/sharc-lab/inr-arch
Invited Paper: Instruction Set Extensions for Post-Quantum Cryptography

Quantum computing is one of the latest break-throughs in the field of computer science, having the potential of breaking the underlying assumptions of public-key cryptography. With the National Institute of Standards and Technology (NIST) having announced that lattice-based KYBER as Key Encapsulation Mechanism (KEM) and DILITHIUMAND FALCON as digital signatures are going to be standardized as the first Post-Quantum Cryptography (PQC) schemes, the scientific community needs to investigate how to efficiently implement these new primitives to ensure a smooth transition. We review in this work the state-of-the-art in Instruction Set Extensions (ISEs) for the lattice-based PQC schemes to be standardized. We categorize them into three groups. Firstly, tightly-integrated implementations that aim to be small and only accelerate the core functions, secondly more generic and bigger ISEs that target more lattice operations, and thirdly a special class that focuses on vectorized processing. While we observe promising results in improving on runtime and energy consumption, the memory footprint is often overlooked in the evaluation, even though this is a serious issue in PQC where keys, ciphertexts and signatures tend to be larger. Additionally, we envision that more generic lattice-based ISEs will surface, and that side-channel and fault attacks will become more important.
Memory-aware Scheduling for Complex Wired Networks with Iterative Graph Optimization

Memory-aware network scheduling is becoming increasingly important for deep neural network (DNN) inference on resource-constrained devices. However, due to the complex cell-level and network-level topologies, memory-aware scheduling becomes very challenging. While previous algorithms all suffer from poor scalability, in this paper, we propose an efficient memory-aware scheduling framework based on iterative computation graph optimization. Our framework features an iterative graph fusion algorithm that simplifies the computation graph while preserving the scheduling optimality. We further propose an integer linear programming formulation together with topology-aware variable pruning to schedule the simplified graph efficiently. We evaluate our method against prior-art algorithms on different networks and demonstrate that our method outperforms existing techniques in all the benchmarks, reducing the peak memory footprint by 13.4%, and achieving better scalability for networks with complex network-level topologies.
FIONA: Photonic-Electronic CoSimulation Framework and Transferable Prototyping for Photonic Accelerator

Recent advances in the architecture design for photonic accelerators have demonstrated great promise to accelerate deep neural network (DNN) applications, and also allude to the essential collaboration of the electronic subsystems for efficient logic arithmetic and memory access. However, available tools to design and evaluate photonic accelerators usually neglect the cross-stack effects or low-level details in real-world scenarios, ranging from programming-stack inefficiency to electronic peripheral implementation complexity. This frustrating fact makes it difficult to holistically estimate the performance metrics of a practical photonic-electronic collaborative computing system. In addition, until now, no toolchain can provide programmable, hardware-reconfigurable, and end-to-end rapid verification for photonic accelerators. Here we present FIONA, a Full-stack Infrastructure for Optical Neural Accelerator, which comprises a photonic-electronic co-simulation framework for multilevel design space exploration (DSE), and a transferable hardware prototyping template for physical verification. Specifically, the co-simulation framework consists of a functional simulator at the instruction set architecture (ISA) level to agilely verify the programming software stack and a register-transfer level (RTL) cycle-accurate simulator to precisely profile the overall system. We also demonstrate LightRocket as a case study of the FIONA toolchain to show the full workflow of designing a Turing-complete photonic accelerator system that supports arbitrary DNN workloads and on-chip training. The toolchain is open-sourced and available at https://github.com/hkust-fiona/.
SAGA: Sparsity-Agnostic Graph Convolutional Network Acceleration with Near-Optimal Workload Balance

Graph Convolutional Networks (GCNs) have shown much promise in resolving sophisticated scientific problems with non-Euclidean data, such as traffic prediction, disease classification, and many others. However, the irregular sparsity of real-world graphs remains a major challenge toward efficient GCN acceleration. In this paper, we propose SAGA, a Sparsity-Agnostic Graph Convolutional Accelerator with near-optimal workload balance. Specifically, it consists of two unique features, an NZ-based scheduling, and a novel accelerator architecture. Unlike conventional GCN accelerators with uneven distribution of sparse matrix, the proposed NZ-based scheduling leverages the metadata encoded in the compression format to enable even distribution of sparse matrix at runtime, thus achieving near-optimal workload balancing. In addition, the proposed architecture, including a task scheduler, an accumulation table, and a partial row accumulation unit, can support the proposed NZ-based scheduling without data preprocessing and reformatting with low overheads. We prototyped the proposed design through FPGAs, and our evaluation results show that SAGA achieves up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1.56}\times$</tex> speedup and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{2.05}\times$</tex> energy savings on average as compared to the prior art [1].
Invited Paper: In-Sensor Radio Frequency Computing for Energy-Efficient Intelligent Radar

Radio Frequency Neural Networks (RFNNs) have demonstrated advantages in realizing intelligent applications across various domains. However, as the model size of deep neural networks rapidly increases, implementing large-scale RFNN in practice requires an extensive number of RF interferometers and consumes a substantial amount of energy. To address this challenge, we propose to utilize low-rank decomposition to transform a large-scale RFNN into a compact RFNN while almost preserving its accuracy. Specifically, we develop a Tensor-Train RFNN (TT-RFNN) where each layer comprises a sequence of low-rank third-order tensors, leading to a notable reduction in parameter count, thereby optimizing RF interferometer utilization in comparison to the original large-scale RFNN. Additionally, considering the inherent physical errors when mapping TT-RFNN to RF device parameters in real-world deployment, from a general perspective, we construct the Robust TT-RFNN (RTT-RFNN) by incorporating a robustness solver on TT-RFNN to enhance its robustness. To adapt the RTT-RFNN to varying requirements of reshaping operations, we further provide a reconfigurable reshaping solution employing RF switch matrices. Empirical evaluations conducted on MNIST and CIFAR-10 datasets show the effectiveness of our proposed method.
PP-Transformer: Enable Efficient Deployment of Transformers Through Pattern Pruning

Transformer models have been widely adopted in the field of Natural Language Processing (NLP) and Computer Vision (CV). However, the excellent performance of Transformers comes at the cost of heavy memory footprints and gigantic computing complexity. To deploy Transformers on resource constrained platforms, e.g., FPGA, diverse weight pruning strategies have been proposed. However, pattern pruning, as an alternative pruning method, is not well explored in the context of Transformers. In this paper, we propose PP-Transformer, a framework specifically designed to efficiently deploy Transformer models on FPGA using pattern pruning. At the algorithm level, we leverage pattern pruning, a coarse-grained structured pruning strategy, to reduce parameter storage. Meanwhile, we have developed a dedicated hardware architecture, featuring a custom computing engine tailored to support pattern pruning algorithm. Experimental results demonstrate that our algorithm achieves up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$2.26\times$</tex> reduction in parameter storage with acceptable accuracy degradation. Additionally, our hardware implementation exhibits <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$839.72\times$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$5.72\times$</tex> speedup in comparison to CPU and GPU implementations.