Reviewerly/exHarmony huggingface dataset:
End-to-end XY Separation for Single Image Blind Deblurring

Single image blind deblurring, only exploiting a blurry observation to reconstruct the sharp image, is a popular yet challenging low-level vision task. Current state-of-the-art deblurring networks mainly follow the coarse-to-fine strategy for architecture design and utilize U-net or its variant, XYDeblur, as the basic units. However, the one-encoder-one-decoder and the recently proposed one-encoder-two-decoder structures of basic units both fail to comprehensively take advantage of the directional separability of 2D deblurring, which increases the learning content of networks, thus leading to performance degradation. To thoroughly decouple the deblurring into two spatially orthogonal parts, we propose a novel substitution for U-net and its variant, called XYU-net. Specifically, it consists of two structurally identical U-nets, named XU-net and YU-net. They share orthogonal parameters by rotating kernels and focus on restoring a 2D blurry image in two spatially orthogonal directions respectively, which not only brings efficiency enhancement but also maintains parameter number. To further reduce the graphics memory demand of XYU-net, we transfer some non-linear transform modules (NLTM) from the outside of the network to its inside and propose the modified version, called MXYU-net. Experimental results on three large blurry image datasets demonstrate the efficiency of XYU-net and MXYU-net compared with U-net and XYDeblur, both as standalone models and as basic units of advanced U-net-based deblurring networks.
Practical Edge Detection via Robust Collaborative Learning

Edge detection, as a core component in a wide range of vision-oriented tasks, is to identify object boundaries and prominent edges in natural images. An edge detector is desired to be both efficient and accurate for practical use. To achieve the goal, two key issues should be concerned: 1) How to liberate deep edge models from inefficient pre-trained backbones that are leveraged by most existing deep learning methods, for saving the computational cost and cutting the model size; and 2) How to mitigate the negative influence from noisy or even wrong labels in training data, which widely exist in edge detection due to the subjectivity and ambiguity of annotators, for the robustness and accuracy. In this paper, we attempt to simultaneously address the above problems via developing a collaborative learning based model, termed PEdger. The principle behind our PEdger is that, the information learned from different training moments and heterogeneous (recurrent and non recurrent in this work) architectures, can be assembled to explore robust knowledge against noisy annotations, even without the help of pre-training on extra data. Extensive ablation studies together with quantitative and qualitative experimental comparisons on the BSDS500 and NYUD datasets are conducted to verify the effectiveness of our design, and demonstrate its superiority over other competitors in terms of accuracy, speed, and model size.
Learning Implicit Entity-object Relations by Bidirectional Generative Alignment for Multimodal NER

The challenge posed by multimodal named entity recognition (MNER) is mainly two-fold: (1) bridging the semantic gap between text and image and (2) matching the entity with its associated object in image. Existing methods fail to capture the implicit entity-object relations, due to the lack of corresponding annotation. In this paper, we propose a bidirectional generative alignment method named BGA-MNER to tackle these issues. Our BGA-MNER consists of image2text and text2image generation with respect to entity-salient content in two modalities. It jointly optimizes the bidirectional reconstruction objectives, leading to aligning the implicit entity-object relations under such direct and powerful constraints. Furthermore, image-text pairs usually contain unmatched components which are noisy for generation. A stage-refined context sampler is proposed to extract the matched cross-modal content for generation. Extensive experiments on two benchmarks demonstrate that our method achieves state-of-the-art performance without image input during inference.
Seeing in Flowing: Adapting CLIP for Action Recognition with Motion Prompts Learning

The Contrastive Language-Image Pre-training (CLIP) has recently shown remarkable generalization on "zero-shot" training and has applied to many downstream tasks. We explore the adaptation of CLIP to achieve a more efficient and generalized action recognition method. We propose that the key lies in explicitly modeling the motion cues flowing in video frames. To that end, we design a two-stream motion modeling block to capture motion and spatial information at the same time. And then, the obtained motion cues are utilized to drive a dynamic prompts learner to generate motion-aware prompts, which contain much semantic information concerning human actions. In addition, we propose a multimodal communication block to achieve a collaborative learning and further improve the performance. We conduct extensive experiments on HMDB-51, UCF-101, and Kinetics-400 datasets. Our method outperforms most existing state-of-the-art methods by a significant margin on "few-shot" and "zero-shot" training. We also achieve competitive performance on "closed-set" training with extremely few trainable parameters and additional computational costs.
Style Transfer Meets Super-Resolution: Advancing Unpaired Infrared-to-Visible Image Translation with Detail Enhancement

The problem of unpaired infrared-to-visible image translation has gained significant attention due to its ability to generate visible images with color information from low-detail grayscale infrared inputs. However, current methodologies often depend on conventional style transfer techniques, which constrain the spatial resolution of the visible output to be equivalent to that of the input infrared image. The fixed generation pattern results in blurry generated results when translating low-resolution infrared inputs, and utilizing high-resolution infrared inputs as a solution necessitates greater computational resources. This spurs us to investigate the challenging unpaired image translation from low-resolution infrared inputs to high-resolution visible outputs, with the ultimate goal of enhancing image details while reducing computational costs. Therefore, we propose a unified framework that integrates the super-resolution process into our unpaired infrared-to-visible image transfer, yielding realistic and high-resolution results. Specifically, we propose the Detail Consistency Loss to establish a connection between the two aforementioned modules, thereby enhancing the quality of visual detail in style transfer results through the super-resolution module. Furthermore, our Texture Perceptual Loss is designed to ensure that the generator generates high-quality visual details accurately and reliably. Experimental results indicate that our method outperforms other comparative approaches when utilizing low-resolution infrared inputs. Remarkably, our approach even surpasses techniques that use high-resolution infrared inputs to generate visible images. Last but equally important, we propose a new and challenging dataset, dubbed as InfraredCity-HD, which comprises 512X512 resolution images, to advance research on high-resolution infrared-related fields.
MADiMa '23: 8th International Workshop on Multimedia Assisted Dietary Management

This abstract provides a summary and overview of the 8th International Workshop on Multimedia Assisted Dietary Management.
Panel: Multimodal Large Foundation Models

The surprisingly fluent predictive performance of LLM (Large Language Models) as well as the high-quality photo-realistic rendering of Diffusion Models has heralded a new beginning in the area of Generative AI. Such kinds of deep learning based models with billions of parameters and pre-trained on massive-scale data-sets are also called Large Foundation Models (LFM). These models not only have caught the public imagination but also have led to an unprecedented surge in interest towards the applications of these models. Instead of the previous approach of developing AI models for specific tasks, more and more researchers are developing large task-agnostic models pre-trained on massive data, which can then be adapted to a variety of downstream tasks via fine-tuning, fewshot learning, or zero-shot learning. Some examples are ChatGPT, LLaMA, GPT-4, Flamingo, MidJourney, Stable-Diffusion and DALLE. Some of them can handle text (e.g., ChatGPT, LLaMA) while some others (e.g., GPT-4 and Flamingo) can utilize multimodal data and can hence be considered Multimodal Large Foundation Models (MLFM).
Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding

The surge in video and social media content underscores the need for a deeper understanding of multimedia data. Most of the existing mature video understanding techniques perform well with short formats and content that requires only shallow understanding, but do not perform well with long format videos that require deep understanding and reasoning. Deep Video Understanding (DVU) Challenge aims to push the boundaries of multimodal extraction, fusion, and analytics to address the problem of holistically analyzing long videos and extract useful knowledge to solve different types of queries. This paper introduces a query-aware method for long video localization and relation discrimination, leveraging an image-language pretrained model. This model adeptly selects frames pertinent to queries, obviating the need for a complete movie-level knowledge graph. Our approach achieved first and fourth positions for two groups of movie-level queries. Sufficient experiments and final rankings demonstrate its effectiveness and robustness.
Hierarchical Reasoning Network with Contrastive Learning for Few-Shot Human-Object Interaction Recognition

Few-shot learning (FSL) for human-object interaction aims at classifying samples of new unseen HOI classes with only a few labeled samples available. Although progress has been made in few-shot human-object interaction, most of the existing methods encounter two issues in handling fine-grained interactions: the inability to capture more subtle interactive clues and the inadequacy in learning from data with low inter-class variance. To tackle the first issue, we propose a hierarchical reasoning network to integrate multi-level interactive clues (from coarse to fine-grained) for strengthening HOI representations. The hierarchical relation module mainly captures and aggregates more discriminative relation information among human parts at multiple levels (including the human instance, action region, and body part levels) and objects via a unified graph and exploits a language-guided attentive fusion way to highlight informative features of each interaction level. To address the second issue, we introduce a contrastive learning mechanism to alleviate the inter-class variance. Compared with the previous ProtoNet-based methods, our model generates more discriminative representations for low inter-class variance data, since it makes full use of potential contrastive pairs in each training episode. Extensive experimental results on two standard benchmarks demonstrate that the proposed model performs favorably against state-of-the-art FS-HOI methods.
Deep Video Understanding with Video-Language Model

Pre-trained video-language models (VLMs) have shown superior performance in high-level video understanding tasks, analyzing multi-modal information, aligning with Deep Video Understanding Challenge (DVUC) requirements.In this paper, we explore pre-trained VLMs' potential in multimodal question answering for long-form videos. We propose a solution called Dual Branches Video Modeling (DBVM), which combines knowledge graph (KG) and VLMs, leveraging their strengths and addressing shortcomings.The KG branch recognizes and localizes entities, fuses multimodal features at different levels, and constructs KGs with entities as nodes and relationships as edges.The VLM branch applies a selection strategy to adapt input movies into acceptable length and a cross-matching strategy to post-process results providing accurate scene descriptions.Experiments conducted on the DVUC dataset validate the effectiveness of our DBVM.
Dual Illumination Image Enhancement Using Automated MSRCR and Illumination Estimation

Images taken in low-light environments like nighttime or cloudy weather are not conducive to monitoring and fail to fulfill the requirements of applications such as image processing and computer vision. This paper focuses on improving a dual illumination image enhancement algorithm to achieve better image quality and reduce the processing time of producing an enhanced image. The underexposed correction image was produced using Automated Multiscale Retinex Color Restoration while the overexposed correction image was produced using illumination estimation, the corrected images were combined to generate a better-quality image. The system was assessed using the metrics: BRISQUE, NIQE, MDM, and CEIQ, and achieved a BRISQUE value of 18.628, NIQE value of 1.744, MDM value of 0.968, and CEIQ value of 3.615.
COMPUTATIONAL MODELLING OF THE ACTIVATION PARAMETERS OF DIELS ALDER REACTION OF QUINOFLAVONOID AND 2, 3 DIMETHYLBUTA-1, 3-DIENE

Kinetics and thermodynamics of Diels alder reaction of quinoflavoniod and 2, 3 dimethylbuta-1,3-diene was studied using ab-initio method with Hartree Fock at 6-311G* basis set in spartan, Conformational search was carried out on the molecules to obtain the structures that has the lowest energy, an indication of the stability of the structure. Reaction path calculation was done on the most stable structure using C26 – C15 and C27 – C14 as reaction coordinate in 20 iterations, this was done to ascertain the reaction mechanism. Geometric calculation was carried out on the structures of the reactant, transition state and products to obtain the bond length, bond angle, dihedral angle and atomic charge distributions. Kinetics and thermodynamic parameters was calculated at 3480C. The reaction proceeded through a concerted asynchronous six-membered cyclic transition state. True saddle point was characterized and verified by the presence of only one imaginary vibrational frequency which smoothly connect the reactant, the transition state and the product. Negative values obtained for ?H* (-66.65kJ/mol shows that the reaction is exothermic and according to literature this may be as a result of conversion of two of the ?-bonds to two new ?-bonds). Values obtained for energy of formation (-1329.974 hatrees, -1329.911 hatrees, and 1330.023 hatrees for GS, TS and PRD respectively) according to Hammond’s postulate, confirms the exothermicity of the reaction. ?Greaction (-60.16kJ/mol, shows that the reaction is favourable, Negative ?S* (-11.29J/mol.K) suggests the spontaneity of the reaction and Ea (-63.76kJ/mol) suggests that the reaction involves transfer of electrons. It was also observed that rate constants decrease with increase in temperature, although very uncommon but suggested a complex reaction mechanism.
Path Planning for Unified Scheduling of Multi-Robot Based on BSO Algorithm

The technology for path planning of independent mobile robots is mature, but multi-robot path planning for unified scheduling and allocation is much more complex than single-robot path planning. This requires consideration of collision problems between robots, general optimal path problems, etc. This paper proposes the use of the BSO algorithm for unified scheduling and allocation of multiple robots to improve the efficiency of task execution. The BSO algorithm is a new type of intelligent optimization algorithm that uses clustering ideas to search for local optimal solutions and obtains global optimal solutions by comparing local optimal solutions. It also uses mutation ideas to increase the diversity of the algorithm and avoid becoming trapped in local optimal solutions. Using the GA/SA algorithm and the proposed BSO algorithm for computer simulation comparison, we obtained the optimal path planning for the three robots under unified scheduling. The total distance of the optimal path obtained by the BSO algorithm was 27.36% and 25.31% shorter than those of the GA and SA algorithms, respectively. To further test the performance of the BSO algorithm, we conducted additional experiments on the unified scheduling of multiple robots. The experimental results show that the proposed BSO algorithm can significantly improve the efficiency. The multi-robot under unified scheduling performs point-to-point path planning without collisions, and they can traverse all task target points in the shortest path without repetition. This algorithm is suitable for multi-robot tasks in large-scale environments.
Introducing VR personas: an immersive and easy-to-use tool for understanding users

Abstract Personas have been established as an indispensable tool in software and product development. They continuously accompany the development process and seek to build empathy and establish an emotional relationship between developers and users. However, this does not always succeed, with the reasons for failure often lying in the personas themselves. If they do not have a sufficient level of detail or do not reflect everyday people, they lose credibility and therefore their purpose of representing the user. Poor communication of personas is another reason they are quickly forgotten. In this paper, we present a new approach to experiencing personas beyond traditional means such as posters or slides. With the help of virtual reality, we create immersive, three-dimensional personas that can be visited in their own living room. The basis of the implementation is a comprehensive dataset, containing aggregated data from over 8000 detailed face-to-face interviews. We base the layout of the apartment, the furniture, and the characters themselves on the archetypal characteristics of their corresponding user group. In the future, we plan to validate whether this approach can be successful in creating a deeper and more sustainable connection between personas and developers and designers.
APPLICATION UTILISATION OF API AND B-SHARE APPLICATION FOR PROVIDING BLOOD AVAILABILITY INFORMATION

lack of information about the availability of blood to be a problem that is not uncommon in the daily life, there are many medical cases that require blood transfusion procedure. Practically, the hospital will entrust the patient’s family to find blood bags with the required blood type. Families with little knowledge about where to find blood will encounter confusing situations. Whereas they have to find it fast. if a blood transfusion is done too late, then even death may occur, because of this, it is expected that the existence of a public service that can provide information quickly and accurately blood that can be utilized by the community. In the development of the technology of internet this day it could be used to increase the public service and fulfill the needed of information, especially for the government who hasn’t provided the information about blood availability location based on geolocation that should be easily accessed by all people. Because information about the availability of blood is very important information to be known to the community as of the Information needed so we would develop the geographical information system to search the location of PMI and hospitals having the blood supply. The chosen method for this program is Requirement Analysis, Conceptual Modeling, Implementation, and testing. This method is the one of development method of software engineering so that produced an application that could solve the problem by providing a software that integrates blood stock information across PMI and hospitals branches in all over Indonesia. This information system integrates blood stock information across PMI branchesandhospitals all over Indonesia, providing access to real time blood stock updates to people, providing information about PMI branchesandhospitals locations across Indonesia in the shape of a map complete with the contact number
Impacts of Information technology and Risk Management on Cybersecurity Governance: Empirical Study on Malaysian Financial Institutions

"Cybersecurity threats have successfully targeted financial institutions worldwide due to the increased connectivity of seamless and borderless financial services. Considering that criminals have employed more sophisticated methods to exploit the financial industry, financial institutions must adopt an integrated framework to counter the attacks and protect financial infrastructure from exploitation. The study examined cybersecurity governance by extending the Integrated System Theory (IST), including information technology (IT) governance and risk management (RM) governance. Questionnaires were distributed to Malaysian financial institutions through corporate social media platforms, email, and Google Form in November 2021. The questionnaire used a five-point Likert scale and comprised Section A which focused on the respondents’ demographic profile, while Section B emphasised the research construction. A total of 128 respondents participated within four weeks. Data obtained from the questionnaire was analysed for descriptive statistics, correlation, and regression analyses. Resultantly, IT governance and RM governance significantly and positively impacted cybersecurity governance. The study provides better insights to the practitioners, academicians, or researchers to identify which factors to be considered and emphasise before developing an integrated cybersecurity governance framework."
Penerapan Algoritma K-Nearest Neighbor (KNN) Untuk Mengklasifikan Jenis Penerimaan Bantuan

One of the villages in North Lampung which has many socio-economic problems is Madukoro. Some of the problems that often occur in This village includes high poverty rates, low levels of education and low levels of public health. The K-Nearst Neighbor (KNN) algorithm method was chosen by the author because it can be used as a solution to determine the classification of aid recipients. The K-Nearst Neighbor (KNN) algorithm will determine beneficiaries based on work level, age and income. Calculation results using Microsoft Excel show that there are 110 PKH assistance classes, 57 elderly assistance classes, 9 BLT classes with a total data of 176 beneficiary data. And from the results of the calculation of government assistance in the village of North Lampung Madukoro using rapid miner, it is known that the PKH assistance class is 110 people with an accuracy rate of 89.57%, the elderly assistance class is 58 people with an accuracy rate of 87.50% and then the BLT class is 9 people. with an accuracy rate of 100.00% with a total data of 176 beneficiary data.
Self-care of informal male workers during the COVID-19 pandemic in the light of Orem’s theory

ABSTRACT Objective: To understand the self-care requirements of informal male workers in carrying out their activities during the COVID-19 pandemic, in the light of Orem’s theory. Method: Qualitative approach, with an exploratory descriptive nature. Data collection was conducted through a closed questionnaire, with questions that refer to the self-care requirements and semi-structured interviews, both conducted in a virtual environment. Ten men working as delivery workers and app drivers participated. Data analysis was performed using thematic content analysis. Results: The diagnosis of the self-care requirements performed by each of the interviewees was presented. Final considerations: The work performed by the participants influences the self-care requirements of the population studied, and it is possible to infer that as specializations of these requirements occur, they are no longer performed or are practiced superficially.