# Detailed Image Captioning and Hashtag Generation
## 4 Experiments and Results
### Browser Extension Implementation  
As an application of the CapFlow model, we developed a Chrome browser extension integrating its image captioning and hashtag generation capabilities directly into the user's web browsing experience. It utilizes HTML, CSS, and JavaScript for the frontend, while the backend API for the captioning model is implemented using Python with Flask.  
To address the limitations of Chrome's default extension popup, which closes when the user interacts outside it, we developed a custom popup solution. Upon clicking the extension button, our custom interface appears and can be moved freely on the screen. This approach enhances usability by allowing users to simultaneously interact with both the webpage and the extension.  
The extension's main functionality is triggered when the user clicks a \"Start\" button in the popup. This action applies a semi-transparent overlay to dim the entire webpage, signaling to the user that the extension is active and waiting for input. Users can then select any image on the webpage for processing.  
Upon image selection, the chosen image is visually highlighted against the dimmed background. The extension then sends the image data to the backend API, which hosts our image captioning and hashtag generation models. These models process the image and return a detailed caption along with relevant hashtags. The results are then displayed within the extension's popup interface. If the user does not select an image after activating the extension, an error message is shown, and the process terminates, returning the webpage to its normal state. Users can restart the process at any time by clicking a \"Start Again\" button, which resets the interface and allows for a new image selection.  
The frontend of the extension uses vanilla JavaScript for DOM manipulation and event handling, with CSS for styling the popup and creating visual effects like the page dimming. To capture the selected image, we employed Chrome's content scripts feature, which allows us to inject JavaScript into the webpage to handle image selection and communicate with our custom popup.  
The communication between different parts of the extension (popup, background scripts, and content scripts) utilizes Chrome's messaging APIs, such as chrome.runtime.sendMessage() and chrome.tabs.sendMessage(). For interactions with the backend server, we use asynchronous HTTP requests, allowing data transfer without reloading the extension popup or the webpage. The backend API, built with Python and Flask, is containerized using Docker to ensure consistency across different environments and simplify deployment. We use a docker-compose.yml file to define and manage the multi-container Docker application, coordinating the services required for the extension's operation.  
This implementation separates the user interface concerns from the complex processing of image captioning and hashtag generation. It allows for independent updates and maintenance of different extension components, contributing to the overall modularity and scalability of the system.  
A video demonstration of our browser extension, CapFlow, is available on YouTube at [[https://youtu.be/9](https://youtu.be/9) vuyXFFfgok]([https://youtu.be/9](https://youtu.be/9) vuyXFFfgok) accessed on 27 November 2024. Screenshots from the video, which showcase the extension's workflow, can be seen in Figure 5.