MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility
  Applications
Authors: name: Aleksandr Algazinov, name: Matt Laing, name: Paul Laban

Accessibility remains a critical concern in today's society, as many
technologies are not developed to support the full range of user needs.
Existing multi-agent systems (MAS) often cannot provide comprehensive
assistance for users in need due to the lack of customization stemming from
closed-source designs. Consequently, individuals with disabilities frequently
encounter significant barriers when attempting to interact with digital
environments. We introduce MATE, a multimodal accessibility MAS, which performs
the modality conversions based on the user's needs. The system is useful for
assisting people with disabilities by ensuring that data will be converted to
an understandable format. For instance, if the user cannot see well and
receives an image, the system converts this image to its audio description.
MATE can be applied to a wide range of domains, industries, and areas, such as
healthcare, and can become a useful assistant for various groups of users. The
system supports multiple types of models, ranging from LLM API calling to using
custom machine learning (ML) classifiers. This flexibility ensures that the
system can be adapted to various needs and is compatible with a wide variety of
hardware. Since the system is expected to run locally, it ensures the privacy
and security of sensitive information. In addition, the framework can be
effectively integrated with institutional technologies (e.g., digital
healthcare service) for real-time user assistance. Furthermore, we introduce
ModCon-Task-Identifier, a model that is capable of extracting the precise
modality conversion task from the user input. Numerous experiments show that
ModCon-Task-Identifier consistently outperforms other LLMs and statistical
models on our custom data. Our code and data are publicly available at
https://github.com/AlgazinovAleksandr/Multi-Agent-MATE.