README.md

Der Roboterlehrer: Interpretable and deterministic MQM-inspired translation evaluation

About The Project

We seek to create a chatbot capable of performing multidimensional translation evaluation with feedback without making any LLM API calls. We hope that this approach is more interpretable and deterministic than existing state-of-the-art.

At the moment we only support the German-English language pair.

(back to top)

Getting Started

Installation

1. Clone this repository with git clone https://github.com/aphil311/teach-bs.git.
2. Install the dependencies with pip install -r requirements.txt.
   • You must downgrade to pip < 24.1 with pip install pip=24.0 to install laser_encoders.
   • You can upgrade after installing.

Usage

1. Run the streamlit app with streamlit run app.py.
2. The chatbot will immediately prompt you with a German to English translation.
   • You can switch to English to German on the sidebar.
3. Scores are computed as the raw arithmetic mean and can be found in the 'scores' sidebar tab.

(back to top)

Roadmap

See the open issues for a full list of proposed features (and known issues).

License

Distributed under the MIT License. See LICENSE.txt for more information.

Acknowledgments

I would like to thank Professor Srinivas Bangalore as well as the TRA 301 TAs their for their invaluable guidance, feedback, and support.

(back to top)