Spaces:
Runtime error
Apply for community grant: Academic project (gpu)
🤖ConvRe🤯 is a benchmark proposed in our EMNLP 2023 main conference paper. It aims to evaluate LLMs' ability on understanding converse relations.
Converse relation is defined as the opposite of semantic relation while keeping the surface form of the triple unchanged. For example, the triple (x, has part, y) is interpreted as "x has a part called y" in normal relation, while "y has a part called x" in converse relation.
Our experiments show that current LLMs (including gpt-4) still face challenges in this benchmark. And we provide a demo of Llama-2-chat-7b to let the users play on it.
Hi @Trayvon , we have assigned a gpu to this space. Note that GPU Grants are provided temporarily and might be removed after some time if the usage is very low.
To learn more about GPUs in Spaces, please check out https://huggingface.co/docs/hub/spaces-gpus
thanks