Unbabel

/

TowerInstruct-7B-v0.1

like 57

Hi JaimeLugo,
Could you please send an example of "{chstory}" so we can try to replicate the error in our end?
It is possible that there might be a slight prompt mismatch, i.e., you may not be using the optimal prompt (we have a list of possible prompts for MT in https://huggingface.co/datasets/Unbabel/TowerBlocks-v0.1 (just filter the task for machine_translation and you'll find all types of prompts there --- there are many to choose from). Our evaluation was conducted using the prompt in the model card.

Hi JaimeLugo,

Adding to the previous answer, we noticed generation_config.json for the model had the incorrect eos_token_id (only the tokenizer had the correct one), causing it to not stop the generation. It should have been fixed, you just need to redownload the model.

Look here for more info: https://huggingface.co/Unbabel/TowerInstruct-7B-v0.1/discussions/4.

Hey DuarteMRAlves

I confirm the consistency has improved significantly... but still errors happen. For example, it once gave me this response "20: What? What the actual fuck?!'<|im_end|>" so it didn't translate at all, and I gave 1500 tokens as input...

Here is my prompt and chinfo as a reference:

  conversation = [  {'role': 'user', 'content': f'Translate the input text into Spanish: \n Input Text: {chstory}'}] 
  prompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)

  inputs = tokenizer(prompt, return_tensors="pt").to(model.device) 
  outputs = model.generate(**inputs, use_cache=True, pad_token_id=tokenizer.eos_token_id, max_length=4000  , top_k=25, prompt_lookup_num_tokens=10,  do_sample=True)
  output_text = tokenizer.decode(outputs[0]) 

  "ChInfo": "0 | Chapter 2: Slave Caravan\n1 | Sunny dreamt of a mountain.\n2 | Jagged and lonesome, it dwarfed other peaks of the mountain chain, cutting the night sky with its sharp edges. A radiant moon bathed its slopes in the ghostly, pale light.\n3 | On one of the slopes, the remnants of an old road stubbornly clung to the rocks. Here and there, weathered paved stones could be seen through the snow. To the right side of the road, a sheer cliff face rose as an impregnable wall. To the left, a silent black sea of nothingness indicated an endless fall. Strong winds crashed into the mountain over and over again, screaming in powerless rage.\n4 | Suddenly, the moon fell over the horizon. The sun rose from the west, streaked across the sky and disappeared in the east. Snowflakes jumped from the ground and returned into the embrace of clouds. Sunny realized that he was seeing the flow of time in reverse.\n5 | In an instant, hundreds of years flew by. The snow retreated, baring the old road. Cold shivers ran down Sunny's back as he noticed human bones littering the ground. A moment later, the bones were gone, and in their place, a slave caravan appeared, moving backwards down the mountain in the clamor of chains.\n6 | Time slowed, stopped, and then resumed its usual pace.\n7 | [Aspirant! Welcome to the Nightmare Spell. Prepare for your First Trial…]\n8 | What… what the hell is this?'\n9 | Step. Step. Another step.\n10 | A dull ache was radiating through Sunny's bleeding feet as he was shivering from cold. His threadbare tunic was nearly useless against the biting wind. His wrists were the main source of agony: badly hurt by the iron shackles, they sent a sharp pang of pain every time the freezing metal touched his broken skin.\n11 | What kind of a situation is this?!'\n12 | Sunny looked up and down, noticing a long chain winding up the road, with dozens and dozens of hollow-eyed people — slaves just like him — shackled to it at small intervals. Ahead of him, a man with broad shoulders and a bloodied back was walking with a measured gait. Behind him, a shifty-looking guy with quick, desperate eyes was quietly cursing under his breath in a language that Sunny did not know, but somehow still understood. From time to time, armed horsemen in ancient-style armor would pass by, giving the slaves menacing looks.\n13 | However you judged it, things were really bad.\n14 | Sunny was more bewildered than panicked. True, these circumstances were not like what the First Nightmares were supposed to be. Usually, freshly chosen aspirants would find themselves in a scenario that presented them with a fair amount of agency: they would become members of privileged or warrior casts, with plenty of access to necessary weapons to at least try to tackle any conflict.\n15 | Starting out as a powerless slave, shackled and already half-dead, was as far from being ideal as one could imagine.\n16 | However, the Spell was as much about challenge as it was about balance. As the old policeman said, it created trials, not executions. So Sunny was pretty sure that, to counter this abysmal start, it would reward him with something good. A powerful Aspect, at least.\n17 | Let's see… how do I do this?'"

Hi Jaime,
Thanks for raising this issue! We also noted that performance on doc-level translation (i.e. with multiple paragraph or new-line separators) seems to be inconsistent and results vary depending on prompts (some prompt engineering is helpful here). TowerInstruct was really only trained to deal with paragraph-level MT.

We are currently working on a new version of TowerInstruct that will be able to deal better with such documents. Stay tuned!