Spaces:
Running
on
Zero
Suggestion: Consider testing if a repetition_penalty value could help reduce repetitions without degrading answer quality
Heya,
I think it's worth checking if the inference might benefit from a Repetition Penalty parameter (like in the code example below the text here: https://huggingface.co/docs/transformers/v4.38.2/en/internal/generation_utils#transformers.RepetitionPenaltyLogitsProcessor )
i love that about open source. you know this model came from the community , so it really means all the more to invite community efforts and lead the way by example , i really appreciate it , and i'll also do the same so let's connect :-)
I have a 24 GB vram GPU, some offloading is happening and it's a bit slow, so far tried repetition_penalty with values of 1.1, 1.2, 1.3, 1.4 which did not seem to improve much but 1.5 already seems significantly better than before
With repetition_penalty=1.5 it generated:
I'm sorry to hear that you are too tired. It is important to take time to rest and recover from a long day. Here are some things you can do to help yourself: 1. Take a deep breath and relax. A deep breath can help you to clear your mind and recharge your energy. 2. Take a nap. A nap can help you to recharge your energy and get some rest. 3. Eat a nutritious meal. A nutritious meal can help you to feel full and refreshed. 4. Get some sleep. A good night's sleep can help you to recover from a long day. 5. Take a break. A break can help you to relax and recharge your energy. 6. Take a shower. A shower can help you to clear your mind and recharge your energy. 7. Take a bath. A bath can help you to relax and recharge your energy. 8. Take a hot shower. A hot shower can help you to clear your mind and recharge your energy. 9. Take a nap. A nap can help you to recharge your energy and get some rest. 10. Take a break. A break can help you to relax
So I jumped to repetition_penalty=2.0 which I'm not sure if it's better or not:
I'm sorry to hear that you are too tired. It is important to take time to rest and recover from a long day. Here are some things you can do to help yourself: 1. Take a deep breath and relax. A deep breath can help you to clear your mind and recharge your energy. 2. Eat a healthy diet. A healthy diet is important for overall health and well-being. Try to eat foods that are rich in vitamins, minerals, and fiber. 3. Get plenty of sleep. Sleep is essential for overall health and well-being. Try to get at least 8 hours of sleep each night. 4. Get enough exercise. Exercise can help you to stay fit and healthy. Try to do a few light workouts or aerobic exercises to help you stay active. 5. Get enough rest. Resting is important for overall health and well-being. Try to get at least 8 hours of rest each night. 6. Get enough sleep. Sleep is essential for overall health and well-being. Try to get at least 8 hours of sleep each night.
I think 1.8 is a good value for this model.
Settings: max_new_tokens = 250, repetition_penalty = 1.8, rtl = False
Create streamer
I'm too tired, what should I do?
Create thread
I'm sorry to hear that you are too tired. It is important to take time to rest and recover from a long day. Here are some things you can do to help yourself: 1. Take a deep breath and relax. A deep breath can help you to clear your mind and recharge your energy. 2. Eat a healthy diet. A healthy diet is important for overall health and well-being. Try to eat foods that are rich in vitamins, minerals, and fiber. 3. Get plenty of sleep. Sleep is essential for overall health and well-being. Try to get at least 8 hours of sleep each night. 4. Get enough exercise. Exercise can help you to stay fit and healthy. Try to do a few vigorous workouts every day. 5. Take a break. If you are feeling tired, try to take a break from your work or school. This can help you to relax and recharge. 6. Take a shower. A shower can help you to clear your mind and recharge your energy. Try to take a shower before bed. 7. Take a nap. A nap can
The script with which I made these tests is available as a gist