Repository: instructlab/instructlab

Issue #707: Able to break chat when inquiring about special characters
Created: 2024-03-21T14:27:43Z
I am making an assumption in the title of this issue that it is caused by special characters. I was using lab chat to see about its ability to make kaomoji, which often involve special characters. After some successful kaomoji output, I got a strange response.
>>> show me a kaomoji that expresses sleepiness                                                                                                                                                                                                [S][default]
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ To express sleepiness, you can use the following                                                                                                                                                                                                        │
│ kaomoji:<dummy00001>































<|p │
│ ad|>













 is ˡ(ˡ‿(ˡ) This one features a yawn sleepy person, often used to express feeling sleepy or drowsy, which is a term used to describe someone who  │
│ is feeling drowsy, or sleepy, often used to express feeling drowsy or sleepy.                                                                                                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 20.622 seconds ─╯

Intrigued, I inquired about 
 and <dummy00001>:
>>> what is "
"?                                                                                                                                                                                                                         [S][default]
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ The term "drowsy" or "sleepy" is not directly related to the concept of "drowsy" or "sleepy". Instead, they describe someone who is feeling drowsy (drowsy is short for "droolsy", which means "drowsy" or "droolsy", which means "drowsy" or           │
│ "droolsy", which means "drowsy" or "droolsy") or sleepy (sleepy is short for "sleepy", which means "sleepy" or "sleepy", which means "sleepy" or "sleepy"), which means they are feeling drowsy (drowsy is short for "droolsy", which means "drowsy" or │
│ "droolsy", which means "drowsy" or "droolsy") or sleepy (sleepy is short for "sleepy", which means "sleepy" or "sleepy", which means "sleepy" or "sleepy"). So, "drowsy" or "sleepy" is not directly related to the concept of "drowsy" or "sleepy",    │
│ but rather, they describe someone who is feeling drowsy (drowsy is short for "droolsy", which means "drowsy" or "droolsy", which means "drowsy" or "droolsy") or sleepy (sleepy is short for "sleepy", which means "sleepy" or "sleepy", which means    │
│ "sleepy" or "sleepy"). Instead, they describe someone who is feeling drowsy (drowsy is short for "droolsy", which means "drowsy" or "droolsy", which means "drowsy" or "droolsy") or sleepy (sleepy is short for "sleepy", which means "sleepy" or      │
│ "sleepy", which means "sleepy" or "sleepy"). So, "drowsy" or "sleepy" is not directly related to the concept of "drowsy" or "sleepy", but rather, they describe someone who                                                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 72.005 seconds ─╯
>>> what is <dummy00001>                                                                                                                                                                                                                       [S][default]
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ The term "╬╬" (U+G) is used to represent the concept of "╬╬", which is short for "U+G", which means "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not directly related to the concept of "U+G", but rather, they describe someone who is      │
│ feeling "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not directly related to the concept of "U+G", but rather, they describe someone who is feeling "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not directly related to the      │
│ concept of "U+G", but rather, they describe someone who is feeling "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not directly related to the concept of "U+G", but rather, they describe someone who is feeling "U+G" or "U+G", which means   │
│ "U+G" or "U+G". So, "╬╬" is not directly related to the concept of "U+G", but rather, they describe someone who is feeling "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not directly related to the concept of "U+G", but rather, they       │
│ describe someone who is feeling "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not directly related to the concept of "U+G", but rather, they describe someone who is feeling "U+G                                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 72.232 seconds ─╯
>>> what does "<dummy00001>" mean when combined with "
"?                                                                                                                                                                                [S][default]
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ When the term "╬╬" (U+G) is combined with the concept of "╬╬", which is short for "U+G", which means "U+G" or "U+G", which means "U+G" or "U+G", it often describes someone who is feeling "U+G" or "U+G", which means "U+G" or "U+G". So, "╬╬" is not  │
│ directly related to the concept of "╬╬", but rather, they describe someone who                                                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 21.904 seconds ─╯
>>> ok thanks                                                                                                                                                                                                                                  [S][default]
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ No problem! I'm here to help! If you have any more questions or need further clarification, just let me know!                                                                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 4.620 seconds ─╯

This was a bit weird, but did at least behave normally after spitting out junk fora bit. Full output of that session is in attached file droolsy.txt.
After a while, I started a new session to look at it more. This time, when I asked about 
, it kept going until I cut it off at 400s. After that point, the chat seems to be broken, and no longer parsed queries correctly. See attached file manner_cut-at-400s.txt for this instance. Unfortunately, the beginning of this output was cut off by the 400s of rambling, but I had asked for a few different kaomoji and it gave them to me normally.
After this, I quit the lab chat session and started a new one. The new session was back to normal behavior.

Duplicate issue #996: Adding rot13 skill generates nonsense responses
Created: 2024-04-25T13:07:49Z
I developed this yaml file originally for a PR:
task_description: |
  The rot13 task enables converting from rot13 to plaintext.
seed_examples:
  - question: Decode \"uryyb\" from rot13.
    answer: |
      hello
  - question: Decode \"guvf vf n grfg\" from rot13.
    answer: |
      this is a test
  - question: Decode \"Erq Ung vf n grpuabybtl pbzcnal\" from rot13.
    answer: |
      Red Hat is a technology company
  - question: Decode \"Lrfgreqnl V jrag gb gur fgber\" from rot13.
    answer: |
      Yesterday I went to the store
  - question: Decode \"Gur dhvpx oebja sbk whzcf bire gur ynml qbt.\" from rot13.
    answer: |
       The quick brown fox jumps over the lazy dog.

When I trained the model and tested this skill in InstructLab, it was unable to successfully decode rot13.

Closing comment (2024-07-25T20:39:36Z): @jbaublitz could this be related to #707?