Changed device_map to force GPU, see #6, https://github.com/tloen/alpaca-lora/issues/21

example-datasets/example-data-limericks.txt

@@ -0,0 +1,478 @@
+There once was a cat with a hat
+Who liked to chase mice and get fat
+But one day he found
+That his hat was unsound
+And ended up looking quite flat
+
+
+There once was a man from Peru
+Whose shoes were a bright shade of blue
+He walked down the street
+With a confident beat
+And everyone said "Who knew?"
+
+
+There once was a girl with a kite
+She flew it with all of her might
+But the wind was too strong
+And before very long
+The kite was nowhere in sight
+
+
+There once was a chef named Pierre
+Whose souffles were the talk of the year
+But one day he slipped
+And the souffle was flipped
+And it landed right in his ear
+
+
+There once was a boy with a ball
+Who played in the park with his doll
+But the ball rolled away
+And he didn't know what to say
+So he went home feeling quite small
+
+
+There once was a bird in a tree
+Who sang a sweet melody
+But then came a storm
+And the bird lost its form
+And its tune was no longer free
+
+
+There once was a man with a beard
+That he thought was quite weird
+But then he realized
+That it was quite prized
+And his confidence was then steered
+  
+  
+There once was a woman named Sue
+Who liked to wear nothing but blue
+But one day she found
+That her clothes were unsound
+And she had to start anew
+
+
+There once was a dog with a bone
+Who wouldn't share it with anyone
+But then came a friend
+And he learned to bend
+And they both had fun in the sun
+
+
+There once was a teacher named Lee
+Who loved to teach history
+But then came a pandemic
+And everything seemed manic
+And Lee had to learn how to teach virtually
+
+
+There once was a man from Brazil
+Whose dance moves were quite the thrill
+He danced every day
+In his own special way
+And everyone watched with goodwill
+
+
+There once was a snail on a leaf
+Who thought life was ever so brief
+But then it started to rain
+And it felt no more pain
+And slid down the leaf like a thief
+
+
+There once was a girl with a pen
+Who wrote poems again and again
+But one day she got stuck
+And her words turned to muck
+And she had to start over, amen
+
+
+There once was a man from the moon
+Who dreamed of coming to Earth soon
+But when he arrived
+He was quite deprived
+And went back to the moon in a swoon
+
+
+There once was a chef named Sue
+Whose food was always on cue
+But one day she tried
+A new recipe guide
+And her customers said "Boo hoo"
+
+
+There once was a boy with a kite
+That flew so high, it was out of sight
+But then the string broke
+And the kite became a joke
+And the boy had to say goodnight
+
+
+There once was a girl with a hat
+That she wore everywhere, even at bat
+But one day it flew off
+And she let out a cough
+And went home feeling quite flat
+
+
+There once was a bird with a beak
+That couldn't find anything to eat
+But then came a worm
+And the bird had a firm
+And satisfying meal, oh so sweet
+
+
+There once was a man with a car
+That he drove like a superstar
+But then came a crash
+And his car was just ash
+And he had to find a new bar
+
+
+There once was a woman named Rose
+Whose garden was the envy of those
+But then came a storm
+And the garden was forlorn
+And Rose had to start over with hose
+
+
+There once was a boy with a dream
+To travel to space, or so it would seem
+He worked hard every day
+And soon found his way
+And lived his life in zero-gravity regime
+
+
+There once was a cat named Jack
+Who loved to sleep on a sack
+But one day he fell
+And it hurt like hell
+And he woke up with a crack
+
+
+There once was a man from the west
+Whose hat was his prize possession, the best
+But then came the wind
+And his hat flew, it was thinned
+And he went on a quest to find it, obsessed
+
+
+There once was a woman named Sue
+Who painted the town with a bright hue
+But one day she ran out
+And she began to pout
+And went to the store to buy some new
+
+
+There once was a boy with a toy
+That he played with every day, oh boy!
+But then it broke
+And he was almost in stroke
+And had to throw it away, oh noy!
+
+
+There once was a man with a bike
+Who liked to go on rides that were alike
+But then came a hill
+And his energy was killed
+And he had to stop and take a hike
+
+
+There once was a woman with a phone
+That she used to connect, talk and groan
+But then came a message
+And she read with a presage
+And her heart skipped a beat, all alone
+
+
+There once was a frog in a pond
+Who thought he was king, oh so fond
+But then came a snake
+And the frog was in stake
+And his kingdom vanished like a bond
+
+
+There once was a boy with a book
+That he read every day with a hook
+But then came a test
+And he failed, it was the best
+And he learned to study more, no rook
+
+
+There once was a woman from Spain
+Whose cooking was known as quite insane
+But then came a guest
+And the dish was a pest
+And she had to start over again, with a grain
+
+
+There once was a man with a nose
+So big, it got stuck in his clothes
+He tried to get it out
+But it was like a trout
+And he walked around looking like a rose
+
+
+There once was a woman named Glenda
+Whose hobby was collecting agenda
+She had hundreds of them
+All in a hem
+And her friends thought she was just a pretenda
+
+
+There once was a dog with no tail
+Who liked to chase cars on a trail
+But then one day he caught one
+And it was quite fun
+And the car started wagging like a sail
+
+
+There once was a boy with a spoon
+Who liked to use it as a harpoon
+He aimed for a fish
+But missed and went swish
+And ended up in a balloon
+
+
+There once was a girl with a dress
+That she wore in a way, quite a mess
+She put it on upside down
+And went into town
+And everyone thought she was just impressed
+
+
+There once was a man with a hat
+That he wore even when he sat
+But then came a bird
+And the hat was absurd
+And he went around looking like a rat
+
+
+There once was a cat named Joe
+Who liked to eat things that weren't foe
+He ate a whole cake
+And a pie in a bake
+And his belly started to grow
+
+
+There once was a woman from France
+Whose cooking was known to enhance
+She put in too much spice
+And it wasn't too nice
+And her guests had to go dance
+
+
+There once was a boy with a toy
+That he used to play with, oh boy!
+He played all day and night
+And it was quite a sight
+And his parents thought he was their coy
+
+
+There once was a man from the moon
+Who thought he was coming to Earth soon
+But then he got lost
+And his mission was tossed
+And he ended up in a cocoon
+
+
+There once was a woman with a shoe
+That she wore every day, like a glue
+But then came a hole
+And her foot went cold
+And she had to buy something new
+
+
+There once was a frog with a hat
+That he wore like a cool cat
+But then came a storm
+And his hat was the norm
+And he went on a quest to get back
+
+
+There once was a boy with a ball
+That he bounced and bounced, and had a ball
+But then came a crack
+And the ball went whack
+And he had to find something else to enthral
+
+
+There once was a man with a beard
+That he thought was quite weird
+He shaved it one day
+And went out to play
+And everyone thought he was a revered
+
+
+There once was a woman with a cake
+That she baked and baked, and took a break
+But then came a mouse
+And ate the whole house
+And she had to start from scratch, for Pete's sake!
+
+
+There once was a boy with a kite
+That he flew all day and all night
+But then came a hawk
+And the kite was a crock
+And the boy had to find a new sight
+
+
+There once was a girl with a book
+That she read in every nook
+But then came a page
+That was quite a rage
+And she had to find a new hook
+
+
+There once was a man with a car
+That he drove like a superstar
+But then came a flat
+And he had to chat
+And ended up walking afar
+
+
+There once was a woman named Marge
+Whose obsession was with a large
+She collected them all
+And hung them on the wall
+And it looked like a barge
+
+
+There once was a boy with a bird
+That he talked to and called "my word"
+But then came a cat
+And the bird was a rat
+And the boy was left feeling absurd
+
+
+There once was a man with a phone
+That he used to talk and groan
+But then came a glitch
+And it turned into a witch
+And he was left feeling alone
+
+
+There once was a girl with a dream
+To fly to the moon, or so it would seem
+But then came a storm
+And her dream was the norm
+And she was left feeling supreme
+
+
+There once was a cat with a hat
+That he wore to the park and all that
+But then came a dog
+And the hat was a slog
+And the cat had to find a new chat
+
+
+There once was a boy with a ball
+That he kicked and kicked, and had a ball
+But then came a net
+And he was left with regret
+And had to find a new call
+
+
+There once was a woman with a spoon
+That she used to play a funny tune
+But then came a crowd
+And it was a little too loud
+And she had to find a new boon
+
+
+There once was a man with a hat
+That he wore even when he sat
+But then came a bee
+And it was quite a spree
+And the man went running like a rat
+
+
+There once was a dog with a bone
+That he liked to chew and moan
+But then came a bird
+And the bone was a turd
+And the dog was left with a groan
+
+
+There once was a boy with a pen
+That he used to write stories, now and then
+But then came a spill
+And the pen went still
+And he had to find a new den
+
+
+There once was a woman with a hat
+That she wore like a mat
+But then came a gust
+And the hat turned to dust
+And the woman was left with a spat
+
+
+There once was a man with a cat
+That he trained to wear a hat
+But then came a mouse
+And the cat was a grouse
+And the man had to find a new pat
+
+
+There once was a girl with a kite
+That she flew up high and out of sight
+But then came a storm
+And the kite was forlorn
+And the girl was left with a plight
+
+
+There once was a boy with a bike
+That he liked to ride and hike
+But then came a bump
+And the bike took a dump
+And the boy was left with a strike
+
+
+There once was a woman with a phone
+That she used to talk and moan
+But then came a beep
+And the phone went to sleep
+And the woman was left all alone
+
+
+There once was a man with a beard
+That he thought was quite weird
+But then came a trend
+And the beard was his friend
+And the man was quite cheered
+
+
+There once was a boy with a ball
+That he kicked around, feeling tall
+But then came a hole
+And the ball lost its role
+And the boy was left with a drawl
+
+
+There once was a woman with a cake
+That she baked and baked, feeling awake
+But then came a burn
+And the cake took a turn
+And the woman was left with a quake
+
+
+There once was a man with a fish
+That he liked to put on a dish
+But then came a cat
+And the fish was flat
+And the man was left with a swish
+
+
+There once was a girl with a toy
+That she played with and felt joy
+But then came a crack
+And the toy was a hack
+And the girl was left feeling coy
+
+
+
+

main.py

@@ -19,7 +19,7 @@ def load_base_model():
         'decapoda-research/llama-7b-hf',
         load_in_8bit=True,
         torch_dtype=torch.float16,
-        device_map='auto'
+        device_map={'':0}
     )
 
 def load_tokenizer():
@@ -371,13 +371,13 @@ def inference_tab():
             with gr.Column():
                 #  temperature, top_p, top_k, repeat_penalty, max_new_tokens
                 temperature = gr.Slider(
-                    minimum=0.01, maximum=1.99, value=0.4, step=0.01,
+                    minimum=0.01, maximum=1.99, value=0.1, step=0.01,
                     label="Temperature",
                     info="Controls the 'temperature' of the softmax distribution during sampling. Higher values (e.g., 1.0) make the model generate more diverse and random outputs, while lower values (e.g., 0.1) make it more deterministic and focused on the highest probability tokens."
                 )
 
                 top_p = gr.Slider(
-                    minimum=0, maximum=1, value=0.3, step=0.01,
+                    minimum=0, maximum=1, value=0.75, step=0.01,
                     label="Top P",
                     info="Sets the nucleus sampling threshold. In nucleus sampling, only the tokens whose cumulative probability exceeds 'top_p' are considered  for sampling. This technique helps to reduce the number of low probability tokens considered during sampling, which can lead to more diverse and coherent outputs."
                 )
@@ -389,7 +389,7 @@ def inference_tab():
                 )
 
                 repeat_penalty = gr.Slider(
-                    minimum=0, maximum=2.5, value=1.0, step=0.01,
+                    minimum=0, maximum=2.5, value=1.2, step=0.01,
                     label="Repeat Penalty",
                     info="Applies a penalty to the probability of tokens that have already been generated, discouraging the model from repeating the same words or phrases. The penalty is applied by dividing the token probability by a factor based on the number of times the token has appeared in the generated text."
                 )