End of training

all_results.json

@@ -0,0 +1,14 @@
+{
+    "epoch": 78.78,
+    "eval_cola_loss": 0.9552000761032104,
+    "eval_cola_matthews_correlation": 0.10724809104897279,
+    "eval_cola_runtime": 7.8141,
+    "eval_cola_samples_per_second": 133.477,
+    "eval_cola_steps_per_second": 16.765,
+    "eval_samples": 1043,
+    "train_loss": 0.40566940747774566,
+    "train_runtime": 7554.577,
+    "train_samples": 8551,
+    "train_samples_per_second": 508.301,
+    "train_steps_per_second": 1.986
+}

cola_outputs.csv

@@ -0,0 +1,1044 @@
+sentence	label	prediction	length
+The sailors rode the breeze clear of the rocks.		1	1	47
+The weights made the rope stretch over the pulley.		1	1	50
+The mechanical doll wriggled itself loose.		1	1	42
+If you had eaten more, you would want less.		1	1	43
+As you eat the most, you want the least.		0	1	40
+The more you would want, the less you would eat.		0	1	48
+I demand that the more John eat, the more he pays.		0	0	50
+Mary listens to the Grateful Dead, she gets depressed.		1	1	54
+The angrier Mary got, the more she looked at pictures.		1	1	54
+The higher the stakes, the lower his expectations are.		1	1	54
+The more Fred is obnoxious, the less attention you should pay to him.		1	1	69
+John was lots more obnoxious than Fred.		1	1	39
+The more people you give beer to, the more people get sick.		1	1	59
+The more does Bill smoke, the more Susan hates him.		0	1	51
+The more pictures of him that appear in the news, the more embarrassed John becomes.		1	1	84
+Every senator seems to become more corrupt, as he talks to more lobbyists.		1	1	74
+Who does John visit Sally because he likes?		0	0	43
+Marianne did not leave.		1	1	23
+He could not] have been working.		1	1	32
+He can not have been working.		1	1	29
+You will believe Bob.		1	1	21
+John has not kissed Mary.		1	1	25
+I said that never in my life had I seen a place like Bangor.		1	1	60
+Mickey looked up it.		0	0	20
+There tended to be a lot of discussion.		1	1	39
+John tried to be a good boy.		1	1	28
+John is eager.		1	1	14
+We want John to win.		1	0	20
+The box contained the ball from the tree.		0	1	41
+The tube was escaped by gas.		0	1	28
+Water bubbled up out of the kettle.		1	1	35
+The tub leaked water.		1	1	21
+What the water did to the bottle was fill it.		0	1	45
+What the water did to the whole bottle was fill it.		0	1	51
+The tank leaked the fluid free.		1	1	31
+John lay the ball in the box.		1	1	29
+John owns the book.		1	1	19
+We persuaded Mary to leave and Sue to stay.		1	1	43
+Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder.		1	1	99
+Mary beautifully plays the violin.		0	1	34
+Clearly, John probably will immediately learn French perfectly.		1	0	63
+Sue gave to Bill a book.		0	0	24
+The men will all leave.		1	1	23
+John went home.		1	1	15
+They represented seriously to the dean Mary as a genuine linguist.		0	0	66
+Us love they.		0	0	13
+It is nice to go abroad.		1	1	24
+Mary intended John to go abroad.		0	1	32
+I remembered having kissed Mary.		1	1	32
+I can't believe Fred won't, either.		1	1	35
+John wants to read Fred's story, and I also want to.		1	1	52
+We wanted to invite someone, but we couldn't decide who to.		0	1	59
+Mary will read Fred's story, and Joe will read Holly's.		1	1	55
+Mary claimed that eating cabbage, Holly shouldn't.		1	0	50
+Mary came to be introduced by the bartender and I also came to be.		0	0	66
+If I can, I will work on it.		1	1	28
+Joe's neuroses bother his patrons, and Sally does too.		1	1	54
+I know which book José didn't read for class, and which book Lilly did it for him.		0	1	82
+This is the book which Bob reviewed, and this is the one which Fred won't do it.		0	0	80
+I know which book Mag read, and which book Bob said that you hadn't.		1	0	68
+I know which book Mag read, and which book Bob read my report that you hadn't.		0	0	78
+I'm sure I would like him to eat fruit more than I would cookies.		1	0	65
+Rusty talked about himself only after Mary did talk about him.		1	1	62
+Fred talked about everything before Rusty did talk about something.		1	1	67
+John often meets Mary.		1	1	22
+The problem perceives easily.		0	0	29
+A hundred men surrounded the fort.		1	1	34
+We elected me.		1	0	14
+Which report that John was incompetent did he submit?		0	1	53
+Mary has always preferred lemons to limes.		1	0	42
+He let the cats which were whining out.		1	1	39
+What did Bill buy?		1	1	18
+Mary saw the boy walking toward the railroad station.		1	1	53
+A proof that the claim had been. made was giver that John had lied.		0	1	67
+He attributed to a short circuit which was caused by an overloaded transducer the fire which destroyed most of my factory.		1	1	122
+The mayor regarded as being absurd the proposal to build a sidewalk from Dartmouth to Smith.		0	1	92
+I want that Bill left to remain a secret.		0	0	41
+I know a man who Tom drives as drives.		0	0	38
+Drowning cats, which is against the law, are hard to rescue.		0	1	60
+Muriel said nothing else than that she had been insulted.		1	1	57
+Himself is understood by Rutherford.		0	0	36
+I feel that Arch will show up.		1	1	30
+The proof this set is recursive is difficult.		0	1	45
+The madrigals which Henry plays the lute and sings sound lousy.		0	1	63
+Tom picked these grapes, and I washed some turnips, and Suzie will prepare these grapes.		1	1	88
+Where did you go and who ate what?		1	1	34
+Which boy's did we elect guardian's employer president?		0	0	55
+How sane is Peter?		1	1	18
+I live at the place where Route 150 crosses the River and my dad lives at the place where Route 150 crosses the Hudson River too.		1	1	129
+I live at the place where Route 150 crosses the Hudson River and my dad lives at it too.		0	1	88
+Who is she trying to make up to now?		1	1	36
+Wind was gotten of a plot to negotiate an honorable end to the war in Vietnam.		0	1	78
+Mike talked about politics yesterday to my friends.		1	1	51
+It was expected by the reporters that the principal would fire some teacher.		1	1	76
+Which hat did Mike quip that she never wore?		0	1	44
+Which girl did Mike quip never wore this hat?		0	1	45
+We donated wire for the convicts to build cages with.		1	1	53
+I won't have some money.		0	1	24
+Do you believe the claim that somebody was looking for something?		1	1	65
+I won't ask you to believe that he tried to force me to give her any money.		1	1	75
+That Sam sometimes didn't sleep must have pleased somebody.		1	1	59
+I talked to Winston about himself.		1	0	34
+That the fuzz wanted him worried John, but that the fuzz wanted her didn't worry Mary.		1	0	86
+I'll work on it if Sam will be working on it.		1	1	45
+I'll work on it if I can.		1	1	25
+Here's a knife with which for you to cut up the onions.		0	1	55
+Fluffy is sick, which not everybody knows.		1	0	42
+Maxwell is quite a doctor.		1	1	26
+The younger woman might have been tall and, and the older one definitely was, blond.		0	1	84
+Sally is tall, and may be blond, and Sheila is short, and definitely is, blond.		1	0	79
+I have to try to finish grading some papers.		1	1	44
+The socks are ready for for you to put on to be planned.		0	1	56
+It is easy to play sonatas on this violin.		1	1	42
+This violin is difficult to play sonatas on.		1	1	44
+My mother is easy to please my father and.		0	0	42
+Poor Bill, it had started to rain and he had no umbrella.		1	1	57
+That the cops spoke to the janitor about it yesterday is terrible, that robbery.		0	1	80
+Every student, and he wears socks, is a swinger.		0	1	48
+That girl was given my binoculars by him.		1	1	41
+Bill didn't allege that Roger had eaten anything.		1	1	49
+Nobody who hates to eat anything should work in a delicatessen.		1	1	63
+Everybody around here who ever buys anything on credit talks in his sleep.		1	1	74
+I can't remember the name of somebody who had misgivings.		0	1	57
+No writer, and no playwright, meets in Vienna.		0	1	46
+No writer, nor any playwright, meets in Vienna.		0	1	47
+That you will marry any student is not certain.		1	1	47
+Felicia kicked the ball off the bench.		1	0	38
+I sent the package halfway around the world.		1	1	44
+Sam gave the ball out of the basket.		0	1	36
+Sam offered the ball out of the basket.		0	1	39
+Park Square has a festive air.		1	1	30
+The worker will have a job.		1	1	27
+No one can forgive that comment to you.		0	1	39
+We launched the rocket to the moon, but it blew up before it got there.		1	1	71
+Sarah promised Catherine her old car, but then gave it to her son instead.		1	1	74
+I lent the book partway to Tony.		0	0	32
+The farmer loaded the cart with apples.		1	1	39
+The farmer dumped the cart with apples.		0	1	39
+Martha carved the baby a toy out of wood.		1	1	41
+The bread cuts easily.		1	0	22
+Janet broke Bill on the finger.		0	1	31
+Janet broke the cup.		1	1	20
+The visitor rang the bell.		1	1	26
+We pulled free.		1	0	15
+That movie always shocks people.		1	1	32
+That movie always shocks.		1	1	25
+Sharon came the room.		0	1	21
+Bill sent a package to Tom.		1	1	27
+That acorn will grow into an oak tree.		1	1	38
+He turned into a frog.		1	1	22
+I mixed the sugar into the butter.		1	1	34
+Brian threw the fence with the stick.		0	1	37
+Mira condemned Terry for the accident.		1	0	38
+We investigated the area for bombs.		1	0	35
+I sensed his eagerness.		1	1	23
+They praised the dedication in the volunteers.		0	1	46
+The earth was believed to be round.		1	1	35
+Sarah smiled a charming smile.		1	1	30
+Sandra beamed a cheerful welcome.		1	1	33
+You've really lived it up.		1	1	26
+Paperback books lift onto the table easily.		0	1	43
+The books lifted onto the table.		0	1	32
+Jessica loaded boxes under the wagon.		1	1	37
+Jessica loaded boxes on the wagon.		1	1	34
+Jessica crammed boxes at the truck.		0	0	35
+Lora buttered at the toast.		0	0	27
+Carla shoveled the walk.		1	1	24
+Nora sent the book.		1	1	19
+Nora sent Peter the book.		1	1	25
+Carla slid the book.		1	1	20
+Carla slid at the book.		0	0	23
+Amanda carried the package to Pamela.		1	1	37
+Packages drive easily to New York.		0	0	34
+The chair pushed.		0	1	17
+We offered a job to her.		1	0	24
+A job offered.		0	0	14
+Brown presented a plaque to Jones.		1	1	34
+Carmen bought Mary a dress.		1	1	27
+Carmen obtained the spare part at the hardware store.		1	1	53
+Michelle kept the desk with the papers.		0	0	39
+Frances hid the presents in the drawer.		1	1	39
+The needle poked the cloth.		1	1	27
+Carrie touched that cat.		1	1	24
+Herman whipped the sugar and the cream.		0	1	39
+Linda taped the picture to the wall.		1	1	36
+Linda taped the picture onto the wall.		1	1	38
+The child and her mother clung together.		1	1	40
+This flyer and that flyer differ.		1	1	33
+This flyer and that flyer differ apart.		0	1	39
+The jeweller scribbled the contract with his name.		0	1	50
+The gardener grew that acorn into an oak tree.		1	1	46
+I shaped a loaf.		0	0	16
+The children amused.		0	1	20
+Susan whispered the news.		1	1	25
+Susan whispered at Rachel.		1	0	26
+Ellen said that melons were selling well.		1	1	41
+Ellen said about the present conditions.		0	1	40
+Ellen warned Helen against skating on thin ice.		1	1	47
+Cynthia nibbled on the carrot .		1	1	31
+Cynthia chewed.		0	0	15
+Paul laughed at Mary.		1	0	21
+Linda winked her lip.		0	1	21
+My heart is pounding me.		0	0	24
+Sharon fainted from hunger.		1	1	27
+The witch poisoned the children.		1	1	32
+A grandfather clock ticked in the hallway.		1	1	42
+I squeaked the door.		0	0	20
+Over the fire there bubbled a fragrant stew.		1	1	44
+Soaring temperatures are predicted for this weekend.		1	1	52
+The fort fluttered with many flags.		0	1	35
+The voices echoed in the hall.		1	1	30
+The stream twists through the valley.		1	1	37
+There presented itself a wonderful opportunity yesterday.		0	0	57
+A wonderful opportunity presented itself to him yesterday.		1	1	58
+Out of the box jumped a little white rabbit.		1	0	44
+Penny skated around the rink.		1	0	29
+Jackie accompanied Rose.		1	1	24
+many information was provided.		0	1	30
+John offers many advice.		0	0	24
+The cake that Jones got was more delicious than the one that Smith got.		1	1	71
+We recommend to eat less cake and pastry.		1	1	41
+I saw that gas can explode.		1	1	27
+He washed her.		1	1	14
+Wash you!		0	1	9
+They have no in.		0	0	16
+Some my jobs are in jeopardy.		0	0	29
+It was the policeman met that several young students in the park last night.		0	1	76
+did the policeman meet in the park?		1	1	35
+John put old books in the box.		1	1	30
+The monkeys proud of their leader.		0	1	34
+John sounded very.		0	0	18
+I don't know if I should agree.		1	1	31
+John bought a book on the table.		1	1	32
+John called the president a fool.		1	1	33
+I forgot how good beer tastes.		1	1	30
+This teacher is a genius.		1	0	25
+A good friend is remained to me by him.		0	0	39
+John ate his noodle quietly.		1	1	28
+A smith hammered the metal.		1	1	27
+Sandy removed her ballet shoes.		1	0	31
+This week will be a difficult one for us.		1	1	41
+Benny worked in a shoe factory when he was a student.		1	1	53
+Everyone hoped that she would sing.		1	1	35
+This proved a decisive factor.		1	1	30
+The crocodile devoured the doughnut.		1	1	36
+John is afraid of Bill.		1	1	23
+John put under the bathtub.		0	1	27
+John placed Kim behind the garage.		1	0	34
+Kim depends for Sandy.		0	1	22
+John put the book in the box.		1	1	29
+John taught English Syntax to new students.		1	1	43
+John regards Bill as a good friend.		1	1	35
+Kim put in the box.		0	1	19
+The chickens seem fond of the farmer.		1	1	37
+The rules require that the executives be polite.		1	1	48
+That Fred was unpopular nominated Bill.		0	1	39
+They were taking a hard look at possible FTA.		1	1	45
+John paid me against the book.		0	0	30
+The committee will study the feasibility of setting up a national computer network.		1	1	83
+We made them be rude.		1	1	21
+After reading the pamphlet, Judy threw it into the garbage can.		1	0	63
+Students studying English reads Conrad's Heart of Darkness while at university.		0	1	79
+I read some of the book.		1	1	24
+Most of the fruit is rotten.		1	1	28
+Neither of students failed.		0	1	27
+I drank some of water.		0	1	22
+It is a golden hair.		1	1	20
+The boy in the doorway waved to his father.		1	1	43
+That dog is so ferocious, it even tried to bite itself.		1	0	55
+He washed yourself.		0	0	19
+John is easy to please Kim.		0	1	27
+There seemed to be intelligent.		0	1	31
+John tried to please Stephen.		1	1	29
+We believed John to be a fountain in the park.		0	1	46
+It tries to leave the country.		0	1	30
+John tries to leave the country.		1	1	32
+Under the bed seems to be a fun place to hide.		1	1	46
+There is believed to be sheep in the park.		0	1	42
+I hope to would study in France.		0	1	32
+John can kick the ball.		1	0	23
+John will rain tomorrow.		0	1	24
+We expect there to will rain.		0	0	29
+John was found in the office.		1	1	29
+Did the child be in the school?		0	1	31
+It did not rain.		1	1	16
+John wants not to leave the town.		1	1	33
+Mary likes to tour art galleries, but Bill hates to.		1	1	52
+They needn't take this exam.		1	1	28
+Ann may spend her vacation in Italy.		1	0	36
+They love to play golf, but I do not.		1	1	37
+George has spent a lot of money, hasn't he?		1	1	43
+As a statesman, scarcely could he do anything worth mentioning.		1	0	63
+We never found any of the unicorns.		1	1	35
+One of Korea's most famous poets wrote these lines.		1	1	51
+Somebody apparently struck the unidentified victim during the early morning hours.		1	1	82
+The car was driven.		1	1	19
+Ricky can be relied on.		1	1	23
+The bed was slept in.		1	1	21
+A pound was weighed by the book.		0	1	32
+Who do you think Tom saw?		1	1	25
+he had spent five thousand dollars.		1	1	35
+Fed knows which politician her to vote for.		0	1	43
+How did you guess that he fixed the computer?		1	1	45
+The committee knows whose efforts to achieve peace the world should honor.		1	1	74
+Which house does your friend live?		0	1	34
+the baker from whom I bought these bagels left.		1	1	47
+I found the place where we can relax.		1	1	37
+Jack is the person with whom Jenny fell in love with.		0	1	53
+There is a bench to sit on.		1	1	27
+I met the man who grows peaches.		1	0	32
+Which topic did you choose without getting his approval?		1	1	56
+Which topic did you get bored because Mary talked about?		0	1	56
+That is the reason why he resigned.		1	1	35
+It bothers me that John coughs.		1	1	31
+To please John is easy.		1	0	23
+Kim is eager to please Tom.		1	1	27
+That we invaded Iraq really bites.		1	1	34
+It annoys me that Fido barks.		1	1	29
+Who achieved the best result was Angela.		0	1	40
+It was the peasant girl who got it.		1	1	35
+That kind of person is hard to find anyone to look after.		1	1	57
+A sick owl doesn't hunt mice.		1	1	29
+Each candidate who has interest in semantics will be admitted to the department.		1	1	80
+Each author whose contribution is written in any language other than English will provide a summary in English.		1	1	111
+I'm sure we even got these tickets!		1	1	35
+I'm even sure we got these tickets!		1	1	35
+It's not because I have any sympathy for urban guerillas that I helped him.		1	1	75
+It isn't because Sue said anything bad about me that I'm angry.		1	0	63
+That he was hungry, John whined.		0	1	32
+1 gave Mary after the party a book.		0	1	35
+Because she's so pleasant, as for Mary I really like her.		0	0	57
+Though he may hate those that criticize Carter, it doesn't matter.		1	1	66
+With no job would John be happy.		1	1	32
+I have much of the manuscript left to type.		1	1	43
+He's a more reliable man.		1	0	25
+Any trouble is what I don't want.		0	0	33
+They may grow as high as bamboo.		1	1	32
+Some of them made as many errors as 20.		1	1	39
+Sally kissed himself.		0	1	21
+Drew believes I think Rosie loves magazine ads.		1	1	47
+Dave, Dan, Erin, Jaime, and Alina left.		1	1	39
+Hopefully, we'll make it through the winter without snow.		1	0	57
+Blue leather shows herself that Betsy is pretty.		0	0	48
+Gwen hit the baseball.		1	1	22
+I hit that you knew the answer.		0	1	31
+I've lost my wallet or I've lost my mind.		1	1	41
+It was a brand new car that he bought.		1	1	38
+He likes cookies and he hates crumb cake.		1	1	41
+They chased the man with the car.		1	1	33
+I didn't have a red cent.		1	1	25
+the book of poems and from Blackwell takes a very long time to read.		0	1	68
+the one with a red cover takes a very long time to read.		1	1	56
+John has a fear of dogs.		1	0	24
+The building's the roof is leaking.		0	1	35
+the panther's the coat is dark black.		0	1	37
+Colin asked if they could get a mortgage.		1	1	41
+The man I saw get into the cab robbed the bank.		1	1	47
+I know you eat asparagus.		1	1	25
+John's drum will always bother me.		1	1	34
+An evil thought struck Dave.		1	1	28
+In the classroom John put the book on the table.		1	1	48
+Phillip gave the medal to the soldier.		1	1	38
+Heidi thinks that Andy to eat salmon flavored candy bars.		0	1	57
+Heidi thinks that Andy should eat salmon flavored candy bars.		1	1	61
+He danced.		1	1	10
+Did Calvin his homework?		0	1	24
+Sylvia was slapping Jeff upside the head in martial arts class.		1	1	63
+If I am a rich man, I'd buy a diamond ring.		0	1	43
+If he were a rich man, he'd buy a diamond ring.		1	1	47
+John is likely to leave.		1	0	24
+The manager laughed.		1	0	20
+Was sunk.		0	0	9
+Who did you think kissed the gorilla?		1	1	37
+Have you seen my model airplane collection?		1	1	43
+It seems that Lucy was mugged.		1	1	30
+I ate a salad that was filled with lima beans.		1	1	46
+There were four men arriving at the station when I pulled up.		1	0	61
+Jean is reluctant to dance.		1	1	27
+I want her to dance.		1	1	20
+Jean persuaded Robert.		1	0	22
+Jean wants Bill to do the Macarena.		1	1	35
+The children admire their mother.		1	1	33
+Who has Peter talked with?		1	1	26
+Heidi likes her violin.		1	1	23
+John thinks that Mary loves himself.		0	1	36
+Which pictures of himself does John like?		1	1	41
+Any owl hunts mice.		1	1	19
+Any man didn't eat dinner.		0	1	26
+A pilot could be flying this plane.		1	1	35
+Everybody who attended last week's huge rally, whoever they were, signed the petition.		1	1	86
+Anybody who attended last week's huge rally signed the petition.		1	1	64
+Any tiger has orange fur, marked with black stripes.		1	1	52
+Any albino tiger has orange fur, marked with black stripes.		1	1	59
+You must pick any flower you see.		1	0	33
+Any pilot on duty today could be flying this plane.		1	1	51
+You may pick every flower, but leave a few for Mary.		0	1	52
+The Dodgers beat the Red Sox and were beaten by the Giants.		1	1	59
+In which car was the man seen?		1	1	30
+The man who Mary loves and Sally hates computed my tax.		1	1	55
+The kennel which Mary made and Fido sleeps has been stolen.		0	1	59
+Tom said he would and Bill actually did eat a raw eggplant.		1	0	59
+The wealthy young man bought that piano for his secret fiancée.		1	1	63
+The dog stole the turkey.		1	0	25
+a tall building.		1	1	16
+This building is tall.		1	0	22
+I like the book which you gave me.		1	1	34
+Captain Wentworth wrote a letter to Anne Elliott.		1	1	49
+She asked was Alison coming to the party.		1	1	41
+They realised that never had Sir Thomas been so offended.		1	0	57
+Fanny regretted having to talk to Aunt Norris.		1	1	46
+Knowing the country well, he took a short cut.		1	1	46
+He left the train with somebody else's wallet in his pocket.		1	1	60
+Magnus went to Ireland.		1	1	23
+Who did John send the book?		1	1	27
+The idea dismayed the Prime Minister that the Dome was dull.		1	1	60
+Came right in he did without so much as a knock.		1	1	48
+Harriet admired Mr Knightley.		1	1	29
+The book is boring.		1	1	19
+Ethel wishes to ask you some awkward questions.		1	1	47
+Emma made Harriet some food.		1	1	28
+The window was broken with a hammer.		1	1	36
+The fugitive lay motionless in order to avoid discovery.		1	1	56
+The guard marched the prisoners round the yard.		1	1	47
+Frank Churchill crossed the street.		1	1	35
+Emma and Harriet were attacked yesterday.		1	1	41
+Fiona might be here by 5 o'clock.		1	1	33
+I am both expecting to get the job and of the opinion that it is a desirable one.		1	1	81
+Pat was awarded the Golden Fleece Award and very upset about it.		1	1	64
+Kim alienated cats and beating his dog.		0	1	39
+They knew that pictures of each other would be on sale.		1	1	55
+Which article did Terry file papers without reading?		0	1	52
+I want to try and buy some whiskey.		1	1	35
+She goes and buying some whiskey.		0	1	33
+Kim and Terry is happy.		0	1	23
+Kim gave a dollar to Bobbie and a dime to Jean.		1	1	47
+This girl in the red coat will put a picture of Bill in the mailbox and on your desk before tomorrow.		1	1	101
+Put a picture of Bill on your desk before tomorrow, this girl in the red coat will put a picture of Bill on your desk before tomorrow.		0	0	134
+Picture of Bill, this girl in the red coat will put a picture of Bill on your desk before tomorrow.		0	0	99
+Ann bought a first edition of Richard III for $1000.		1	1	52
+John became deadly afraid of flying.		1	1	36
+What John promised is to be gentle.		1	1	35
+Henri wants to buy which books about cooking?		1	1	45
+I will fix the computer for Carol.		1	1	34
+Smith loaned a valuable collection of manuscripts to the library.		1	1	65
+The paper is cut by Mary.		1	1	25
+It was from six to nine or it took place from six to nine.		1	1	58
+Suddenly, there arrived two inspectors from the INS.		1	1	52
+The soup cooks, thickens.		1	0	25
+I gave money to the charity.		1	1	28
+John asked.		1	1	11
+I saw even the student.		1	1	23
+John whispered Mary left.		0	1	25
+Mary wonders that Bill will come.		0	1	33
+Sophie will theater.		0	1	20
+John finished the cake and drank the lemonade.		1	1	46
+Herself likes Mary's mother.		0	0	28
+Each of the boys fought with some of the other boys.		1	1	52
+John's mother likes himself.		0	1	28
+Mary revealed himself to John.		0	1	30
+Mary believes that Bill saw himself.		1	0	36
+John heard that they criticized themselves.		0	1	43
+Mary thinks that she is smart.		1	1	30
+Henry found that Bill is sad.		1	1	29
+John considers himself proud of Mary.		1	1	37
+You should sit before there.		0	0	28
+There is a nurse available.		1	0	27
+Everyone hopes everyone to sleep.		0	1	33
+Everyone hopes that he will sleep.		1	1	34
+Only Churchill remembered Churchill giving the Blood, Sweat and Tears speech.		0	1	77
+John promise Mary to shave himself.		1	1	35
+John convinced Bill that Mary should sleep.		1	0	43
+I saw Bill while you did so Mary.		0	1	33
+What Mary did Bill was give a book.		0	1	35
+The bears sniffed		1	0	17
+to steal talismans from witches is dangerous		1	1	44
+Anson demonized		0	1	15
+You kicked yourselves		1	1	21
+The book that I said that I'd never read.		1	0	41
+Anson demonized up the Khyber		0	1	29
+It is not true that I might be going to the party.		1	1	50
+Kick you!		0	1	9
+Jason persuaded the doctor to treat Medea		1	1	41
+Who did you get an accurate description of?		1	1	43
+The Peter we all like was at the party		1	1	38
+The government's imposition of a fine.		1	1	38
+Did Athena help us?		1	1	19
+You wanted to meet the man I saw.		1	1	33
+Anson demonized David every day		1	1	31
+There is a programme about Euripides on a Radio 4 tonight?		1	1	58
+Medea denied poisoning the phoenix.		1	1	35
+The elixir was hidden in a hole in the ground		1	1	45
+Extremely frantically, Anson danced at Trade		1	1	44
+What did you ask who Medea gave?		0	1	32
+It's Anson that I like		1	1	22
+Medea tried the nurse to poison her children.		0	1	45
+I felt that I know you.		1	1	23
+Will he can do it?		1	0	18
+I ate that she was happy		0	0	24
+Anson believed himself to be handsome.		1	1	38
+We arrived about six o'clock		1	0	28
+Gilgamesh is having fought the dragon.		0	1	38
+Anson thought that himself was happy.		0	0	37
+Fortunately Hermione passed biology		1	1	35
+I might be not going to the party but washing my hair		1	0	53
+I thought he liked myself		0	0	25
+The butler sent the poison to Dinah.		1	1	36
+Jason happens to appear to seem to be sick.		1	1	43
+What did you ask who saw?		0	1	25
+We linguists love to argue		1	1	26
+He can will go		0	1	14
+How fierce the battle?		0	1	22
+Which king did you ask which city invaded?		0	1	42
+It is some disgruntled old pigs in those ditches that humans love to eat.		1	1	73
+That banana is eating the monkey.		1	1	33
+Burn them!		1	1	10
+It stinks that Aphrodite is omnipotent.		1	1	39
+Agamemnon seems to be a maniac		1	1	30
+Will Anson come to the party?		1	1	29
+I wondered who Medea had poisoned.		1	0	34
+I inquired when could we leave.		0	0	31
+They kicked them		1	1	16
+The monkey is ate the banana		0	1	28
+I would like to could swim		1	0	26
+I kicked myself		1	0	15
+The bookcase ran		0	0	16
+I shaved myself.		1	0	16
+Anson became a muscle bound.		1	1	28
+Somebody just left - guess who.		1	1	31
+They claimed they had settled on something, but it wasn't clear what they had settled on.		1	1	89
+If Sam was going, Sally would know where.		1	1	41
+They're going to serve the guests something, but it's unclear what.		1	1	67
+She's reading. I can't imagine what.		1	0	36
+John said Joan saw someone from her graduating class.		1	1	53
+John ate dinner but I don't know who.		0	0	37
+She mailed John a letter, but I don't know to whom.		0	0	51
+I served leek soup to my guests.		1	1	32
+I served my guests.		1	1	19
+She was bathing, but I couldn't make out who.		0	1	45
+She knew French for Tom.		0	1	24
+John is tall on several occasions.		0	1	34
+The ship sank, but I don't know with what.		0	1	42
+They noticed the painting, but I don't know for how long.		0	1	57
+John was tall, but I don't know on what occasions.		0	1	50
+Joan ate dinner with someone but I don't know who.		1	1	50
+Joan ate dinner with someone but I don't know who with.		1	1	55
+I know that Meg's attracted to Harry, but they don't know who.		0	0	62
+Since Jill said Joe had invited Sue, we didn't have to ask who.		0	1	63
+I know that Meg's attracted to Harry, but they don't know who.		1	0	62
+She said she had spoken to everybody, but he wasn't sure who.		0	0	61
+Each of the performers came in, but were sitting so far back that we couldn't see who.		0	1	86
+She didn't talk to one student.		1	1	31
+She doesn't meet anyone for dinner, but they can't figure out who.		0	1	66
+Everyone relies on someone. It's unclear who.		1	1	45
+Each student wrote a paper on a Mayan language, but I don't remember which one.		1	0	79
+The newspaper has reported that they are about to appoint someone, but I can't remember who the newspaper has reported that they are about to appoint.		1	1	150
+The newspaper has reported that they are about to appoint someone, but I can't remember who they are about to appoint.		1	1	118
+Most columnists claim that a senior White House official has been briefing them, and the newspaper today reveals which one.		1	1	123
+Most columnists claim that a senior White House official has been briefing them, but none will reveal which one.		1	1	112
+Bill wondered how many papers Sandy had read, but he didn't care which ones.		1	0	76
+I never know which papers Sandy has read, but I usually know how many.		1	1	70
+Sandy had read how many papers?!		1	1	32
+Everybody gets on well with a certain relative, but often only his therapist knows which one.		1	1	93
+Which book did each author recommend?		1	0	37
+His or her least known work.		1	1	28
+They were going to meet sometime on Sunday, but the faculty didn't know when.		1	1	77
+John likes some students, but I don't know who.		1	1	47
+I don't know who John likes.		1	1	28
+John likes some students, but I don't know who John likes some students.		0	1	72
+Joan said she talked to the students, but Fred couldn't figure out who.		0	1	71
+He announced he had eaten the asparagus, but we didn't know what.		0	0	65
+She was reading the books under the table, but Fred didn't know what books.		1	0	75
+He announced he would marry the woman he loved most, but none of his relatives could figure out who.		1	1	100
+She talked to John or Mary but I don't know which.		1	1	50
+She talked to John or Mary but I don't know which one.		1	1	54
+She talked to Harry, but I don't know who else.		1	1	47
+I will see them, but I don't know how many of them.		1	1	51
+Everyone who knows either Susan or Laura likes her.		1	1	51
+She said she talked to three students but I don't know how many.		0	0	64
+She said she talked to those students but I don't know how many.		0	0	64
+He shouted again, but I don't know who to.		1	0	42
+She was dancing with somebody, but I don't know who with.		1	1	57
+Several firefighters were injured, but it's not known.		1	1	54
+Meg is attracted to Harry, but they don't know who she is attracted to.		1	0	71
+Sandy was trying to work out which students would be able to solve a certain problem, but she wouldn't tell us which one.		1	1	121
+Sandy was trying to work out which students would be able to solve a certain problem, but she wouldn't tell us which one.		0	1	121
+John and someone were dancing together, but I don't know who.		0	1	61
+The TA's have been arguing about whether some student or other should pass, but I can't now remember which one.		1	0	111
+It has been determined that somebody will be appointed; it's just not clear yet who.		0	1	84
+Sally asked if somebody was going to fail math class, but I can't remember who.		0	1	79
+The TA's have been arguing about whether some student or other should pass, but I can't now remember which one.		0	1	111
+Sandy is very anxious to see if the students will be able to solve the homework problem in a particular way, but she won't tell us which.		1	0	137
+Sandy is very anxious to see if the students will be able to solve the homework problem in a particular way, but she won't tell us in which way.		1	0	144
+Clinton is anxious to find out which budget dilemmas Panetta would be willing to tackle in a certain way, but he won't say in which.		1	1	132
+Sandy is wondering whether there will be students who have to drop the class for a certain reason, but she won't reveal what.		1	1	125
+In which way is Sandy very anxious to see if the students will be able to solve the homework problem?		0	1	101
+In which way is Clinton anxious to find out which budget dilemmas Panetta would be willing to solve?		0	1	100
+I know how many assignments I've graded, but I don't know how many Bill has.		1	1	76
+What did you leave before they did?		0	1	35
+What did you leave before they started playing?		0	1	47
+Sandy was trying to work out which students would be able to solve a certain problem.		1	1	85
+The administration has issued a statement that it is willing to meet with one of the student groups.		1	1	100
+Sandy was trying to work out which students would be able to solve a problem.		1	1	77
+The administration has issued a statement that it is willing to meet a student group.		1	1	85
+The administration has issued a statement that it is willing to meet a student group, but I'm not sure which one.		1	1	113
+I think Agnes said that Bill would speak, but I don't remember what about.		1	1	74
+Agnes wondered how John could eat but it's not clear what.		0	1	58
+Tony sent Mo a picture that he painted, but it's not clear with what.		0	1	69
+She's been dancing but we don't know with whom.		1	1	47
+Who did they see someone?		0	1	25
+It was believed by everybody that Mary was a thief.		1	1	51
+That professor is feared by all students.		1	1	41
+Mary was respected by John.		1	1	27
+Ted was bitten by the spider.		1	1	29
+The book was by John written.		0	1	29
+The argument was summed by the coach up.		0	1	40
+The paper was written up by John.		1	1	33
+The paper was written by John up.		0	0	33
+John was spoken to by Mary.		1	1	27
+John was spoken by Mary to.		0	1	27
+The book was seen by Mary.		1	1	26
+John was seen the book.		0	0	23
+The book was written.		1	1	21
+John was spoke by Mary to.		0	0	26
+The table was wiped clean by John.		1	1	34
+The table was wiped by John clean.		0	0	34
+Mary was given by John the book.		0	0	32
+John was believed to be telling the truth by Mary.		1	1	50
+John was believed by Mary to be telling the truth.		1	1	50
+The car was driven by John to Maine.		1	1	36
+It was believed by the students that they would have an exam.		1	1	61
+The magazines were sent to herself by Mary.		0	0	43
+Chocolate eggs were hidden from each other by the children.		0	1	59
+The magazines were sent by Mary to herself.		1	0	43
+Chocolate eggs were hidden from no child by any adult.		1	1	54
+Tabs were kept on each agent by the other.		1	1	42
+Chocolate eggs were hidden from every child by his mother.		1	1	58
+Books were taken from no student and given to Mary.		1	1	51
+Books were taken from no student and given to Mary by any professor.		0	1	68
+Books were taken from each student by the other.		1	1	48
+Books were taken from each student and given to Mary.		1	1	53
+Books were taken from each student and given to Mary by the other.		0	1	66
+Jack hates Sue and is loved by Mary.		1	1	36
+Vera sent a baby alligator to Max and a leather dinosaur to Phyllis.		1	1	68
+Either Sam plays the bassoon or Jekyll the oboe.		1	1	48
+Sam doesn't play bassoon, nor Medusa oboe.		1	1	42
+Bill ate the peaches, but Harry the grapes.		0	1	43
+I no more could have stolen that steak than Jack the diamonds.		1	1	62
+Bill ate more peaches than Harry did grapes.		1	1	44
+Bill ate the peaches and Harry did the grapes.		0	1	46
+Tom will smoke the grass, and Reuben has the hash.		0	1	50
+If the ants were called elephants and elephants ants, I'd be able to squash an elephant.		1	1	88
+Simon quickly dropped the gold, and Jack the diamonds.		1	1	54
+Bob tried to wash himself, and Mary to read the funnies.		1	1	56
+Harry told Sue that Albania is a lovely place for a vacation, and Tom told Sally that Albania is a lovely place for a vacation.		1	0	127
+Harry told Sue that Albania is a lovely place for a vacation, and Tom.		1	1	70
+Max seemed to be trying to begin to love Harriet, and Fred to be trying to begin to love Sue.		1	1	93
+Max seemed to be trying to force Ted to leave the room, and Walt, Ira.		1	0	70
+Max seemed to be trying to force Ted to leave the room, and Walt to stay a little longer.		0	1	89
+Arizona elected Goldwater Senator, and Massachusetts, McCormack.		0	1	64
+Millie will send the President an obscene telegram, and Paul, the Secretary a rude letter.		0	1	90
+Maytag will give a brand-new dryer to the winner of the Mrs.		0	1	60
+Bill didn't eat the peaches, nor Harry.		0	1	39
+Bill ate the peaches, and Harry did, too.		1	1	41
+Bill must quickly eat the peaches, and Harry must slowly.		0	1	57
+Whenever Russia has made a major political blunder, the U.S. has too.		1	1	69
+Bill's story about Sue and Max's about Kathy both amazed me.		1	0	60
+I bought three quarts of wine and two of Clorox.		1	1	48
+Scientists at the South Hanoi Institute of Technology have succeeded in raising one dog with five legs, another with a cow's liver, and a third with no head.		1	1	157
+Bill's story about Sue may be amazing, but Max's is virtually incredible.		1	1	73
+I like Bill's yellow shirt, but not Max's.		1	0	42
+Bill's funny story about Sue and Max's boring one about Kathy both amazed me.		1	0	77
+Bill's wine from France and Ted's from California cannot be compared.		1	1	69
+As a teacher, you have to deal simultaneously with the administration's pressure on you to succeed, and the children's to be a nice guy.		0	1	136
+Neither Von Karajan's recording of Beethoven's 6th on Columbia nor Klemperer's on Angel has the right tempo.		1	1	108
+Gould's performance of Bach on the piano doesn't please me anywhere as much as Ross's on the harpsichord.		0	0	105
+Tom's dog with one eye attacked Frank's with three legs.		0	1	56
+Because Steve's of a spider's eye had been stolen, I borrowed Fred's diagram of a snake's fang.		0	1	95
+Neither von Karajan's recording of Beethoven's 6th on Columbia nor Klemperer's has the right tempo.		1	1	99
+Tom's dog with one eye attacked Fred's.		1	1	39
+I borrowed Fred's diagram of a snake's eye because Steve's had been stolen.		1	1	75
+Jerry attempted to blow up the Pentagon.		1	1	40
+So fast did he run that nobody could catch him.		1	1	47
+Bill bought a red house, and Max bought one too.		1	1	48
+Who always drinks milk?		1	1	23
+The book which inspired them was very long.		1	1	43
+The book what inspired them was very long.		0	1	42
+I know the person whose mother died.		1	1	36
+The person whose mother's dog we were all fond of.		1	1	50
+I wonder whose mother died.		1	1	27
+I wonder whose mother's dog died.		1	1	33
+I wonder to whom they dedicated the building.		1	0	45
+Give me the phone number of the person whose mother's dog died.		1	0	63
+This is the senator to whose mother's friend's sister's I sent the letter.		1	1	74
+I want goes to the store.		0	1	25
+I wonder what to be a clown on the cover of.		0	1	44
+Bother you that Kim left!		0	1	25
+A student who to talk to us just walked in.		0	1	43
+Whose bagels do you like?		1	1	25
+I wonder in whom to place my trust.		1	1	35
+There were several old rocks songs that she and I were the only two who knew.		1	0	77
+It was to to amuse us that Kim was singing that they wanted.		0	1	60
+What they feared most was to be no one available to help them.		0	1	62
+We tried to amuse them that Kim was singing.		0	1	44
+Mary asked me if, in St. Louis, John could rent a house cheap.		1	1	62
+Mary arranged for, in St. Louis, John to rent a house cheap.		0	0	60
+It would be unwise for there to be no fire exit.		1	1	48
+I believe there to be no way out.		1	0	33
+I wonder in whom them to place their trust.		0	1	43
+I wonder whom us to trust.		0	1	26
+I wonder who for us to trust.		0	1	29
+I wonder who to place my trust in.		1	0	34
+I know the people that voted in the election.		1	1	45
+I threw away a book that Sandy thought we had read.		1	1	51
+I thought that you were sick.		1	0	29
+I dislike the people in who we placed our trust.		0	1	48
+I dislike the company in which we placed our trust.		1	1	51
+I dislike the people in whose house we stayed.		1	1	46
+I dislike the person with whom we were talking.		1	1	47
+Jones, that we were talking to last night, always watches football games alone.		0	1	79
+A letter was received that Jones would be upset by.		1	1	51
+A letter was received Jones would be upset by.		0	1	46
+I saw someone yesterday I hadn't seen for years.		1	1	48
+Something happened I couldn't really talk about.		1	1	48
+The only person that I like whose kids Dana is willing to put up with is Pat.		1	0	77
+The book that I like which everyone else in the class hates was written by John.		1	0	80
+The only person whose kids Dana is willing to put up with was written by John.		0	0	78
+The book that I like - everyone else in the class hates.		0	1	56
+The only person whose kids Dana is willing to put up with is Pat.		1	1	65
+Which book's, author did you meet?		1	1	34
+Which boy's, mother, did you meet who you liked?		0	0	48
+Which book's, author did you meet who you liked?		0	1	48
+Which boy's, mother, did you meet?		1	1	34
+All who lost money in the scam are eligible for the program.		1	1	60
+Who for Sandy to talk to is still enrolled in the class?		0	1	56
+Who who you like does Sandy also like?		1	1	38
+Everything you like is on the table.		1	1	36
+The bills passed by the House yesterday that we objected to were vetoed.		1	0	72
+The only people being added to our group who were at Harvard were students.		1	1	75
+We like ourselves.		1	0	18
+Nobody likes us.		1	0	16
+Leslie likes ourselves.		0	1	23
+Ourselves like ourselves.		0	1	25
+She voted for herself.		1	0	22
+We gave us presents.		0	1	20
+We gave ourselves presents.		1	1	27
+We gave presents to ourselves.		1	1	30
+We gave us to the cause.		0	1	24
+We gave ourselves to the cause.		1	1	31
+Leslie told us about us.		0	1	24
+Leslie told ourselves about us.		0	1	31
+We think that Leslie likes us.		1	1	30
+We think that Leslie likes ourselves.		0	1	37
+Our friends like us.		1	0	20
+Those pictures of us offended us.		1	1	33
+We found your letter to ourselves in the trash.		0	1	47
+Vote for you!		0	1	13
+Vote for yourself!		1	1	18
+We appeared to them to vote for themselves.		0	1	43
+We admired the pictures of us in the album.		1	1	43
+We admired the pictures of ourselves in the album.		1	1	50
+Leslie used a pen.		1	0	18
+We put the pigs in a pen.		1	1	25
+We need to pen the pigs to keep them from getting into the corn.		1	1	64
+They should pen the letter quickly.		1	1	35
+The car won't run.		1	1	18
+This dye will run.		1	1	18
+She can run an accelerator.		1	0	27
+These stockings will run.		1	1	25
+We need another run to win.		1	1	27
+Lee saw the student with a telescope.		1	1	37
+I forgot how good beer tastes.		1	1	30
+Visiting relatives can be boring.		1	1	33
+If only Superman would stop flying planes!		1	1	42
+That's a new car dealership.		1	0	28
+I know you like the back of my hand.		1	1	36
+Max is on the phone now.		1	1	24
+I saw her duck.		1	1	15
+I'm creating a committee. Kim – you're in charge.		1	1	49
+Lights go out at ten. There will be no talking afterwards.		1	1	58
+They found the book on the atom.		1	1	32
+Which experts testified against defendants who exposed them?		1	1	60
+List all experts for the defendant who represented himself.		1	1	59
+List associates of each defendant who speaks Spanish.		1	1	53
+They lost themselves' books.		0	1	28
+Some sentences go on and on and on.		1	1	35
+Sentences some go on and on and on and on.		0	1	42
+That surprised me.		1	1	18
+I noticed the.		0	1	14
+They were interested in his.		1	1	28
+This is my favorite.		1	1	20
+A large dog chased a small cat.		1	1	31
+Some people yell at noisy dogs in my neighborhood.		1	1	50
+Some people yell at the dogs in my neighborhood.		1	1	48
+Some people yell at the dogs.		1	1	29
+Some people yell at noisy dogs.		1	1	31
+Some people yell at dogs.		1	1	25
+Some people consider the noisy dogs dangerous.		1	1	46
+Some people consider the dogs in my neighborhood dangerous.		1	1	59
+Some people consider noisy dogs in my neighborhood dangerous.		1	0	61
+Some people consider the dogs dangerous.		1	1	40
+Some people consider noisy dogs dangerous.		1	1	42
+Some people consider dogs in my neighborhood dangerous.		1	1	55
+Some people consider dogs dangerous.		1	1	36
+People with children who use drugs should be locked up.		1	1	55
+This disease gave Leslie a fever in Rome.		1	1	41
+The love of my life and mother of my children would never do such a thing.		1	1	74
+Most elections are quickly forgotten, but the election of 2000, everyone will remember for a long time.		1	1	103
+It is painting by Klee or drawing by Miro that the museum displays no.		0	0	70
+The defendant denied the accusation.		1	1	36
+The teacher disappeared the problem.		0	1	36
+The teacher handed the student.		0	1	31
+The bird sings.		1	1	15
+The bird sing.		0	1	14
+Birds sings.		0	1	12
+The birds give the worm a tug.		1	1	30
+The bird give the worm a tug.		0	1	29
+The birds gives the worm a tug.		0	1	31
+Terry delighted in my pain.		1	0	27
+Terry delighted.		0	0	16
+Terry delighted my pain.		0	1	24
+Kerry remarked it was late.		1	1	27
+What additional categories and rules would be required to handle these verbs?		1	1	77
+We created a monster.		1	0	21
+I was already aware of fact.		0	1	28
+The defendant deny the allegation.		0	1	34
+The defendants denies the allegation.		0	1	37
+The defendant walks.		1	1	20
+The defendant walk.		0	1	19
+The defendants walks.		0	1	21
+How many feature structures categories can label the first daughter?		1	1	68
+The child put the toy on the table.		1	1	35
+The teacher became angry with the students.		1	0	43
+The teacher became.		0	1	19
+The jury believed the defendant lied.		1	1	37
+The guests dined.		1	1	17
+We relied on Leslie.		1	0	20
+We relied above Leslie.		0	1	23
+We celebrated in the streets.		1	0	29
+We celebrated in the streets in the rain on Tuesday in the morning.		1	1	67
+The children are happy of ice cream.		0	1	36
+The children are fond with the ice cream.		0	1	41
+The children are fond that they have ice cream.		0	1	47
+A magazine appeared on the newsstands.		1	1	38
+A magazine about crime appeared on the newsstands.		1	1	50
+Newsweek appeared on the newsstands.		1	1	36
+Newsweek about crime appeared on the newsstands.		0	1	48
+The report that crime was declining surprised many people.		1	1	58
+The book surprised many people.		1	1	31
+The book that crime was declining surprised many people.		0	1	56
+The storm arrived after the picnic.		1	1	35
+The storm arrived while the picnic.		0	1	35
+The storm arrived while we ate lunch.		1	1	37
+This dogs barked.		0	1	17
+These dogs barked.		1	0	18
+A chair was broken.		1	1	19
+They want them arrested.		1	1	24
+They preferred them arrested.		1	1	29
+We preferred them on our team.		1	1	30
+With my parents as supportive as they are, I'll be in fine shape.		1	1	65
+We walks.		0	1	9
+Few dog barked.		0	1	15
+The dogs barked.		1	1	16
+I walk and Dana runs.		1	1	21
+They like us.		1	1	13
+Us like them.		0	0	13
+Kim likes dogs.		1	0	15
+Dogs like Kim.		1	1	14
+The person responsible confessed.		1	1	33
+The person confessed responsible.		0	1	33
+The cat slept soundly and furry.		0	1	32
+The soundly and furry cat slept.		0	1	32
+Chris walks, Pat eats broccoli, and Sandy plays squash.		1	0	55
+There was some particular dog who saved every family.		1	1	53
+Susan frightens her.		1	1	20
+Susan told her a story.		1	1	23
+Susan told a story to her.		1	1	26
+Susan devoted herself to linguistics.		1	1	37
+Nobody told Susan about herself.		1	1	32
+That picture of Susan offended her.		1	1	35
+He offended Sandy.		1	1	18
+I enjoy yourself.		0	0	17
+They talk to themselves.		1	1	24
+Nobody told Susan.		1	1	18
+Protect yourself!		1	1	17
+Protect you!		0	1	12
+I met the person who left.		1	1	26
+Leslie slept.		1	1	13
+Chris handed Bo.		0	0	16
+Dana walked and Leslie ran.		1	1	27
+Dana walking and Leslie ran.		0	1	28
+Dana walking and Leslie running.		0	1	32
+the putter of books left.		0	0	25
+Kris donated a book to the library.		1	1	35
+The police sprayed the protesters with water.		1	1	45
+The police sprayed water on the protesters.		1	1	43
+The students drove cars.		1	1	24
+These cars drive easily.		1	0	24
+The horse kicked me black and blue.		1	1	35
+They yelled.		1	1	12
+The horse raced past the barn fell.		1	1	35
+The horse that was raced past the barn fell.		1	1	44
+The boat seen down the river sank.		1	1	34
+The evidence assembled by the prosecution convinced the jury.		1	1	61
+Lou forgot the umbrella.		1	1	24
+Lou forgot the umbrella in the closet.		1	1	38
+Lou hoped the umbrella was broken.		1	1	34
+Lou hoped the umbrella in the closet.		0	1	37
+Lou put the umbrella was broken.		0	1	32
+Lou put the umbrella in the closet.		1	1	35
+The artist drew the child with a pencil.		1	0	40
+The dog bit the cat.		1	1	20
+The cat was bitten the mouse.		0	1	29
+The cat was bitten the mouse by the dog.		0	1	40
+Chris was handed Sandy a note by Pat.		0	1	37
+Chris was handed a note.		1	1	24
+Chris was handed Sandy a note.		0	1	30
+TV puts dumb ideas in children's heads.		1	1	39
+Dumb ideas are put in children's heads by TV.		1	1	45
+Dumb ideas are put in children's heads.		1	1	39
+Dumb ideas are put notions in children's heads by TV.		0	1	53
+The patient died.		1	1	17
+The patient was died.		0	1	21
+Chris was handed.		0	1	17
+TV puts dumb ideas.		0	1	19
+He was arrested by the police.		1	1	30
+The cat got bitten.		1	1	19
+The cat were bitten by the dog.		0	1	31
+There is a monster in Loch Ness.		1	1	32
+It is obvious that Pat is lying.		1	1	32
+Pat is the captain of the team.		1	1	31
+Pat is hate Chris.		0	1	18
+There is a unicorn in the garden.		1	1	33
+There was a felon elected to the city council.		1	1	46
+There is a seat available.		1	0	26
+A seat available was in the last row.		0	1	37
+Many people were fond of Pat.		1	0	29
+People are looking through the window.		1	1	38
+A felon was elected to the city council.		1	1	40
+There loved Sandy.		0	1	18
+We talked to them about there.		0	1	30
+It mattered that the Giants had lost.		1	1	37
+That dogs bark annoys people.		1	1	29
+It annoys people that dogs bark.		1	1	32
+That Chris knew the answer occurred to Pat.		1	1	43
+It never occurred to Pat that Chris knew the answer.		1	1	52
+That the Cardinal won the game gave Sandy a thrill.		1	1	51
+It gave Sandy a thrill that the Cardinal won the game.		1	1	54
+That Sandy had lied suggested.		0	1	30
+It loved Sandy.		0	0	15
+Cohen proved the independence of the continuum hypothesis.		1	1	58
+Cohen proved that the continuum hypothesis was independent.		1	1	59
+We forgot our invitations.		1	1	26
+Nobody saw Pat.		1	1	15
+That Fido barks annoys me.		1	1	26
+Fido barks.		1	1	11
+Chris dreads the bucket.		1	1	24
+The candidates bring advantage to the voters.		1	1	45
+Tabs are kept on suspected drug dealers by the FBI.		1	1	51
+Advantage is taken of every opportunity for improvement.		1	1	56
+The bucket was kicked by Pat.		1	1	29
+John is sad.		1	1	12
+John loaded the wagon full with hay.		1	1	36
+John loaded the wagon with hay green.		0	1	37
+I presented John with it dead.		0	1	30
+Of whom are you thinking?		1	1	25
+John became rich.		1	1	17
+I gave John gold apples.		1	0	24
+how silly is Bill considered?		1	1	29
+how mad was Bill made?		1	1	22
+John is sick.		1	1	13
+John left singing.		1	1	18
+John is near Larry.		1	1	19
+John gave Bill the dog dead.		1	1	28
+Bill was struck by John as stupid.		0	0	34
+John was struck as sick.		0	1	24
+John was struck by Bill's idiocy.		1	0	33
+John promised Bill to leave.		1	1	28
+John tried to leave.		1	0	20
+To leave would be a pleasure.		1	1	29
+John was struck by Bill as pompous.		0	0	35
+John was promised by Bill to leave.		0	0	35
+They make good cooks.		1	1	21
+there is nothing to do.		1	0	23
+John has something for Bill to do.		1	1	34
+I am counting on Bill to incriminate himself.		1	0	45
+On whom are you counting to incriminate himself?		1	0	48
+I am counting on Bill to get there on time.		1	1	43
+I would prefer to leave.		1	1	24
+I would hate for John to leave.		1	1	31
+I would prefer for John to leave.		1	1	33
+It was hated for John to leave.		0	1	31
+John decided for Bill to get the prize.		0	1	39
+John decided Bill to get the prize.		0	1	35
+to die is no fun.		1	1	17
+John wants to leave.		1	1	20
+John counted on Bill to get there on time.		1	0	42
+I bought Bill a book to read.		1	1	29
+John told Mary that it would be important to leave early.		1	1	57
+John told Mary that it was important to Fred to leave early.		1	1	60
+John, told Mary that it would be appropriate to leave together.		1	1	63
+the election of John president surprised me.		0	1	44
+John's arriving dead surprised me.		1	1	34
+the attempt by John to leave surprised me.		1	1	42
+John left orders to follow Pete.		1	1	32
+John left us orders to follow Pete.		1	1	35
+John left orders not to be disturbed.		1	1	37
+That he is here is clear.		1	0	25
+It is a problem that he is here.		1	1	32
+It bothers me that he is here.		1	1	30
+John regretted it that Bill had a good time.		1	0	44
+John believes it that Bill is here.		0	0	35
+John believes it sincerely that Bill is here.		0	1	45
+John is aware of it that Bill is here.		0	1	38
+John felt it that Bill was tardy.		0	1	33
+John believed it that Bill was tardy.		0	0	37
+It was believed that Bill was tardy.		1	1	36
+That John is reluctant seems.		0	1	29
+It is the problem that he is here.		0	0	34
+That he is here is the problem.		1	1	31
+The problem we are discussing is George.		1	1	40
+It is to give up to leave.		0	1	26
+It would prove our theory to be untenable for carrots to be vegetables.		1	1	71
+It was believed to be illegal by them to do that.		0	1	49
+John grudgingly accepted judgments of his incompetence as an auto mechanic.		1	1	75
+It was to John that I gave the book.		1	1	36
+I bought it to read.		1	1	20
+I bought it to give to Pete.		1	1	28
+I gave it to Pete to take to the fair.		1	1	38
+I gave Pete the book to impress.		0	1	32
+I wrote to Bill.		1	1	16
+I presented it to Bill to read.		1	1	31
+I presented Bill with it to read.		0	0	33
+I gave a book to Bill to read.		1	1	30
+John thinks it would upset himself to die.		1	1	42
+John made Bill mad at himself.		1	0	30
+John made Bill master of himself.		1	1	33
+The correspondence school made Bill a good typist.		1	1	50
+The correspondence school sent Bill a good typist.		1	1	50
+John considers Bill silly.		1	1	26
+John considers Bill to be silly.		1	1	32
+John bought a dog for himself to play with.		0	0	43
+John arranged for himself to get the prize.		1	0	43
+John talked to Bill about himself.		1	1	34

eval_cola_results.json

@@ -0,0 +1,8 @@
+{
+    "eval_cola_loss": 0.9552000761032104,
+    "eval_cola_matthews_correlation": 0.10724809104897279,
+    "eval_cola_runtime": 7.8141,
+    "eval_cola_samples_per_second": 133.477,
+    "eval_cola_steps_per_second": 16.765,
+    "eval_samples": 1043
+}

predict_results_cola.txt

@@ -0,0 +1,1064 @@
+index	prediction
+0	acceptable
+1	acceptable
+2	acceptable
+3	acceptable
+4	acceptable
+5	acceptable
+6	acceptable
+7	acceptable
+8	acceptable
+9	acceptable
+10	unacceptable
+11	acceptable
+12	unacceptable
+13	unacceptable
+14	acceptable
+15	acceptable
+16	acceptable
+17	acceptable
+18	acceptable
+19	acceptable
+20	acceptable
+21	acceptable
+22	acceptable
+23	acceptable
+24	acceptable
+25	unacceptable
+26	acceptable
+27	unacceptable
+28	acceptable
+29	acceptable
+30	acceptable
+31	acceptable
+32	acceptable
+33	unacceptable
+34	acceptable
+35	unacceptable
+36	acceptable
+37	acceptable
+38	unacceptable
+39	acceptable
+40	acceptable
+41	unacceptable
+42	acceptable
+43	acceptable
+44	acceptable
+45	acceptable
+46	acceptable
+47	acceptable
+48	acceptable
+49	acceptable
+50	acceptable
+51	unacceptable
+52	acceptable
+53	unacceptable
+54	acceptable
+55	acceptable
+56	acceptable
+57	acceptable
+58	unacceptable
+59	unacceptable
+60	acceptable
+61	acceptable
+62	acceptable
+63	unacceptable
+64	acceptable
+65	acceptable
+66	acceptable
+67	unacceptable
+68	acceptable
+69	acceptable
+70	acceptable
+71	acceptable
+72	acceptable
+73	acceptable
+74	acceptable
+75	acceptable
+76	acceptable
+77	acceptable
+78	acceptable
+79	unacceptable
+80	acceptable
+81	acceptable
+82	unacceptable
+83	acceptable
+84	acceptable
+85	acceptable
+86	acceptable
+87	acceptable
+88	acceptable
+89	acceptable
+90	acceptable
+91	acceptable
+92	acceptable
+93	acceptable
+94	acceptable
+95	acceptable
+96	acceptable
+97	acceptable
+98	unacceptable
+99	acceptable
+100	unacceptable
+101	acceptable
+102	acceptable
+103	acceptable
+104	acceptable
+105	unacceptable
+106	acceptable
+107	acceptable
+108	acceptable
+109	unacceptable
+110	unacceptable
+111	acceptable
+112	acceptable
+113	acceptable
+114	acceptable
+115	unacceptable
+116	acceptable
+117	acceptable
+118	acceptable
+119	acceptable
+120	acceptable
+121	unacceptable
+122	acceptable
+123	unacceptable
+124	acceptable
+125	acceptable
+126	unacceptable
+127	acceptable
+128	acceptable
+129	acceptable
+130	unacceptable
+131	acceptable
+132	acceptable
+133	acceptable
+134	acceptable
+135	unacceptable
+136	acceptable
+137	acceptable
+138	acceptable
+139	acceptable
+140	unacceptable
+141	acceptable
+142	acceptable
+143	acceptable
+144	acceptable
+145	unacceptable
+146	unacceptable
+147	acceptable
+148	acceptable
+149	acceptable
+150	unacceptable
+151	acceptable
+152	unacceptable
+153	acceptable
+154	acceptable
+155	acceptable
+156	unacceptable
+157	unacceptable
+158	acceptable
+159	acceptable
+160	acceptable
+161	acceptable
+162	acceptable
+163	acceptable
+164	unacceptable
+165	unacceptable
+166	unacceptable
+167	unacceptable
+168	acceptable
+169	acceptable
+170	acceptable
+171	acceptable
+172	acceptable
+173	unacceptable
+174	acceptable
+175	unacceptable
+176	acceptable
+177	acceptable
+178	acceptable
+179	acceptable
+180	acceptable
+181	acceptable
+182	unacceptable
+183	acceptable
+184	unacceptable
+185	acceptable
+186	acceptable
+187	acceptable
+188	unacceptable
+189	acceptable
+190	acceptable
+191	unacceptable
+192	unacceptable
+193	acceptable
+194	acceptable
+195	acceptable
+196	acceptable
+197	unacceptable
+198	acceptable
+199	acceptable
+200	acceptable
+201	acceptable
+202	acceptable
+203	acceptable
+204	acceptable
+205	acceptable
+206	acceptable
+207	acceptable
+208	unacceptable
+209	acceptable
+210	acceptable
+211	acceptable
+212	acceptable
+213	acceptable
+214	acceptable
+215	acceptable
+216	unacceptable
+217	acceptable
+218	unacceptable
+219	acceptable
+220	acceptable
+221	unacceptable
+222	acceptable
+223	acceptable
+224	acceptable
+225	acceptable
+226	acceptable
+227	acceptable
+228	acceptable
+229	acceptable
+230	acceptable
+231	acceptable
+232	acceptable
+233	acceptable
+234	acceptable
+235	acceptable
+236	acceptable
+237	acceptable
+238	acceptable
+239	acceptable
+240	acceptable
+241	acceptable
+242	acceptable
+243	acceptable
+244	unacceptable
+245	acceptable
+246	acceptable
+247	acceptable
+248	acceptable
+249	acceptable
+250	acceptable
+251	acceptable
+252	unacceptable
+253	acceptable
+254	acceptable
+255	acceptable
+256	unacceptable
+257	unacceptable
+258	unacceptable
+259	acceptable
+260	unacceptable
+261	unacceptable
+262	acceptable
+263	acceptable
+264	acceptable
+265	acceptable
+266	acceptable
+267	acceptable
+268	acceptable
+269	acceptable
+270	unacceptable
+271	acceptable
+272	acceptable
+273	acceptable
+274	unacceptable
+275	acceptable
+276	unacceptable
+277	acceptable
+278	acceptable
+279	acceptable
+280	acceptable
+281	unacceptable
+282	acceptable
+283	acceptable
+284	unacceptable
+285	acceptable
+286	acceptable
+287	unacceptable
+288	unacceptable
+289	acceptable
+290	acceptable
+291	acceptable
+292	acceptable
+293	unacceptable
+294	acceptable
+295	acceptable
+296	acceptable
+297	acceptable
+298	acceptable
+299	acceptable
+300	acceptable
+301	unacceptable
+302	unacceptable
+303	acceptable
+304	acceptable
+305	acceptable
+306	acceptable
+307	acceptable
+308	acceptable
+309	acceptable
+310	acceptable
+311	acceptable
+312	acceptable
+313	acceptable
+314	unacceptable
+315	acceptable
+316	acceptable
+317	unacceptable
+318	acceptable
+319	acceptable
+320	acceptable
+321	acceptable
+322	acceptable
+323	acceptable
+324	acceptable
+325	acceptable
+326	acceptable
+327	acceptable
+328	acceptable
+329	acceptable
+330	acceptable
+331	acceptable
+332	acceptable
+333	acceptable
+334	acceptable
+335	acceptable
+336	acceptable
+337	acceptable
+338	acceptable
+339	acceptable
+340	acceptable
+341	acceptable
+342	acceptable
+343	acceptable
+344	acceptable
+345	acceptable
+346	acceptable
+347	acceptable
+348	acceptable
+349	acceptable
+350	acceptable
+351	acceptable
+352	acceptable
+353	acceptable
+354	unacceptable
+355	acceptable
+356	acceptable
+357	acceptable
+358	acceptable
+359	acceptable
+360	acceptable
+361	acceptable
+362	acceptable
+363	acceptable
+364	acceptable
+365	acceptable
+366	acceptable
+367	acceptable
+368	acceptable
+369	acceptable
+370	acceptable
+371	acceptable
+372	acceptable
+373	acceptable
+374	acceptable
+375	acceptable
+376	acceptable
+377	acceptable
+378	acceptable
+379	acceptable
+380	acceptable
+381	acceptable
+382	acceptable
+383	acceptable
+384	acceptable
+385	unacceptable
+386	acceptable
+387	acceptable
+388	acceptable
+389	acceptable
+390	acceptable
+391	unacceptable
+392	acceptable
+393	acceptable
+394	acceptable
+395	unacceptable
+396	acceptable
+397	acceptable
+398	acceptable
+399	unacceptable
+400	acceptable
+401	unacceptable
+402	acceptable
+403	acceptable
+404	acceptable
+405	acceptable
+406	acceptable
+407	acceptable
+408	acceptable
+409	acceptable
+410	acceptable
+411	acceptable
+412	unacceptable
+413	acceptable
+414	acceptable
+415	unacceptable
+416	acceptable
+417	acceptable
+418	acceptable
+419	acceptable
+420	acceptable
+421	acceptable
+422	acceptable
+423	acceptable
+424	unacceptable
+425	acceptable
+426	acceptable
+427	acceptable
+428	acceptable
+429	acceptable
+430	unacceptable
+431	acceptable
+432	unacceptable
+433	acceptable
+434	acceptable
+435	acceptable
+436	acceptable
+437	acceptable
+438	acceptable
+439	acceptable
+440	unacceptable
+441	acceptable
+442	acceptable
+443	acceptable
+444	acceptable
+445	unacceptable
+446	acceptable
+447	acceptable
+448	acceptable
+449	acceptable
+450	unacceptable
+451	acceptable
+452	acceptable
+453	acceptable
+454	acceptable
+455	acceptable
+456	unacceptable
+457	acceptable
+458	acceptable
+459	acceptable
+460	acceptable
+461	acceptable
+462	acceptable
+463	acceptable
+464	acceptable
+465	acceptable
+466	acceptable
+467	acceptable
+468	acceptable
+469	unacceptable
+470	acceptable
+471	acceptable
+472	unacceptable
+473	unacceptable
+474	unacceptable
+475	acceptable
+476	acceptable
+477	acceptable
+478	acceptable
+479	acceptable
+480	acceptable
+481	acceptable
+482	acceptable
+483	unacceptable
+484	acceptable
+485	acceptable
+486	acceptable
+487	acceptable
+488	acceptable
+489	acceptable
+490	acceptable
+491	acceptable
+492	acceptable
+493	acceptable
+494	acceptable
+495	unacceptable
+496	unacceptable
+497	acceptable
+498	acceptable
+499	acceptable
+500	acceptable
+501	acceptable
+502	acceptable
+503	acceptable
+504	unacceptable
+505	unacceptable
+506	acceptable
+507	acceptable
+508	acceptable
+509	acceptable
+510	acceptable
+511	acceptable
+512	acceptable
+513	acceptable
+514	acceptable
+515	acceptable
+516	acceptable
+517	acceptable
+518	acceptable
+519	acceptable
+520	unacceptable
+521	acceptable
+522	acceptable
+523	acceptable
+524	acceptable
+525	acceptable
+526	acceptable
+527	acceptable
+528	acceptable
+529	acceptable
+530	acceptable
+531	acceptable
+532	acceptable
+533	acceptable
+534	acceptable
+535	acceptable
+536	acceptable
+537	acceptable
+538	acceptable
+539	acceptable
+540	acceptable
+541	acceptable
+542	acceptable
+543	acceptable
+544	acceptable
+545	acceptable
+546	acceptable
+547	acceptable
+548	unacceptable
+549	acceptable
+550	acceptable
+551	acceptable
+552	acceptable
+553	acceptable
+554	acceptable
+555	acceptable
+556	acceptable
+557	acceptable
+558	unacceptable
+559	acceptable
+560	unacceptable
+561	acceptable
+562	acceptable
+563	acceptable
+564	acceptable
+565	acceptable
+566	acceptable
+567	acceptable
+568	acceptable
+569	acceptable
+570	acceptable
+571	acceptable
+572	acceptable
+573	acceptable
+574	unacceptable
+575	unacceptable
+576	acceptable
+577	acceptable
+578	acceptable
+579	acceptable
+580	acceptable
+581	acceptable
+582	acceptable
+583	acceptable
+584	acceptable
+585	acceptable
+586	acceptable
+587	acceptable
+588	acceptable
+589	acceptable
+590	acceptable
+591	acceptable
+592	acceptable
+593	unacceptable
+594	acceptable
+595	unacceptable
+596	acceptable
+597	acceptable
+598	unacceptable
+599	acceptable
+600	unacceptable
+601	acceptable
+602	acceptable
+603	unacceptable
+604	acceptable
+605	unacceptable
+606	acceptable
+607	acceptable
+608	acceptable
+609	acceptable
+610	acceptable
+611	acceptable
+612	acceptable
+613	acceptable
+614	acceptable
+615	acceptable
+616	acceptable
+617	unacceptable
+618	acceptable
+619	acceptable
+620	acceptable
+621	acceptable
+622	acceptable
+623	acceptable
+624	acceptable
+625	unacceptable
+626	acceptable
+627	acceptable
+628	acceptable
+629	acceptable
+630	acceptable
+631	acceptable
+632	acceptable
+633	acceptable
+634	acceptable
+635	unacceptable
+636	acceptable
+637	acceptable
+638	acceptable
+639	acceptable
+640	acceptable
+641	acceptable
+642	acceptable
+643	acceptable
+644	acceptable
+645	acceptable
+646	acceptable
+647	acceptable
+648	unacceptable
+649	unacceptable
+650	acceptable
+651	acceptable
+652	acceptable
+653	acceptable
+654	acceptable
+655	acceptable
+656	acceptable
+657	acceptable
+658	acceptable
+659	acceptable
+660	acceptable
+661	acceptable
+662	acceptable
+663	acceptable
+664	acceptable
+665	acceptable
+666	acceptable
+667	acceptable
+668	acceptable
+669	acceptable
+670	acceptable
+671	acceptable
+672	acceptable
+673	unacceptable
+674	unacceptable
+675	unacceptable
+676	acceptable
+677	acceptable
+678	acceptable
+679	acceptable
+680	unacceptable
+681	acceptable
+682	acceptable
+683	acceptable
+684	acceptable
+685	acceptable
+686	acceptable
+687	acceptable
+688	acceptable
+689	acceptable
+690	unacceptable
+691	unacceptable
+692	acceptable
+693	unacceptable
+694	acceptable
+695	acceptable
+696	acceptable
+697	acceptable
+698	acceptable
+699	acceptable
+700	acceptable
+701	acceptable
+702	acceptable
+703	acceptable
+704	unacceptable
+705	unacceptable
+706	acceptable
+707	acceptable
+708	acceptable
+709	acceptable
+710	unacceptable
+711	acceptable
+712	acceptable
+713	unacceptable
+714	acceptable
+715	acceptable
+716	acceptable
+717	acceptable
+718	acceptable
+719	acceptable
+720	acceptable
+721	acceptable
+722	unacceptable
+723	unacceptable
+724	acceptable
+725	acceptable
+726	acceptable
+727	acceptable
+728	acceptable
+729	acceptable
+730	acceptable
+731	acceptable
+732	acceptable
+733	acceptable
+734	unacceptable
+735	acceptable
+736	acceptable
+737	acceptable
+738	acceptable
+739	unacceptable
+740	unacceptable
+741	unacceptable
+742	acceptable
+743	unacceptable
+744	unacceptable
+745	acceptable
+746	acceptable
+747	acceptable
+748	unacceptable
+749	acceptable
+750	acceptable
+751	acceptable
+752	acceptable
+753	acceptable
+754	acceptable
+755	acceptable
+756	acceptable
+757	unacceptable
+758	acceptable
+759	acceptable
+760	acceptable
+761	acceptable
+762	acceptable
+763	acceptable
+764	acceptable
+765	unacceptable
+766	unacceptable
+767	acceptable
+768	unacceptable
+769	acceptable
+770	acceptable
+771	acceptable
+772	acceptable
+773	acceptable
+774	acceptable
+775	acceptable
+776	acceptable
+777	acceptable
+778	acceptable
+779	acceptable
+780	acceptable
+781	acceptable
+782	acceptable
+783	unacceptable
+784	acceptable
+785	acceptable
+786	acceptable
+787	acceptable
+788	acceptable
+789	acceptable
+790	acceptable
+791	acceptable
+792	unacceptable
+793	acceptable
+794	acceptable
+795	unacceptable
+796	acceptable
+797	acceptable
+798	acceptable
+799	acceptable
+800	acceptable
+801	acceptable
+802	acceptable
+803	unacceptable
+804	acceptable
+805	acceptable
+806	unacceptable
+807	acceptable
+808	acceptable
+809	acceptable
+810	acceptable
+811	acceptable
+812	acceptable
+813	acceptable
+814	acceptable
+815	acceptable
+816	acceptable
+817	acceptable
+818	acceptable
+819	acceptable
+820	acceptable
+821	unacceptable
+822	acceptable
+823	acceptable
+824	acceptable
+825	unacceptable
+826	acceptable
+827	acceptable
+828	unacceptable
+829	acceptable
+830	acceptable
+831	acceptable
+832	acceptable
+833	acceptable
+834	acceptable
+835	acceptable
+836	unacceptable
+837	acceptable
+838	acceptable
+839	acceptable
+840	acceptable
+841	acceptable
+842	unacceptable
+843	acceptable
+844	acceptable
+845	acceptable
+846	unacceptable
+847	unacceptable
+848	acceptable
+849	unacceptable
+850	unacceptable
+851	acceptable
+852	acceptable
+853	acceptable
+854	unacceptable
+855	acceptable
+856	acceptable
+857	acceptable
+858	unacceptable
+859	acceptable
+860	acceptable
+861	acceptable
+862	acceptable
+863	acceptable
+864	acceptable
+865	acceptable
+866	acceptable
+867	acceptable
+868	acceptable
+869	acceptable
+870	unacceptable
+871	unacceptable
+872	acceptable
+873	unacceptable
+874	acceptable
+875	acceptable
+876	acceptable
+877	unacceptable
+878	acceptable
+879	acceptable
+880	acceptable
+881	acceptable
+882	acceptable
+883	unacceptable
+884	acceptable
+885	acceptable
+886	acceptable
+887	acceptable
+888	unacceptable
+889	acceptable
+890	acceptable
+891	acceptable
+892	unacceptable
+893	acceptable
+894	acceptable
+895	acceptable
+896	unacceptable
+897	acceptable
+898	unacceptable
+899	unacceptable
+900	acceptable
+901	acceptable
+902	acceptable
+903	acceptable
+904	acceptable
+905	acceptable
+906	acceptable
+907	unacceptable
+908	acceptable
+909	acceptable
+910	acceptable
+911	acceptable
+912	acceptable
+913	acceptable
+914	acceptable
+915	acceptable
+916	unacceptable
+917	acceptable
+918	acceptable
+919	acceptable
+920	acceptable
+921	acceptable
+922	acceptable
+923	unacceptable
+924	unacceptable
+925	acceptable
+926	acceptable
+927	acceptable
+928	acceptable
+929	acceptable
+930	acceptable
+931	acceptable
+932	acceptable
+933	acceptable
+934	acceptable
+935	acceptable
+936	acceptable
+937	acceptable
+938	acceptable
+939	acceptable
+940	acceptable
+941	acceptable
+942	acceptable
+943	acceptable
+944	acceptable
+945	acceptable
+946	unacceptable
+947	acceptable
+948	acceptable
+949	unacceptable
+950	acceptable
+951	acceptable
+952	unacceptable
+953	unacceptable
+954	acceptable
+955	acceptable
+956	acceptable
+957	acceptable
+958	acceptable
+959	unacceptable
+960	acceptable
+961	acceptable
+962	acceptable
+963	acceptable
+964	acceptable
+965	unacceptable
+966	acceptable
+967	acceptable
+968	acceptable
+969	acceptable
+970	acceptable
+971	acceptable
+972	acceptable
+973	acceptable
+974	acceptable
+975	acceptable
+976	acceptable
+977	unacceptable
+978	acceptable
+979	acceptable
+980	acceptable
+981	acceptable
+982	acceptable
+983	acceptable
+984	acceptable
+985	acceptable
+986	acceptable
+987	acceptable
+988	acceptable
+989	acceptable
+990	acceptable
+991	unacceptable
+992	acceptable
+993	unacceptable
+994	acceptable
+995	acceptable
+996	unacceptable
+997	acceptable
+998	unacceptable
+999	unacceptable
+1000	unacceptable
+1001	acceptable
+1002	unacceptable
+1003	acceptable
+1004	acceptable
+1005	acceptable
+1006	acceptable
+1007	acceptable
+1008	unacceptable
+1009	unacceptable
+1010	acceptable
+1011	acceptable
+1012	acceptable
+1013	acceptable
+1014	acceptable
+1015	unacceptable
+1016	acceptable
+1017	unacceptable
+1018	unacceptable
+1019	unacceptable
+1020	acceptable
+1021	unacceptable
+1022	unacceptable
+1023	acceptable
+1024	unacceptable
+1025	acceptable
+1026	acceptable
+1027	acceptable
+1028	acceptable
+1029	acceptable
+1030	unacceptable
+1031	unacceptable
+1032	acceptable
+1033	acceptable
+1034	acceptable
+1035	acceptable
+1036	acceptable
+1037	acceptable
+1038	acceptable
+1039	acceptable
+1040	acceptable
+1041	acceptable
+1042	unacceptable
+1043	unacceptable
+1044	acceptable
+1045	acceptable
+1046	acceptable
+1047	unacceptable
+1048	acceptable
+1049	acceptable
+1050	acceptable
+1051	unacceptable
+1052	unacceptable
+1053	acceptable
+1054	unacceptable
+1055	acceptable
+1056	acceptable
+1057	unacceptable
+1058	acceptable
+1059	unacceptable
+1060	acceptable
+1061	acceptable
+1062	acceptable

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 78.78,
+    "train_loss": 0.40566940747774566,
+    "train_runtime": 7554.577,
+    "train_samples": 8551,
+    "train_samples_per_second": 508.301,
+    "train_steps_per_second": 1.986
+}

trainer_state.json

@@ -0,0 +1,415 @@
+{
+  "best_metric": 0.1436834264468159,
+  "best_model_checkpoint": "cola-pixel-handwritten-mean-vatrpp-256-64-4-5e-5-15000-42/checkpoint-1600",
+  "epoch": 78.77611940298507,
+  "global_step": 2600,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 3.03,
+      "learning_rate": 2.5e-05,
+      "loss": 0.6426,
+      "step": 100
+    },
+    {
+      "epoch": 3.03,
+      "eval_loss": 0.6255418658256531,
+      "eval_matthews_correlation": 0.0,
+      "eval_runtime": 8.2752,
+      "eval_samples_per_second": 126.039,
+      "eval_steps_per_second": 15.83,
+      "step": 100
+    },
+    {
+      "epoch": 6.06,
+      "learning_rate": 5e-05,
+      "loss": 0.6176,
+      "step": 200
+    },
+    {
+      "epoch": 6.06,
+      "eval_loss": 0.630817711353302,
+      "eval_matthews_correlation": 0.0,
+      "eval_runtime": 8.1785,
+      "eval_samples_per_second": 127.53,
+      "eval_steps_per_second": 16.018,
+      "step": 200
+    },
+    {
+      "epoch": 9.09,
+      "learning_rate": 4.9662162162162164e-05,
+      "loss": 0.6183,
+      "step": 300
+    },
+    {
+      "epoch": 9.09,
+      "eval_loss": 0.6186701059341431,
+      "eval_matthews_correlation": 0.0,
+      "eval_runtime": 7.8187,
+      "eval_samples_per_second": 133.398,
+      "eval_steps_per_second": 16.755,
+      "step": 300
+    },
+    {
+      "epoch": 12.12,
+      "learning_rate": 4.9324324324324325e-05,
+      "loss": 0.6162,
+      "step": 400
+    },
+    {
+      "epoch": 12.12,
+      "eval_loss": 0.6158396005630493,
+      "eval_matthews_correlation": 0.0,
+      "eval_runtime": 7.9658,
+      "eval_samples_per_second": 130.935,
+      "eval_steps_per_second": 16.445,
+      "step": 400
+    },
+    {
+      "epoch": 15.15,
+      "learning_rate": 4.8986486486486486e-05,
+      "loss": 0.614,
+      "step": 500
+    },
+    {
+      "epoch": 15.15,
+      "eval_loss": 0.6250273585319519,
+      "eval_matthews_correlation": -0.02929206145132745,
+      "eval_runtime": 7.9393,
+      "eval_samples_per_second": 131.373,
+      "eval_steps_per_second": 16.5,
+      "step": 500
+    },
+    {
+      "epoch": 18.18,
+      "learning_rate": 4.8648648648648654e-05,
+      "loss": 0.6096,
+      "step": 600
+    },
+    {
+      "epoch": 18.18,
+      "eval_loss": 0.618541419506073,
+      "eval_matthews_correlation": 0.0,
+      "eval_runtime": 7.9167,
+      "eval_samples_per_second": 131.746,
+      "eval_steps_per_second": 16.547,
+      "step": 600
+    },
+    {
+      "epoch": 21.21,
+      "learning_rate": 4.8310810810810816e-05,
+      "loss": 0.6055,
+      "step": 700
+    },
+    {
+      "epoch": 21.21,
+      "eval_loss": 0.6223562359809875,
+      "eval_matthews_correlation": 0.017470726784935973,
+      "eval_runtime": 7.909,
+      "eval_samples_per_second": 131.875,
+      "eval_steps_per_second": 16.563,
+      "step": 700
+    },
+    {
+      "epoch": 24.24,
+      "learning_rate": 4.7979729729729736e-05,
+      "loss": 0.6001,
+      "step": 800
+    },
+    {
+      "epoch": 24.24,
+      "eval_loss": 0.6550844311714172,
+      "eval_matthews_correlation": 0.13008617806157513,
+      "eval_runtime": 8.0719,
+      "eval_samples_per_second": 129.214,
+      "eval_steps_per_second": 16.229,
+      "step": 800
+    },
+    {
+      "epoch": 27.27,
+      "learning_rate": 4.764527027027027e-05,
+      "loss": 0.5909,
+      "step": 900
+    },
+    {
+      "epoch": 27.27,
+      "eval_loss": 0.6533612012863159,
+      "eval_matthews_correlation": 0.056561954805504064,
+      "eval_runtime": 8.0431,
+      "eval_samples_per_second": 129.676,
+      "eval_steps_per_second": 16.287,
+      "step": 900
+    },
+    {
+      "epoch": 30.3,
+      "learning_rate": 4.730743243243244e-05,
+      "loss": 0.5726,
+      "step": 1000
+    },
+    {
+      "epoch": 30.3,
+      "eval_loss": 0.6678630709648132,
+      "eval_matthews_correlation": 0.10285544103286985,
+      "eval_runtime": 7.9909,
+      "eval_samples_per_second": 130.524,
+      "eval_steps_per_second": 16.394,
+      "step": 1000
+    },
+    {
+      "epoch": 33.33,
+      "learning_rate": 4.697297297297297e-05,
+      "loss": 0.5524,
+      "step": 1100
+    },
+    {
+      "epoch": 33.33,
+      "eval_loss": 0.6901304721832275,
+      "eval_matthews_correlation": 0.06313184843830015,
+      "eval_runtime": 8.3103,
+      "eval_samples_per_second": 125.507,
+      "eval_steps_per_second": 15.764,
+      "step": 1100
+    },
+    {
+      "epoch": 36.36,
+      "learning_rate": 4.663513513513514e-05,
+      "loss": 0.5167,
+      "step": 1200
+    },
+    {
+      "epoch": 36.36,
+      "eval_loss": 0.7026833891868591,
+      "eval_matthews_correlation": 0.09479027897712053,
+      "eval_runtime": 8.0473,
+      "eval_samples_per_second": 129.608,
+      "eval_steps_per_second": 16.279,
+      "step": 1200
+    },
+    {
+      "epoch": 39.39,
+      "learning_rate": 4.6297297297297295e-05,
+      "loss": 0.4779,
+      "step": 1300
+    },
+    {
+      "epoch": 39.39,
+      "eval_loss": 0.7578131556510925,
+      "eval_matthews_correlation": 0.10121883963858187,
+      "eval_runtime": 8.2334,
+      "eval_samples_per_second": 126.679,
+      "eval_steps_per_second": 15.911,
+      "step": 1300
+    },
+    {
+      "epoch": 42.42,
+      "learning_rate": 4.5959459459459463e-05,
+      "loss": 0.4271,
+      "step": 1400
+    },
+    {
+      "epoch": 42.42,
+      "eval_loss": 0.8021395802497864,
+      "eval_matthews_correlation": 0.11075235134282446,
+      "eval_runtime": 8.0267,
+      "eval_samples_per_second": 129.941,
+      "eval_steps_per_second": 16.32,
+      "step": 1400
+    },
+    {
+      "epoch": 45.45,
+      "learning_rate": 4.5621621621621625e-05,
+      "loss": 0.3888,
+      "step": 1500
+    },
+    {
+      "epoch": 45.45,
+      "eval_loss": 0.8813392519950867,
+      "eval_matthews_correlation": 0.10250522978751038,
+      "eval_runtime": 7.9207,
+      "eval_samples_per_second": 131.68,
+      "eval_steps_per_second": 16.539,
+      "step": 1500
+    },
+    {
+      "epoch": 48.48,
+      "learning_rate": 4.5283783783783786e-05,
+      "loss": 0.3428,
+      "step": 1600
+    },
+    {
+      "epoch": 48.48,
+      "eval_loss": 0.9361783862113953,
+      "eval_matthews_correlation": 0.1436834264468159,
+      "eval_runtime": 7.8976,
+      "eval_samples_per_second": 132.066,
+      "eval_steps_per_second": 16.587,
+      "step": 1600
+    },
+    {
+      "epoch": 51.51,
+      "learning_rate": 4.494594594594595e-05,
+      "loss": 0.2977,
+      "step": 1700
+    },
+    {
+      "epoch": 51.51,
+      "eval_loss": 1.078627347946167,
+      "eval_matthews_correlation": 0.11181658143300324,
+      "eval_runtime": 7.9966,
+      "eval_samples_per_second": 130.43,
+      "eval_steps_per_second": 16.382,
+      "step": 1700
+    },
+    {
+      "epoch": 54.54,
+      "learning_rate": 4.460810810810811e-05,
+      "loss": 0.2642,
+      "step": 1800
+    },
+    {
+      "epoch": 54.54,
+      "eval_loss": 1.0609544515609741,
+      "eval_matthews_correlation": 0.09013603883941315,
+      "eval_runtime": 7.8862,
+      "eval_samples_per_second": 132.257,
+      "eval_steps_per_second": 16.611,
+      "step": 1800
+    },
+    {
+      "epoch": 57.57,
+      "learning_rate": 4.427027027027027e-05,
+      "loss": 0.2272,
+      "step": 1900
+    },
+    {
+      "epoch": 57.57,
+      "eval_loss": 1.183494210243225,
+      "eval_matthews_correlation": 0.11545854045964393,
+      "eval_runtime": 7.9453,
+      "eval_samples_per_second": 131.273,
+      "eval_steps_per_second": 16.488,
+      "step": 1900
+    },
+    {
+      "epoch": 60.6,
+      "learning_rate": 4.393243243243244e-05,
+      "loss": 0.1915,
+      "step": 2000
+    },
+    {
+      "epoch": 60.6,
+      "eval_loss": 1.2531063556671143,
+      "eval_matthews_correlation": 0.12244528836896967,
+      "eval_runtime": 8.0199,
+      "eval_samples_per_second": 130.051,
+      "eval_steps_per_second": 16.334,
+      "step": 2000
+    },
+    {
+      "epoch": 63.63,
+      "learning_rate": 4.359459459459459e-05,
+      "loss": 0.1691,
+      "step": 2100
+    },
+    {
+      "epoch": 63.63,
+      "eval_loss": 1.3903430700302124,
+      "eval_matthews_correlation": 0.07541181195571064,
+      "eval_runtime": 7.9376,
+      "eval_samples_per_second": 131.4,
+      "eval_steps_per_second": 16.504,
+      "step": 2100
+    },
+    {
+      "epoch": 66.66,
+      "learning_rate": 4.325675675675676e-05,
+      "loss": 0.1491,
+      "step": 2200
+    },
+    {
+      "epoch": 66.66,
+      "eval_loss": 1.4947072267532349,
+      "eval_matthews_correlation": 0.06742580984707468,
+      "eval_runtime": 7.9502,
+      "eval_samples_per_second": 131.191,
+      "eval_steps_per_second": 16.477,
+      "step": 2200
+    },
+    {
+      "epoch": 69.69,
+      "learning_rate": 4.291891891891892e-05,
+      "loss": 0.1339,
+      "step": 2300
+    },
+    {
+      "epoch": 69.69,
+      "eval_loss": 1.5433533191680908,
+      "eval_matthews_correlation": 0.0736455413240434,
+      "eval_runtime": 8.2493,
+      "eval_samples_per_second": 126.434,
+      "eval_steps_per_second": 15.88,
+      "step": 2300
+    },
+    {
+      "epoch": 72.72,
+      "learning_rate": 4.258108108108108e-05,
+      "loss": 0.1164,
+      "step": 2400
+    },
+    {
+      "epoch": 72.72,
+      "eval_loss": 1.5793086290359497,
+      "eval_matthews_correlation": 0.11645872984022461,
+      "eval_runtime": 8.2848,
+      "eval_samples_per_second": 125.893,
+      "eval_steps_per_second": 15.812,
+      "step": 2400
+    },
+    {
+      "epoch": 75.75,
+      "learning_rate": 4.2243243243243244e-05,
+      "loss": 0.1078,
+      "step": 2500
+    },
+    {
+      "epoch": 75.75,
+      "eval_loss": 1.5938163995742798,
+      "eval_matthews_correlation": 0.09946100084630931,
+      "eval_runtime": 8.3263,
+      "eval_samples_per_second": 125.266,
+      "eval_steps_per_second": 15.733,
+      "step": 2500
+    },
+    {
+      "epoch": 78.78,
+      "learning_rate": 4.1905405405405406e-05,
+      "loss": 0.0974,
+      "step": 2600
+    },
+    {
+      "epoch": 78.78,
+      "eval_loss": 1.700919508934021,
+      "eval_matthews_correlation": 0.07568068132313144,
+      "eval_runtime": 8.4149,
+      "eval_samples_per_second": 123.947,
+      "eval_steps_per_second": 15.568,
+      "step": 2600
+    },
+    {
+      "epoch": 78.78,
+      "step": 2600,
+      "total_flos": 1.7353653717551284e+19,
+      "train_loss": 0.40566940747774566,
+      "train_runtime": 7554.577,
+      "train_samples_per_second": 508.301,
+      "train_steps_per_second": 1.986
+    }
+  ],
+  "max_steps": 15000,
+  "num_train_epochs": 455,
+  "total_flos": 1.7353653717551284e+19,
+  "trial_name": null,
+  "trial_params": null
+}