pszemraj's picture
Upload evals-outputs/GAUNTLET.md with huggingface_hub
0044939
|
raw
history blame
31.8 kB
# gauntlet results
These are are this model's output results on my "summarization gauntlet". You can find more info about that [here on my dropbox for it](https://www.dropbox.com/sh/axu1xlscrrexy55/AADAm01-4Zs3POyHQrgbDAsda?dl=0) or at [this dataset](https://huggingface.co/datasets/pszemraj/summcomparer-gauntlet-v0p1).
- if you aren't familiar with it, one thing to note is some of the docs **purposefully** are "messy"/have spelling errors etc.
parameters
```json
{
"model_name_or_path": "pszemraj/long-t5-tglobal-base-sci-simplify-elife",
"use_cuda": true,
"token_batch_length": 16384,
"batch_stride": 16,
"max_length_ratio": 0.25,
"load_in_8bit": false,
"compile_model": true,
"optimum_onnx": false,
"device": "cuda",
"inference_params": {
"min_length": 8,
"max_length": 4096,
"no_repeat_ngram_size": 3,
"encoder_no_repeat_ngram_size": 4,
"repetition_penalty": 2.5,
"num_beams": 10,
"num_beam_groups": 1,
"length_penalty": 1.0,
"early_stopping": true,
"do_sample": false
},
"textsum_version": "0.2.0"
}
```
- Created: `2023-11-28T19:56:42.336280`
## ASR-whisper-rpunctuated_Noam Chomsky, Fundam_1669853561_0_part1_summary
Language is the language that we use to communicate with each other, and so it's important for us to be able to understand how our brain works. It also helps us to understand things that are not part of our consciousness, such as what happens in our minds when we think about something. In fact, Galileo was astonished by the fact that using a few dozen sounds, he could create an infinite variety of thoughts and find ways to express all his secrets and allow him to answer any questions that he might pose. Then, at the end of the 19th century, another group of scientists called structuralists developed a new approach to studying language, which they called the bilingualism program. However, it was still unclear how these two groups would work together to study language. One of the first steps was to try to identify the internal languages of people who speak different languages. A second challenge was to find out how the speakers select a particular word or phrase from the internal language, and then how the speaker externalizes this expression into sensory motor systems. This led to the development of a theory called X-Bar Theory, which has been used to explain many aspects of language over the last several hundred years. Now, some of the most interesting results have been found in the field of computational cognitive science, which uses computer simulations to investigate how language works. These experiments show that there are three fundamental properties of language: structure, compositionality, and dislocation (which is one of the basic principles of language). They also show that human language does not depend on order and arrangement; instead, it relies on a simple mathematical principle called structure dependence. If you look back at the history of language, you'll see that humans began to develop their own language around 100,000 years ago, before language had ever appeared. That suggests that language may have evolved along with evolution.
---
Section Scores for ASR-whisper-rpunctuated_Noam Chomsky, Fundam_1669853561_0_part1_summary:
- -1.3959
---
## ASR-whisper-rpunctuated_Noam Chomsky, Fundam_1669853631_0_part2_summary
The idea of merging two things into a single object has been around since the mid-1990s, but it was not clear exactly how it works. In fact, there are many different ways that merge can be used to solve this problem. One way is to create a new set of objects, which will contain at least some of the items that you'd like to merge. Another way is by creating a workspace in which all the items you would like to combine are already present. This makes it easier for you to move from one place to the next without having to change your work space every time you use merge. It also makes it possible to make more complex objects, such as words or sentences, so that they can be easily merged together. Now, we'll look at what happens when merge is applied to language, and why it's difficult to explain it.
Many of the problems that arise in language are related to how we think about explanation. For example, when we look at a word, we can see that it's made up of two different types of verbs. One type of verb is called let, and the other is called make. So It's not possible to say that John Sawville was seen walking on the street last night. Now, as far as the question you're trying to answer, Norvin has a partial solution. The problem is that perception verbs have a tendency to cause people to leave their homes. But there's also a phenomenon called head movement, which is one of the most common problems in languages.
---
Section Scores for ASR-whisper-rpunctuated_Noam Chomsky, Fundam_1669853631_0_part2_summary:
- -1.4053
- -1.271
---
## ASRnlp_law_lecture_week_1_v_2_c_transcription_1_summary
The world's first robot lawyer is a great example of how technology can help us to understand the world around us. But there are also many other reasons why we should be skeptical about this new technology, such as the fact that it can bias our language in ways that make it harder for us to communicate with each other. For example, some languages have different notions of male and female in their language, which makes it difficult to translate from one language to the other. These biases could lead to people being more likely to become doctors or nurses than they would be if they were translated from English. So what do we need to know about natural language processing so that we can use these tools to learn about legal and social science?
---
Section Scores for ASRnlp_law_lecture_week_1_v_2_c_transcription_1_summary:
- -1.3317
---
## ASRnlp_law_lecture_week_2_v_2_c_transcription_2_summary
We're looking for people who are interested in studying economics, sociology and political science to help us understand how these three disciplines work together to solve social problems. The first step is to take a piece of text and break it down into pieces so that you can identify the words or phrases that are relevant to the topic. The second step is then to use this information to build a mathematical model of the word or phrase that will be used to represent the document. For example, we might want to find out how many times a word has been used in a given sentence. In order to do this, we need to look at all the articles that have been published over the course of the last five hundred thousand years. This was done by building a large corpus of academic publications from three different social sciences: economics (economics), sociology (sociology)and political science (political science). The results showed that even though economics had the highest number of publications, Sociology had the least amount of publications about discrimination and inequality. However, they also had the lowest number of papers about prejudice and inequality in equity. These results suggest that there is much more research being done on these issues than previously thought.
---
Section Scores for ASRnlp_law_lecture_week_2_v_2_c_transcription_2_summary:
- -1.3092
---
## ASRnlp_law_lecture_week_3_part_1_v_2_c_transcription_3_summary
The language that we use to communicate with each other is very different from the language in which we are speaking. For example, if you speak English and you say kick the bucket, you'll be more likely to get kicked out of the bathroom than when you say Bucket together. However, it is not clear why this is the case because there are many ways to reduce the number of words that show up in a language. In fact, one way to reduce vocabulary is to look at phrases that are used by people who are on the right or left side of the political spectrum. This is called feature selection and it can identify words that make sense for both Democrats and Republicans. But how often do newspapers use these phrases? How often do they use republican phrases and how often does they use Republican phrases? To answer this question, Ginscoach looked at the list of phrases that have been most commonly used by democrats as well as republicans. They found that some of these phrases were associated with a person's ability to pay back a loan, while others were related to their financial situation. These phrases also predicted whether or not the person would pay back the loan even if they did not know what they were going to do. So next time I'll go through the slides again and see if anybody has any questions about them.
---
Section Scores for ASRnlp_law_lecture_week_3_part_1_v_2_c_transcription_3_summary:
- -1.4267
---
## Emie_dissertation_cleansed_summary
In the 1950s and 1960s, a group of American and British film noir filmmakers made films that depicted modernity. These films are often called "post-war" films because they were produced in two different countries: America and Britain. However, it was not clear what happens to the moving individual as he navigates urban space after World War II. To address this question, Emilie Schzemraj set out to find out how post-war film noir directors record the movement of their protagonists through urban spaces. The first film is an American WWII veteran named Frank Enley who flees from his fellow veteran Joe Parkson on a plane across Los Angeles. The second film is a British man named Carol Reed who travels through Berlin with her lover Ivo Kern. When Susanne returns to West Berlin, she seeks to reconcile her wartime identity with her new role as a romantic interest in Germany's most notorious gangster. By recording the movements of the protagonists, the camera allows the spectator to understand how these characters struggle to resolve their troubled identities.
In the 1950s and 1960s, film noir was a genre in which films were made to record the physical world as it unfolded. The film's main protagonists are Ivo (played by Walter Benjamin), Susanne (the actress played by Lisa Mullen), and Halendar (the actor played by Karl Kracauer). When Ivo arrives at the train station, he is confronted with two checkpoints: one on the East side of the city, and another on the West side. As they approach the border, Ivo decides that if he can get away from this checkpoint, then he will be able to return to his home in West Berlin. But when Ivo returns to West Berlin after being kidnapped, the police start checking his identity papers and ticket. This makes it harder for Ivo to find out whether he has been absolved of his war-time past. It also makes it more difficult for me to reconcile my sense of self with the reality of living in a post-war European city. These films help us to understand how we have lost our sense of ourselves in the aftermath of World War II.
---
Section Scores for Emie_dissertation_cleansed_summary:
- -1.2339
- -1.3072
---
## OCR_ML4HLecture02image__summary
Computer vision is a field of computer science that uses artificial intelligence to analyze images. It can also be used to identify and quantify diseases such as cancer, Alzheimer' s disease and heart disease. The first step in this process is to divide the image into smaller regions called superpixels so that each superpixel has a different color, texture or shape. This helps to reduce the complexity of the image from thousands of millions of pixels to few hundred thousand Superpixels. However, it is not possible to do this without losing some of the important properties of the visual scene: for example, all the pixels in an image are uniform in color or texture, which makes it difficult to distinguish between normal and diseased eyes. A better understanding of how computer vision works could lead to new treatments for these diseases.
---
Section Scores for OCR_ML4HLecture02image__summary:
- -1.1811
---
## OCR_ML4HLecture04RepresentationLearning.pptx__summary
A computer model of a person's health can be used to predict future health states and diseases. However, it is not possible to accurately represent the patient' s health state in this way. To overcome this problem, researchers have developed new representation learning algorithms that are easier to interpret than traditional representation learning methods. For example, an algorithm called Sequence-To-Sequence Learning (or SOMVAE for short) uses a neural network to encode data into a vector form. Then, using a technique called Generative modeling, the model learns which features of the data are most likely to be similar or different from the original data. This allows the model to better understand how the patients' health state changes over time. Further experiments show that SOMVAE performs better than other representation learning approaches when there is too much data to use as a model. These results suggest that machine learning may be a useful tool for improving clinical predictions through unsupervised time series representation learning.
---
Section Scores for OCR_ML4HLecture04RepresentationLearning.pptx__summary:
- -1.2589
---
## OCR_ML4HLecture05-NLP.pptx__summary
Clinical reports contain important information about patients' health status, but they are written in difficult, specialized languages. For example, the patient's name is often missing or concatenated with short phrases and acronyms, which can be overloaded. This makes it difficult to get access to these clinical reports because many sections of the report are long and detailed; other sections are empty or only have some captions. To address this problem, Ezurich has developed a technique called machine learning for health care (or MLM for short) that uses natural language processing as a tool to help clinicians understand how their patients respond to treatment. The MLM technique allows clinicians to use computer simulations to predict what will happen when a patient is diagnosed with a specific disease. It also helps clinicians identify drugs that may be useful for treating certain conditions. In the future, MLM could be used to develop new treatments for diseases such as leukemia and cancer.
---
Section Scores for OCR_ML4HLecture05-NLP.pptx__summary:
- -1.291
---
## OCR_PAPER_Hong et al. - 2022 - CogVideo Large-scale Pretraining for Text-to-Video Generation via Transformers-annotated__summary
Video generation is one of the most challenging tasks in computer vision, because it requires a large number of images and video clips to be generated. However, these images and videos are often not well understood, which makes it difficult for the model to understand the movements of the human body. One way to overcome this problem is to use pre-trained transformers, which have been shown to be better at generating movies than other types of neural models. Previous work has focused on predicting the next frame of a video, but there are still many challenges with this task. For example, when a lion drinks water, the model tends to misinterpret the movement of the lion' s lips, making it hard to correctly generate an action such as drinking water. To overcome these challenges, Wu, Zheng, Hong, and Jie Tang developed a new open-source computational model called Cog Video. The model was trained using a previously unpublished pretrained model that already had good command of image and video relations. The results show that CogVideo performs better than all publicly available modeling methods at a larger margin in both machine and human assessments. In addition, the training method also improves the accuracy of the model by ensuring that the model can control how much changes occur during the generation process.
---
Section Scores for OCR_PAPER_Hong et al. - 2022 - CogVideo Large-scale Pretraining for Text-to-Video Generation via Transformers-annotated__summary:
- -1.2985
---
## OCR_PAPER_Kandpal, Nieto, Jin - 2022 - Music Enhancement via Image Translation and Vocoding-annotated__summary
The rise of the Internet has led to a large number of music enthusiasts and influencers looking for ways to improve their listening experience. For example, musicians often record low-quality audio recordings on cheap and accessible devices that are not treated with any noise or resonance. However, many aspects of these recordings are unknown, such as the frequency response of the device, the size and shape of the recording space, and the sound quality of the environment. To address this problem, Adobe Research developed a new approach called MelZMel, which is based on recent advances in conditional image processing and vocoder. This approach uses a technique called Diffwave to generate high-quality speech waves from a mixture of vocals, drums and other musical sources. These waveforms are then processed by a deep learning model that transforms them into real-time sounds. In the experiments, human participants were asked to identify which song they thought was better than the one they had heard before. The results showed that the method used by Adobe Research achieved a much higher level of perceptual improvement than several existing methods. Further experiments show that the proposed method can be used to enhance any type of audio signal without the need for retraining. Future work could also focus on improving the ability of computers to distinguish between different types of audio signals.
---
Section Scores for OCR_PAPER_Kandpal, Nieto, Jin - 2022 - Music Enhancement via Image Translation and Vocoding-annotated__summary:
- -1.2641
---
## OCR_PAPER_dall-e-2-annotated__summary
Computer vision is a field of computer science that relies on the ability of computers to recognize and interpret images. In particular, computer vision has emerged as a powerful tool for learning how to represent images in ways that capture both their semantics and their style. However, it is still not clear how computer vision can be used to create realistic images. To address this question, Alex Nichol developed a new approach called CLIP, which uses a technique called diffusion models to generate images from captioned text. These models are computationally efficient and produce high-quality images, but they do not have the same features as traditional computer vision systems. Instead, they use a combination of diffusion models and artificial neural networks to learn what an image looks like when it is encoded by CLIP. The experiments show that using diffusion models improves the quality of the images produced by the system, while also improving the accuracy of the predictions made by the model. Further work is now needed to find out more about how Dhariwal' s two-stage approach works.
---
Section Scores for OCR_PAPER_dall-e-2-annotated__summary:
- -1.2469
---
## The Most Dangerous Game--Richard Connell_summary
The most dangerous game in the world is a sport called hunting, and it takes place on an island off the coast of Rio de Janeiro. It is known as Ship Trap Island because it has a bad reputation amongst seafaring people. Some sailors have a strange dread that they cannot see through the blackness of the night. However, there is no such thing as a ship-trap island; it is only a few miles away from the Caribbean Sea. There are two classes of people who live on this island: those who hunt big game, and those who don't. One class lives for danger, while the other lives for pleasure. But how do we know when we are in danger? To answer this question, Richard Connell set out to find out. He found a small island where he had built a house so that he could go hunting with his quarry. When General Zaroff saw Rainsford walking along the edge of the island, he asked him what he was doing. The general replied, "I am going to tell you about my new quarry." The general went on to say that rainsford was a good hunter, but not a great hunter. In fact, one of the best hunters in the history of the world has never been able to kill a bear or a lion. Now, after more than three hundred hours of searching, Richard connell has developed a new sense of fear.
---
Section Scores for The Most Dangerous Game--Richard Connell_summary:
- -1.3965
---
## gpt_peter_testing_group_exemplars_summary
When I was a kid, my parents would always tell me that they wanted me to grow up in a certain way so that I could become a better person. Now, when I am a grown-up, I want to know how much money I've made from selling things to making me a wealthy man. How do you make sure that you have enough money to live off of? If you are going to go on a camping trip this weekend, what kind of food do you plan to bring with you? What is the most efficient way to produce heat as human beings? To answer this question, we need to create an artificial intelligence that can detect and stop bad things happening. This will help us to find ways to prevent them from happening.
---
Section Scores for gpt_peter_testing_group_exemplars_summary:
- -1.3363
---
## navy seals copy pasta_summary
You're little bitch, and you're not able to kill me.
---
Section Scores for navy seals copy pasta_summary:
- -1.5555
---
## script_findingnemo_summary
The ocean is a great place to live, but it's also a dangerous place. There are over 400 different species of jellyfish that live in the ocean, and they all have their own unique identity. This makes it difficult for them to distinguish between food and seaweed. A fish called Marlin wants to go to school so he can learn more about how the ocean works. However, when he first arrives at school, he sees a white boat with a shark on its back. He doesn't know what kind of shark he'd like to see. When he gets to school, his teacher tells him that he has to answer a question about the ocean. If he does not answer this question, then he won't be able to find his way back to school. Now, we need to get our heads out of the water and find our way back home.
Nemo is a sea turtle that lives in the Pacific Ocean. He and his father have been searching the ocean for several days, but they couldn't find him. The boat was too slow to stop the sharks so they went out into the ocean. They bumped into a small fish called Small Fish and then dived thousands of feet down into the deepest part of the ocean where they could see a light from a big terrible creature with razor-sharp teeth. It turns out that these sharks are not the same as the ones that were taken off the coral reef by divers. This means that they may be on their way to Sydney within the next few days. But before they can get there, they need to find their missing son. To do this, two little sea turtles must swim along the east Australian current and search for their missing child. If they don't come back soon, they're going to have to wait until Darla arrives at the harbor. There'll be plenty of time between now and Darla's arrival.
---
Section Scores for script_findingnemo_summary:
- -1.3283
- -1.3068
---
## script_frozendisney_summary
A group of men, including a reindeer called Sven, drag giant blocks of frozen ice through the water. Then, they sing a song called "the frozen heart" as they cut through the ice and break it down. In the middle of the night, two young Sami boys—Kristoff and Sven—struggle to get an ice block out of the freezing water. When they finally manage to do this, they're left behind on a small sled that pulls away from them. They then head back to their new home in the kingdom of Arendelle, where Elsa sleeps with her little sister Anna. But when she wakes up, she realizes that she's not going to be able to remember how she used to play snowballs or build snowmanes. Instead, she'll need to learn how to control her power so that she doesn't fall apart. She'll also need to find ways to stop people from getting too close to her.
It's winter, and it's time for us to go back to our normal lives. We've been living in the mountains for a long time, but this winter is so cold that we have no idea where we're going or what we'll be doing when we get back. Now, Anna and her sister Kristoff are trying to find their way back to their normal life. Then, a snowman comes up behind them and knocks them down. Anna grabs on to Olaf’s arm and pulls him along with her. When they reach the top of the mountain, Sven leaps off his sledding sled into the wind-swept ice. He turns to face Anna, who looks at him as if she had never seen winter before. Anna smiles and tells him how much she loves him. She goes on to say that she doesn't want anything to do with him anymore. But now, she's not sure whether she'll ever return to her normal life again. Instead, she wants to bring summer back. To do so, she must stop Elsa from destroying Arendelle by bringing back summer.
Olaf and Anna are stranded in the middle of a winter storm. They have to find a way to get back to Arendelle, where they'll be reunited with their beloved Reindeer King Hans. But it isn't clear what true love is. The wind whips around them as they try to make it across the ice-covered mountains. When they reach the top of the mountain, they see a sharp ice clawing its way into the castle. Kristoff jumps on his back and climbs over the wall that separates the two kingdoms. Sven pushes him out of the way, but he doesn't look like a real reindeer. Instead, he looks up at the sky and sees a huge snowflake floating in the air. He turns away from Elsa and kisses her face. It was only a few days ago that Hans had been thrown into an brig by a group of French dignitary guards. Now, Hans has found a new place to live for the rest of his life. In one last wave, she pulls all of the Snowflake into a gigantic snowfleet in the sky. She waves it away so that everyone can enjoy a summer day.
---
Section Scores for script_frozendisney_summary:
- -1.2191
- -1.4298
- -1.3958
---
## script_strangersonatrain_summary
The story of Bruno Anthony and Guy Haines takes place on a train in the United States. It is about a young tennis player who wants to marry his wife's daughter so that they can have a child together. A taxi pulls up and stops at a station, where a chauffeur drives up and hand out modest looking suitcases including a bundle of tennis-rackets as well as a few cases of luggage. The passenger wearing the sport shoes is a middle-aged man who has been playing tennis for many years. He also wears a pair of black and white brogues, which are all we saw of him. At the same time, two young men are sitting in one of the parlor cars. One of these young men looks at Bruno's gold tie pin and says: "I don't look like a tennis player." This suggests that he doesn't know what he's going to do when he gets divorced.
Miriam Burton was a young woman who lived in Washington, D.C. She had been involved in a murder that took place at an amusement park on the way to her father's house. Now, she and Guy are trying to find out what happened last night. One of the suspects is a man called Bruno Haines, who has been working as a tennis player for sixteen hours every day. As they walk along the street, Anne and Guy look at each other with a renewed sense of happiness. They realize that it would be great to have someone else love them so much that they could kill for them. The next step will be to talk to their father about this mystery.
Guy Haines is a professional tennis player who has recently been arrested for the killing of his wife, Miriam Haines. The police are trying to find the lighter that Guy left on the island in order to prove that he was involved in the murder. He also told Anne that if she wanted him to go back and pick up her lighter so they wouldn't be able to find it, that would have made her an accessory. Now, Guy walks out of the living room and sits near the window, looking down at the light in his pocket. Guy then takes another cigarette from his pocket and stares down at it as if waiting for someone to tell him where he'd found it. Guy goes on to tell Anne about how he had tried to get her to return to the island after dark one night and picked up his lighter just so the police could not find it. As soon as Guy gets there, he starts to talk to Anne with a voice that sounds like a whisper: "I'm going to wait until the sun comes up before I play this game." It's hard to believe that such a sudden change in tone can lead to such dramatic changes in behavior.
---
Section Scores for script_strangersonatrain_summary:
- -1.3103
- -1.2116
- -1.3679
---
## script_sunsetblvd._summary
Sunset Boulevard is one of the most famous streets in Los Angeles, and it has been used as a backdrop for some of the greatest movies ever made. It is also known as "Sunset Boulveard" because its street sign was stencilled on an old curbstope with dead leaves, burnt matchs and cigarette Butts. The film's main character, Joseph Gillis, plays a shortstop in a baseball game called the World Series. When Gillis arrives at the parking lot, he finds that there are two men standing outside his car. One of them is carrying a small briefcase, while the other is holding a typewriter. Gillis grabs the keys from the back of his car and starts to drive down Sunset Boulevard. A few minutes later, Gillis turns around and sees a group of people walking towards him. He walks over to their car and pulls it out of the garage, but they don't seem to know what's going on. Instead, they find themselves behind a fast-moving truck. They decide to make a quick U-turn and start after Gillis. This leads to a large empty garage, where Joe Gillis waits until he can buy a new car. At the same time, Max Von Mayerling comes into the house and asks Gillis to write a script for a movie about Salome.
It's New Year' s Eve in Los Angeles, and some of the people are having a good time. One of the guests is a young man called Artie Green who has been working as a screenwriter for over twenty years. Gillis makes his way to the front of the room, where he sits with a glass of champagne on his head. The musicians start playing away, but Gillis turns his eyes away from them so that they don't look at him. In the middle of the night, Gillis and Artie make their way to a small one-room apartment near the corner of Sunset and Bronson avenues. There are only a handful of people there, most of whom are wearing sweaters and shorts, and few of the women are dressed in something that suggests party wear. When Gillis arrives, he takes out a bottle of wine and puts it into a bowl filled with punch. By the end of the evening, all the guests have had a wonderful time, and Gillis can't wait to get back to work.
It's been a long time since I've made a picture, but now I'm back to make another one. This time it's about a woman called Norma desmond, who lives in an old Hollywood house with eighteen master bedrooms and a sunken tub inside the bathroom. She is trying to find out where she lives, how much she knows about him, and what he does for a living. When she arrives at the front door of the house, she sees that there are two policemen on the other side of the room. One of them is wearing a typewriter, while the other is carrying a suitcase full of his belongings. The second man is holding a camera, which is being set up around the house. The third man is standing at the bottom of the staircase, waiting for her to come down. As she sits at the mirror, she hears a sound of gunshots coming from the corner of the bedroom. In the next part of the scene, Gillis walks towards her, looking at her blankly as she stares at herself. He goes on to tell her that if she didn't get away with it, she would never be able to work again.
---
Section Scores for script_sunsetblvd._summary:
- -1.3918
- -1.3679
- -1.3256
---