Karl Childers (Billy Bob Thornton) is an intellectually disabled Arkansas man who has been in the custody of the state mental hospital since the age of 12 for having killed his mother and her lover. Although thoroughly institutionalized, Karl is deemed fit to be released into the outside world. Prior to his release, he is interviewed by a local college newspaper reporter, to whom he recounts committing the murders with a Kaiser blade, saying, "Some folks call it a sling blade. I call it a kaiser blade." Karl says he thought the man was raping his mother. When he discovered that his mother was a willing participant in the affair, he killed her also.
Thanks to the doctor in charge of his institutionalization (James Hampton), Karl lands a job at a repair shop in the small town where he was born and raised. He befriends 12-year-old Frank Wheatley (Lucas Black) and shares some of the details of his past, including the killings. Frank reveals that his father was killed - hit by a train - leaving him and his mother on their own. He later admits that he lied, and that his father committed suicide.
Frank introduces Karl to his mother, Linda (Natalie Canerday), as well as her gay friend, Vaughan Cunningham (John Ritter), the manager of the dollar store where she is employed. Despite Vaughan's concerns about Karl's history in the mental hospital, Linda allows him to move into her garage, which angers Linda's abusive alcoholic boyfriend, Doyle Hargraves (Dwight Yoakam). Karl bonds with Linda, who makes him biscuits to eat, and also with Vaughan, who tells Karl that a gay man and a mentally challenged man face similar obstacles of intolerance and ridicule in small-town America.
Karl quickly becomes a father figure to Frank, who misses his father and despises Doyle. Karl is haunted by the task given to him by his parents when he was six or eight years old to dispose of his premature, unwanted, newborn brother. He visits his father (Robert Duvall), who has become a mentally unbalanced hermit living in the dilapidated home where Karl grew up. Karl's parents performed an abortion, causing the baby to "come out too soon," and Karl was given a bloody towel wrapped around the baby, which survived the abortion. Karl was instructed to "get rid of it," but when Karl detected movement inside the towel, he inspected it, discovering "a little ol' boy" that "wasn't no bigger than a squirrel."
While recounting this story to Frank, Frank asks why Karl just didn't keep the baby, but Karl replies he had no way to care for a baby. He placed the baby, still in the bloody towel, inside a shoe box and buried the baby alive, saying he felt it was better to just "return him to the good Lord right off the bat," because of the abuse and neglect he himself had received at the hands of his own parents. Karl tells his father that killing the baby was wrong, and that he had wanted to kill his father for making him do it, but eventually decided that he wasn't worth the effort.
Meanwhile, Doyle becomes increasingly abusive towards Karl and Frank, leading to an eventual drunken outburst and physical confrontation with Linda and Frank, angering the boy. Linda then kicks Doyle out of the house (despite his threats to kill her if she ever left him). The next day, Linda and Doyle reconcile. Knowing that he has the upper hand again, Doyle confronts Karl and Frank and announces his plan to move into the house permanently; he plans "big changes" including Karl's removal from the house. Karl begins to realize that, eventually, Frank is going to kill Doyle and end up just like him. In order to prevent this, Karl makes Frank promise to spend the night at Vaughan's house, and asks Vaughan to pick up Linda from work and have her stay over also.
Karl returns to Linda's house, but seems undecided about whether to enter. When confronted, a drunk Doyle asks what Karl is doing with the lawnmower blade he had sharpened and fashioned into a weapon. Karl replies, "I aim to kill you with it." Karl asks how to reach the police by phone, Doyle says Karl should dial 911 and request "an ambulance, or a 'hearst'". Karl kills Doyle with two chopping blows of the lawnmower blade to the head. Karl then calls the police to turn himself in, and requests a "hearst" be sent for Doyle. He eats biscuits while waiting for the police.
Returned to the state hospital, he seems a different person than he was during his previous incarceration, having learned the value of sacrificing one's self to save others. He silences a sexual predator (played by J. T. Walsh) who had previously forced him to listen to tales of his horrible deeds.
Output: Why did Karl's brother "come too soon"?