GGML version of emozilla/LLongMA-2-7b-storysummarizer. Use n_ctx=8192 and rope_freq_scale=0.5. For an excellent user interface, check out LM Studio.
LLongMA-2-7b-storysummarizer
This is a fine-tuned version of conceptofmind/LLongMA-2-7b intended for summarization and literary analysis of fiction stories.
It contains custom modeling code to use Flash Attention 2 during inference, which provides a significant speedup, especially at longer context lengths.
To enable, pass trust_remote_code=True,use_flash_attention=True to AutoModelForCausalLM.
How to Use
SOME_FICTION
### SUMMARY:
or
SOME_FICTION
### ANALYSIS:
Example outputs
Outputs on the text of Waystation City (6,904 tokens)
Summary
temperature=0.8, repetition_penalty=1.1,do_sample=True
A nineteen-seventies pair of twins approaches Micheala and Gerty at a cafe. They ask them to write down their story of their disappearance. Gerty takes notes on their story as they tell it. The twins' names are Daphne and Claude. The twins explain that they came from London, England and were leading a movement against fascism there. But in Waystation City, they were stuck in the limbo between timelines with no real way of returning home to continue their work. In fact, it was harder for them to remember their lives at home as time went on. They tell Gerty that they cannot stay and Micheala encourages them not to go down to the stairs where Waystation City's ferry drivers will take them home. Daphne writes an inscription on a wall, leaving her mark as one of the disappeared, but she and Claude relent. Together they approach the docks with Gerty and Micheala trailing behind. There, the twins discuss which hole to go through and they agree it is the one showing their hometown in the 1970s. Daphne goes first and then Claude after he slips on a wet surface. The ferry drivers see him but Gerty yells for him to return from the hole as she writes his story down with her notes. But he cannot hear her over the slip of metal slats turning into stone and wood on the docks. He jumps in anyway and disappears into the other world through the hole. Micheala leaves a cigarette and Gerty drops her ticket back home to show that she's decided to stay in Waystation City where it is possible to grow, change, and live in the spaces between.
Analysis
temperature=0.8, repetition_penalty=1.1,do_sample=True
Notes In this final chapter, there is a brief but important flashback to the twins' home in England at the beginning of the 1970s, when racism and fascism are rampant. It is interesting to note the author's choice to set the story at such a pivotal time for immigrants in Britain. The twins feel like they have to choose between being their "true" selves in their home country or staying where they are as Waystation citizens. But this was also the time when racism and fascism were so openly practiced that anti-racists were fighting against these institutions with all of their might. It is interesting how there's an implication here that it's only at the risk of their "true" identities that the twins can go back to England. In this chapter, Gerty changes her name as well, choosing the gender neutral Michael. She does not specify why she decided to change her name or what inspired her choice. The story implies, but never says outright, that she changed her name in part because of the twins' disappearance. This suggests an important question: how far must a person be willing to go in order to stay true to themselves? It is clear by this point that Gerty has changed quite a bit since coming to Waystation City and her decision to change her name reflects the extent to which she has altered her identity. She realizes that she has settled into a comfortable, new life here with new clothes, new hobbies, and friends. Ultimately, this chapter suggests that while it is important to stay true to one's past, there are also new things to discover and experience in the present. Gerty has done just this by settling in Waystation City, embracing her new life and even changing her name and identity with it.
Training
The model was trained on emozilla/booksum-summary-analysis_llama-8192, which is adapted from kmfoda/booksum. The training run was performed using axolotl. The run can be viewed on wandb.