KevSun/IELTS_essay_comments

This generative language model is designed to provide specific comments on five aspects: task achievement, coherence and cohesion, vocabulary, grammar, and overall assessment, , after analyzing an English essay. The model is particularly useful for essays in the IELTS (International English Language Testing System) exams. It assists L2 learners and IELTS students in improving their writing abilities by correcting mistakes and offering detailed feedback and suggestions.

Using the text similarity method to evaluate in the dataset with 1500 IELTS essays with the five aspects of comments, the model can acheive the following results: acc = 0.93, f1 score =0.92, which is remarkably impressive.

The following is an example to illustrate how to use our model to make inference:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

# Define the model path where your trained model is saved
model_path = "./model/"  # Update this to your saved model path

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path)

# Load the model with 4-bit quantization configuration
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=False,
)

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    quantization_config=bnb_config,
    device_map="auto"
)

# Ensure the model is in evaluation mode
#model.eval()

# Define the new input text
input_text = "More people decided to have children in their later age than in the past. Why? ..."
# Tokenize the input
inputs = tokenizer(input_text, return_tensors="pt", truncation=True, padding='max_length', max_length=512)

# Move the inputs to the appropriate device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
inputs = {key: value.to(device) for key, value in inputs.items()}

# Generate predictions with no gradient computation
with torch.no_grad():
    outputs = model.generate(**inputs, max_length=512)

# Decode the generated predictions
decoded_outputs = tokenizer.decode(outputs[0], skip_special_tokens=True)

#print("Generated Output:")
#print(decoded_outputs)


# Parse the structured output
output_keys = ["Task Achievement", "Coherence and Cohesion", "Vocabulary", "Grammar", "Overall"]
comments = {}

# Split the decoded output into sections based on the output_keys
for key in output_keys:
    # Find the start of the key
    start_index = decoded_outputs.find(f"{key}:")
    if start_index != -1:
        # Find the end of the key (next key start or end of text)
        end_index = len(decoded_outputs)
        for next_key in output_keys:
            if next_key != key:
                next_index = decoded_outputs.find(f"{next_key}:", start_index + len(key))
                if next_index != -1 and next_index < end_index:
                    end_index = next_index
        # Extract the comment
        comments[key] = decoded_outputs[start_index + len(key) + 1:end_index].strip()

# Display the comments
for key, comment in comments.items():
    print(f"{key}: {comment}")
'''

The sample result is as follows:

input: "You should spend about 40 minutes on this task.\nMore people decided to have children in their later age than in the past. Why? Do advantages of this outweigh disadvantages?\nGive reasons for your answer and include any relevant examples from your own knowledge or experience. \nYou should write at least 250 words.", In these career oriented world people are postponing their family planning to late stage of their life. Perhaps, the most important reason is goal oriented. In my opinion, benifits of having offsprings in later age offsets the drawbacks entailed by this trend, as one can focus on their career and enjoy their life.\nThe major disadvantage of this trend offers is related to health. Having childrens in later stage may pose a risk to the mother because they may not be healthy enough to give birth to child. These may result in birth of a child with abnormal syndrome. For example, a research published in gynegology has found that females giving birth after the age of 35 years are highly prone to give birth to a diseased child.\nFirstly, primary reason behind planning children in later stage is that these days people are becoming more career oriented. These means that they have to put their energy, time and money in acheving their dreams.\nThese may make them competent for upbring of their childrens in a well manner.\nMoreover, having childrens at later stage offers several advantages. The first obvious benifit is that people can acheive the heights in their careers as they can focus on theri career. They can devote all their resources and energy in to acheving their dreams. For instance, a growing body of research have found that 80% of people are successful in their career those who have planned their offspring in later stages of life.\nSecondly, people can enjoy their life because they donot have responsibility of nuturing their childrens. This further encourages them to explore the world in the way they want. For example, they can travel to the tourist places, explore the landscape and culture of other nations. Also they can develop their hobbies.\nIn conclusion, despite of having a disadvantage of helath, having childrens in later stge of life offers more advantages of acheving career and enjoying life.",

"output":

[Task Achievement]: -The candidate has addressed the given task adequately by discussing the reasons for people choosing to have children later in life and comparing the advantages and disadvantages of this trend. - The response is reasonably clear and coherent, with ideas presented in a logical order. - The candidate has covered all parts of the task and supported arguments with examples. "Coherence and Cohesion": The essay is well-organized with clear transitions between paragraphs. - The use of linking devices, such as "Firstly" and "Secondly," helps connect ideas smoothly. - There is a logical flow of information, making the essay easy to follow.

[Vocabulary]: The candidate has used a range of vocabulary, including some advanced words and "syndrome." - There are some inaccuracies in vocabulary, (should be "benefits") and "acheving" (should be "achieving"). - The vocabulary is generally appropriate for the context. - Specific Vocabulary Mistakes and Corrections: | Mistake | Correction | |---|---| | benifits | benefits | | acheving | achieving |

[Grammar]: The essay demonstrates a variety of sentence structures, including simple, compound, and complex sentences. - There are some grammatical errors, (should be "This means") and "offer is" (should be "offers"). - The use of punctuation is generally correct. -

[Overall]: Considering the holistic performance across all criteria, the essay deserves an overall band score of 7. - The essay meets the task requirements, is well-organized, and uses a range of vocabulary. - There are some areas for improvement in terms of vocabulary accuracy and grammatical errors.

Strengths - Clear and coherent organization - Logical presentation of ideas - Use of examples to support arguments Areas for Improvement- Accuracy in vocabulary usage - Elimination of grammatical errors - Expanding the range of sentence structures Suggestions for Enhancement - The candidate could benefit from using a wider range of advanced vocabulary and ensuring accuracy.

• Practicing sentence construction and grammar rules would help improve grammatical accuracy.
• Reading high-quality essays and paying attention to sentence structure can enhance writing skills. '''