project: Multitask Learning for Agent-Action Identification
Project Overview This project aims to develop a multitask learning model for identifying agents and actions in text data. The model is trained on a custom dataset of text examples, where each example is annotated with the agents and actions present in the text. Project Structure The project is organized into the following directories and files: dataset/: contains the custom dataset class for loading and processing the text data dataset.py: defines the dataset class data_collator.py: defines the data collator class model/: contains the multitask learning model architecture model.py: defines the model architecture training/: contains the training loop and evaluation code main.py: contains the training loop and evaluation code data/: contains the dataset files for training, validation, and testing train.csv: training dataset val.csv: validation dataset test.csv: testing dataset requirements.txt: lists the dependencies required to run the project Dataset The dataset consists of text examples, where each example is annotated with the agents and actions present in the text. The dataset is split into training, validation, and testing sets. Training Set: 80% of the dataset (10,000 examples) Validation Set: 10% of the dataset (1,250 examples) Testing Set: 10% of the dataset (1,250 examples) Model The model is a multitask learning model based on the BERT architecture. The model is trained to predict both agents and actions simultaneously. Model Architecture: BERT encoder Two classification heads for agents and actions Model Parameters: BERT encoder: 110M parameters Classification heads: 10M parameters Training The model is trained using the Trainer class from the Hugging Face library. The training loop is defined in main.py. Training Hyperparameters: Batch size: 16 Number of epochs: 3 Learning rate: 1e-5 Training Time: approximately 10 hours on a single NVIDIA V100 GPU Evaluation The model is evaluated on the validation set during training. The evaluation metric is accuracy. Evaluation Metric: accuracy Evaluation Frequency: every 500 steps Requirements The project requires the following dependencies: Python: 3.8+ Transformers: 4.20.1+ Torch: 1.12.0+ Pandas: 1.4.2+ Usage To train the model, run the following command: Bash python main.py To evaluate the model, run the following command: Bash python main.py --mode eval License This project is licensed under the MIT License. Acknowledgments This project was inspired by the work of [Dennis Duncan]. Contributing Contributions are welcome! Please open an issue or submit a pull request to contribute to the project.