MSc Practicum Everitt-Ryan 2024

Our research aims to advance the application of Natural Language Processing (NLP) techniques in the detection of biases in job advertisements, which can affect applicant diversity by embedding prejudices within text such as age and gender.

Our hypothesis is that LLMs, due to their reasoning skills, can de-bias the HR ecosystem. We aim to validate this through three research questions:

1. Can foundation LLMs faithfully identify implicit and subtle bias in job descriptions? Ideally, we hope to study the implicit information extraction capability of LLMs without being trained explicitly for the task. We plan to test various LLMs and scales (<= n billion-parameter models) to establish our hypothesis.
2. Which prompting method (zero-shot, few-shot, or chain-of-thought) most efficiently identifies bias in text? We aim to carry out comprehensive experiments on different prompting strategies already proposed in the literature. We also aim to propose different prompting templates covering the three settings and document our observations across the LLMs.
3. Does domain adaptation via fine-tuning foundational LLMs improve on prompt tuning? We aim to perform domain adaptation by fine-tuning the selected models on our datasets. We will seek to investigate if fine-tuning improves or lowers their performance under the same prompt settings.