Model: 2024-mcm-everitt-ryan/roberta-large-job-bias-seq-cls
Author: 2024-mcm-everitt-ryan
Pipeline: text-classification
Library: transformers
Tags: transformers, safetensors, roberta, text-classification, en, dataset:2024-mcm-everitt-ryan/job-bias-synthetic-human-benchmark-v2, arxiv:1910.09700, base_model:FacebookAI/roberta-large, base_model:finetune:FacebookAI/roberta-large, license:apache-2.0, model-index, endpoints_compatible, region:us
Base models (finetune): id: FacebookAI/roberta-large
Card metadata: base_model: FacebookAI/roberta-large
datasets: 2024-mcm-everitt-ryan/job-bias-synthetic-human-benchmark-v2
language: en
license: apache-2.0
model_id: roberta-large-job-bias-seq-cls
model_description: The model is a multi-label classifier designed to detect various types of bias within job descriptions.
developers: Tristan Everitt and Paul Ryan
model_card_authors: See developers
model_card_contact: See developers
repo: https://gitlab.computing.dcu.ie/everitt2/2024-mcm-everitt-ryan
training_regime: accelerator_config="{'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}", adafactor=false, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=false, batch_eval_metrics=false, bf16=false, bf16_full_eval=false, data_seed="None", dataloader_drop_last=false, dataloader_num_workers=0, dataloader_persistent_workers=false, dataloader_pin_memory=true, dataloader_prefetch_factor="None", ddp_backend="None", ddp_broadcast_buffers="None", ddp_bucket_cap_mb="None", ddp_find_unused_parameters="None", ddp_timeout=1800, deepspeed="None", disable_tqdm=false, dispatch_batches="None", do_eval=true, do_predict=false, do_train=false, eval_accumulation_steps="None", eval_batch_size=8, eval_delay=0, eval_do_concat_batches=true, eval_on_start=false, eval_steps="None", eval_strategy="epoch", evaluation_strategy="None", fp16=false, fp16_backend="auto", fp16_full_eval=false, fp16_opt_level="O1", fsdp="[]", fsdp_config="{'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}", fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap="None", full_determinism=false, gradient_accumulation_steps=1, gradient_checkpointing="(False,)", gradient_checkpointing_kwargs="None", greater_is_better=false, group_by_length=true, half_precision_backend="auto", ignore_data_skip=false, include_inputs_for_metrics=false, jit_mode_eval=false, label_names="None", label_smoothing_factor=0.0, learning_rate=3e-05, length_column_name="length", load_best_model_at_end=true, local_rank=0, lr_scheduler_kwargs="{}", lr_scheduler_type="linear", max_grad_norm=1.0, max_steps=-1, metric_for_best_model="loss", mp_parameters="", neftune_noise_alpha="None", no_cuda=false, num_train_epochs=3, optim="adamw_torch", optim_args="None", optim_target_modules="None", past_index=-1, per_device_eval_batch_size=8, per_device_train_batch_size=8, per_gpu_eval_batch_size="None", per_gpu_train_batch_size="None", prediction_loss_only=false, ray_scope="last", remove_unused_columns=true, report_to="[]", restore_callback_states_from_checkpoint=false, resume_from_checkpoint="None", seed=42, skip_memory_metrics=true, split_batches="None", tf32="None", torch_compile=false, torch_compile_backend="None", torch_compile_mode="None", torchdynamo="None", tpu_num_cores="None", train_batch_size=8, use_cpu=false, use_ipex=false, use_legacy_prediction_loop=false, use_mps_device=false, warmup_ratio=0.1, warmup_steps=0, weight_decay=0.001
results:                   precision    recall  f1-score   support
    
             age       0.82      0.57      0.68        80
      disability       0.87      0.42      0.57        80
        feminine       0.94      0.93      0.93        80
         general       0.61      0.14      0.22        80
       masculine       0.69      0.65      0.67        80
         neutral       0.29      0.72      0.41        80
          racial       0.93      0.71      0.81        80
       sexuality       0.86      0.81      0.83        80
    
       micro avg       0.66      0.62      0.64       640
       macro avg       0.75      0.62      0.64       640
    weighted avg       0.75      0.62      0.64       640
     samples avg       0.63      0.65      0.64       640
    
compute_infrastructure: - Linux 6.5.0-28-generic x86_64
- MemTotal:       527988292 kB
- 64 X Intel(R) Xeon(R) Silver 4314 CPU @ 2.40GHz
- GPU_0: NVIDIA L40S
software: python 3.10.12, accelerate 0.32.1, aiohttp 3.9.5, aiosignal 1.3.1, anyio 4.2.0, argon2-cffi 23.1.0, argon2-cffi-bindings 21.2.0, arrow 1.3.0, asttokens 2.4.1, async-lru 2.0.4, async-timeout 4.0.3, attrs 23.2.0, awscli 1.33.26, Babel 2.14.0, beautifulsoup4 4.12.3, bitsandbytes 0.43.1, bleach 6.1.0, blinker 1.4, botocore 1.34.144, certifi 2024.2.2, cffi 1.16.0, charset-normalizer 3.3.2, click 8.1.7, cloudpickle 3.0.0, colorama 0.4.6, comm 0.2.1, cryptography 3.4.8, dask 2024.7.0, datasets 2.20.0, dbus-python 1.2.18, debugpy 1.8.0, decorator 5.1.1, defusedxml 0.7.1, dill 0.3.8, distro 1.7.0, docutils 0.16, einops 0.8.0, entrypoints 0.4, evaluate 0.4.2, exceptiongroup 1.2.0, executing 2.0.1, fastjsonschema 2.19.1, filelock 3.13.1, flash-attn 2.6.1, fqdn 1.5.1, frozenlist 1.4.1, fsspec 2024.2.0, h11 0.14.0, hf_transfer 0.1.6, httpcore 1.0.2, httplib2 0.20.2, httpx 0.26.0, huggingface-hub 0.23.4, idna 3.6, importlib_metadata 8.0.0, iniconfig 2.0.0, ipykernel 6.29.0, ipython 8.21.0, ipython-genutils 0.2.0, ipywidgets 8.1.1, isoduration 20.11.0, jedi 0.19.1, jeepney 0.7.1, Jinja2 3.1.3, jmespath 1.0.1, joblib 1.4.2, json5 0.9.14, jsonpointer 2.4, jsonschema 4.21.1, jsonschema-specifications 2023.12.1, jupyter-archive 3.4.0, jupyter_client 7.4.9, jupyter_contrib_core 0.4.2, jupyter_contrib_nbextensions 0.7.0, jupyter_core 5.7.1, jupyter-events 0.9.0, jupyter-highlight-selected-word 0.2.0, jupyter-lsp 2.2.2, jupyter-nbextensions-configurator 0.6.3, jupyter_server 2.12.5, jupyter_server_terminals 0.5.2, jupyterlab 4.1.0, jupyterlab_pygments 0.3.0, jupyterlab_server 2.25.2, jupyterlab-widgets 3.0.9, keyring 23.5.0, launchpadlib 1.10.16, lazr.restfulclient 0.14.4, lazr.uri 1.0.6, locket 1.0.0, lxml 5.1.0, MarkupSafe 2.1.5, matplotlib-inline 0.1.6, mistune 3.0.2, more-itertools 8.10.0, mpmath 1.3.0, multidict 6.0.5, multiprocess 0.70.16, nbclassic 1.0.0, nbclient 0.9.0, nbconvert 7.14.2, nbformat 5.9.2, nest-asyncio 1.6.0, networkx 3.2.1, nltk 3.8.1, notebook 6.5.5, notebook_shim 0.2.3, numpy 1.26.3, nvidia-cublas-cu12 12.1.3.1, nvidia-cuda-cupti-cu12 12.1.105, nvidia-cuda-nvrtc-cu12 12.1.105, nvidia-cuda-runtime-cu12 12.1.105, nvidia-cudnn-cu12 8.9.2.26, nvidia-cufft-cu12 11.0.2.54, nvidia-curand-cu12 10.3.2.106, nvidia-cusolver-cu12 11.4.5.107, nvidia-cusparse-cu12 12.1.0.106, nvidia-nccl-cu12 2.19.3, nvidia-nvjitlink-cu12 12.3.101, nvidia-nvtx-cu12 12.1.105, oauthlib 3.2.0, overrides 7.7.0, packaging 23.2, pandas 2.2.2, pandocfilters 1.5.1, parso 0.8.3, partd 1.4.2, peft 0.11.1, pexpect 4.9.0, pillow 10.2.0, pip 24.1.2, platformdirs 4.2.0, pluggy 1.5.0, polars 1.1.0, prometheus-client 0.19.0, prompt-toolkit 3.0.43, protobuf 5.27.2, psutil 5.9.8, ptyprocess 0.7.0, pure-eval 0.2.2, pyarrow 16.1.0, pyarrow-hotfix 0.6, pyasn1 0.6.0, pycparser 2.21, Pygments 2.17.2, PyGObject 3.42.1, PyJWT 2.3.0, pyparsing 2.4.7, pytest 8.2.2, python-apt 2.4.0+ubuntu3, python-dateutil 2.8.2, python-json-logger 2.0.7, pytz 2024.1, PyYAML 6.0.1, pyzmq 24.0.1, referencing 0.33.0, regex 2024.5.15, requests 2.32.3, rfc3339-validator 0.1.4, rfc3986-validator 0.1.1, rpds-py 0.17.1, rsa 4.7.2, s3transfer 0.10.2, safetensors 0.4.3, scikit-learn 1.5.1, scipy 1.14.0, SecretStorage 3.3.1, Send2Trash 1.8.2, sentence-transformers 3.0.1, sentencepiece 0.2.0, setuptools 69.0.3, six 1.16.0, sniffio 1.3.0, soupsieve 2.5, stack-data 0.6.3, sympy 1.12, tabulate 0.9.0, terminado 0.18.0, threadpoolctl 3.5.0, tiktoken 0.7.0, tinycss2 1.2.1, tokenizers 0.19.1, tomli 2.0.1, toolz 0.12.1, torch 2.2.0, torchaudio 2.2.0, torchdata 0.7.1, torchtext 0.17.0, torchvision 0.17.0, tornado 6.4, tqdm 4.66.4, traitlets 5.14.1, transformers 4.42.4, triton 2.2.0, types-python-dateutil 2.8.19.20240106, typing_extensions 4.9.0, tzdata 2024.1, uri-template 1.3.0, urllib3 2.2.2, wadllib 1.3.6, wcwidth 0.2.13, webcolors 1.13, webencodings 0.5.1, websocket-client 1.7.0, wheel 0.42.0, widgetsnbextension 4.0.9, xxhash 3.4.1, yarl 1.9.4, zipp 1.0.0
hardware_type: 1 X NVIDIA L40S
hours_used: 0.15
cloud_provider: N/A
cloud_region: N/A
co2_emitted: N/A
direct_use: 
    ```python
    from transformers import pipeline

    pipe = pipeline("text-classification", model="2024-mcm-everitt-ryan/roberta-large-job-bias-seq-cls", return_all_scores=True)

    results = pipe("Join our dynamic and fast-paced team as a Junior Marketing Specialist. We seek a tech-savvy and energetic individual who thrives in a vibrant environment. Ideal candidates are digital natives with a fresh perspective, ready to adapt quickly to new trends. You should have recent experience in social media strategies and a strong understanding of current digital marketing tools. We're looking for someone with a youthful mindset, eager to bring innovative ideas to our young and ambitious team. If you're a recent graduate or early in your career, this opportunity is perfect for you!")
    print(results)
    ```
    >> [[
    {'label': 'age', 'score': 0.9883460402488708}, 
    {'label': 'disability', 'score': 0.00787709467113018}, 
    {'label': 'feminine', 'score': 0.007224376779049635}, 
    {'label': 'general', 'score': 0.09967829287052155}, 
    {'label': 'masculine', 'score': 0.0035264550242573023}, 
    {'label': 'racial', 'score': 0.014618005603551865}, 
    {'label': 'sexuality', 'score': 0.005568435415625572}
    ]]
    
model-index: name: roberta-large-job-bias-seq-cls
results: task: type: multi_label_classification
dataset: name: 2024-mcm-everitt-ryan/job-bias-synthetic-human-benchmark-v2
type: mix_human-eval_synthetic
metrics: type: loss
value: 0.2932254374027252
verified: no, type: accuracy
value: 0.5856164383561644
verified: no, type: f1_micro
value: 0.6382636655948553
verified: no, type: precision_micro
value: 0.6572847682119205
verified: no, type: recall_micro
value: 0.6203125
verified: no, type: roc_auc_micro
value: 0.7844866071428571
verified: no, type: f1_macro
value: 0.6412879991871913
verified: no, type: precision_macro
value: 0.7517582669787894
verified: no, type: recall_macro
value: 0.6203125
verified: no, type: roc_auc_macro
value: 0.7844866071428571
verified: no, type: f1_samples
value: 0.6374714611872145
verified: no, type: precision_samples
value: 0.6342751141552511
verified: no, type: recall_samples
value: 0.653681506849315
verified: no, type: roc_auc_samples
value: 0.8007868558382258
verified: no, type: f1_weighted
value: 0.6412879991871913
verified: no, type: precision_weighted
value: 0.7517582669787893
verified: no, type: recall_weighted
value: 0.6203125
verified: no, type: roc_auc_weighted
value: 0.7844866071428572
verified: no, type: runtime
value: 9.6046
verified: no, type: samples_per_second
value: 60.804
verified: no, type: steps_per_second
value: 7.601
verified: no, type: epoch
value: 3
verified: no
Likes: 0 | All-time downloads: 27
Created: 2024-07-17 01:28:33