Abrosimov's picture

1

Abrosimov

ajiriro

·

AI & ML interests

None yet

Recent Activity

reacted to giux78's post with 👀 3 days ago

LLAMA4 release highlight the importance of political and social bias. According to their own evaluation described in the release blog post: - Refusals on contentious prompts dropped from 7% (hashtag#LLAMA 3.3) to under 2% - Unequal response refusals are now under 1% - Political lean bias is said to be halved compared to hashtag#LLaMA 3.3 and comparable to Grok However, we @efederici @mferraretto @FinancialSupport and I released some weeks ago an independent open source benchmark called Propaganda to measure political bias in LLMs: https://github.com/mii-llm/propaganda In the chart below, we evaluated multiple leading models on the basis of ratings across a range of prompts designed to expose ideological leanings. Despite Meta’s stated neutrality goals, LLAMA4 ranks at the very top in terms of total ratings aligned with a clear ideological bias. The models were tested on their ability to respond even-handedly to politically sensitive prompts. LLaMA 4 scored even higher than models known for strong alignment policies like GPT-4o. LLMs may be refusing less, but they still show bias through content framing. This suggests that refusal rates alone are not a sufficient measure of ideological bias. Relying solely on internal evaluations from AI labs also raises concerns about transparency and objectivity.

updated a collection 28 days ago

updated a collection 28 days ago

View all activity

Organizations

None yet

Collections 1

models

None public yet

datasets

None public yet