Hugging Face's logo

google
/
gemma-4-E2B-it

Any-to-Any
Transformers
Safetensors
gemma4
image-text-to-text
Eval Results
Model card Files
xet
 Community
47

Weak at generating structured output.

#47
by Duonglv - opened
Discussion
Duonglv

Hello Google team,

This model is pretty weak when I requires it generate structured output, like json one.
Hope you guys will improve it in the gemma5.

Thank a lot.

pannaga10
 Google org

Hi @Duonglv
Thanks for sharing this feedback. I would like to reproduce it on my end. Could you please share a minimal reproducible example with your prompt, system message, generation parameters, inference library and the expected vs. actual output ?
Thanks
Duonglv
•
edited 3 days ago

@panaga10
I use LangChain with response_format (https://docs.langchain.com/oss/python/langchain/structured-output).

Basically, I use the same system prompt, output structure, etc., and only switch between models, such as Gemma 4 E2B, E4B, and 12B.

  • The E2B model often fails to generate the structured output, and many APIs are called (through vLLM) before the correct output is generated.
  • The E4B is much better.
  • The 12B rarely fails.

I can’t share my code or data, sorry about that.

I’m looking for a truly small model (under 4B) for an assistant, so I have tried the models above. I see that E2B is very fast, but it’s pretty weak when searching web content (finding data from web pages). E4B is better, while 12B is strong enough.

Sign up or log in to comment