Still thinks loudly even if disabled
I'm using litert-lm on 0.16.0 android. It thinks loudly (as a regular output):
val extraContext = if (enableThinking)
mapOf("enable_thinking" to "true")
else
mapOf("enable_thinking" to "false")
conversation.sendMessageAsync(
Contents.of(contents),
object : MessageCallback {
override fun onMessage(message: Message) {
resultListener(message.toString(), false, message.channels["thought"])
}
override fun onDone() {
resultListener("", true, null)
}
override fun onError(throwable: Throwable) {
if (throwable is CancellationException) {
logger.warn("The inference is cancelled.")
resultListener("", true, null)
} else {
logger.error("onError", throwable)
onError(throwable.message)
}
}
},
extraContext ?: emptyMap(),
)
I've also tried mapOf("enable_thinking" to false) as written here: https://github.com/google-ai-edge/LiteRT-LM/issues/1426#issuecomment-4859851237
Can you please provide me the model link you used ?
Correct one is-https://huggingface.co/litert-community/Qwen3-1.7B/blob/main/Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm
That Make sense! The prompt template issue in https://huggingface.co/litert-community/Qwen3-1.7B/blob/main/Qwen3_1.7B.litertlm was causing the thinking channel flags to be ignored. We have fixed this in https://huggingface.co/litert-community/Qwen3-1.7B/blob/main/Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm.
Please try using that model version and let us know if you still face any issues!
In addition, you can use the ThinkingConfig(enableThinking = enable) API. https://github.com/google-ai-edge/LiteRT-LM/blob/5c490382d7b2257bf39d31074f46561b6e1ac613/kotlin/java/com/google/ai/edge/litertlm/Config.kt#L132
@whhone I did not know that, thanks!
Here and below - i'm using and passing ThinkingConfig as @whhone suggested.
@VinodMamilla i've did the testing:
- on a JVM:
- Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm is NOT printing thinking output even if enabled - NOT as expected
- Qwen3_1.7B.litertlm DOES print if enabled and does NOT if disabled - as expected
- on Android device (Xiaomi, 8Gb, Qualcomm):
- Qwen3-1.7B_dynamic_wi4b32_afp32.litertlm is crashing the system
- Qwen3_1.7B.litertlm DOES print if enabled and does NOT if disabled - as expected. However the
Messagedoes not have "thought" channel and prints it as a regular output
@VinodMamilla Is it aligned with what you expect?
"<think>" chunk on Qwen is hardcoded, so there is no solution UNLESS you change the template on native layer.(I'm doing