LoJexLLM

/

Unlimited-OCR-MLX

like 11

  I tested this model locally and the MLX inference output seems incorrect.

  The Java/HTTP wrapper was not the cause; direct calls to the model server returned wrong text. Examples:

  - UI screenshot -> `11 cott -Compatible`
  - Chinese table image -> `12 oppon १९-Compatible`
  - bundled demo image -> invalid text like `12 oppon iaz`

  I also found two likely bugs in `inference.py`:

  1. `model.generate()` returns prompt + generated tokens, but the code decodes the full sequence, causing `document parsing.` to leak into OCR output.

  Patch:

  ```python
  output_tokens = output_ids[0, len(extended_ids):].tolist()
  result = self.decode_text(output_tokens)

  2. The original image placeholder token is not removed after image feature slots are inserted:

  extended_ids = input_ids[:1] + [0] * total_image_feats + input_ids[1:]

  This leaves <｜place▁holder▁no▁0｜> in the prompt. I patched it to:

  extended_ids = input_ids[:1] + [0] * total_image_feats + input_ids[2:]

  Even after these fixes, the OCR output is still incorrect. A weight shape check also shows SAM relative position mismatches, e.g. checkpoint (27, 64) vs model (127, 64).

  Could you provide the expected prompt/image placeholder format and a known-good minimal inference script/output?