Question: are you perhaps short of a marble?
Answer: are you perhaps 'short of a marble' / way of describing someone as ignorant. No, it's about being crazy. Crazy people have "lost their marbles" so if Geralt is short of a marble... He's barmy.