Yeah, the crossbow is only useful for utility, not damage. There might be some mods that fix it, but you're better off using Grapeshot or Dancing Star if you need a damaging ranged attack.