Yes, it is against the rules to refuse to roll the dice. It is considered poor craps etiquette to do so.