Quantum probability theory in cognitive science applies quantum principles such as superposition and contextuality to understand complex human cognitive phenomena, providing a nuanced view of decision-making and judgment.