The minister insists all this was a matter of poor administration of his campaign - but that may raise a question about the competence of a man responsible for running two big Whitehall departments , work and pensions and the Welsh office .