The "Raffles" stories have two distinct phases. In the first phase, Raffles and Bunny are men-about-town who also commit burglaries. Raffles is a famous gentleman cricketer, a marvellous spin bowler who is often invited to social events that would be out of his reach otherwise. "I was asked about for my cricket", he comments after this period is over. It ends when they are caught and exposed on an ocean voyage while attempting another theft; Raffles dives overboard and is presumed drowned. These stories were collected in The Amateur Cracksman. Other stories set in this period, written after Raffles had been "killed off", were collected in A Thief in the Night.
The second phase begins some time later when Bunny – having served a prison sentence – is summoned to the house of a rich invalid. This turns out to be Raffles himself, back in England in disguise. Then begins their "professional" period, exiled from Society, in which they are straightforward thieves trying to earn a living while keeping Raffles's identity a secret. They finally volunteer for the Boer War, where Bunny is wounded and Raffles dies in battle after exposing an enemy spy. These stories were originally collected in The Black Mask, although they were subsequently published in one volume with the phase one stories. The last few stories in A Thief in the Night were set during this period as well.
Like Sherlock Holmes after his disappearance into the Reichenbach Falls, Raffles was never quite the same after his reappearance. The "classic" Raffles elements are all found in the first stories: cricket, high society, West End clubs, Bond Street jewellers – and two men in immaculate evening dress pulling off impossible robberies.

Question: Where was Bunny before being summoned to the house of a rich invalid?
Answer: prison
Alternative answer: Prison