Yes, you can definitely take your budgie outside in his cage. In fact, if you are with him throughout the time it is out, then it will surely enjoy the overall experience. But make sure not to leave him alone for too long or to stop monitoring his behavior.