Taking money from your 401(k), either via a loan or withdrawal, doesn't affect your credit. What’s more, taking money from your IRA or other retirement accounts, has no bearing on your credit or credit score.