You can take metals possession without penalty after the age of 59 1/2 or more.
If you withdraw before this age, you may have to pay a 10% penalty unless you have a first-time home purchase or have to pay for college.
For traditional Gold IRAs, you have to be more than 70 1/2 to begin the withdrawals. For Roth Gold, you can start the withdrawals after 5 years for tax-free and penalty-free possessions.