The Indigenous Australians weren't miners, and parts of Australia have never been mined, so it would be perfectly possible to bury something in the past and be sure it wouldn't be found. Characters want to go somewhere, or somewhen, to carry out experiments or some kind of breeding program. Fugitives want somewhere to live undisturbed for a few months or years. Australia's your place again. The advantages of using these are that your readers may already have some familiarity with the places and events.
Documentation is likely to be quite good.
It's not at all unlikely that a contemporary time-traveller would know a fair bit about colonial days. Settings and situations to consider include the goldfields, the convict ships, early settlements, exploration and expeditions, drovers, the outback, or attack by bushrangers. In colonial days there was scope for new arrivals to come and go, and a time-travelling character could be readily assumed to have arrived from some other part of the colony. Convicts don't make convincing romance heroes, nor do their guards. One of the most difficult parts of writing that novel was coming up with a hero who would be a true man of his time yet with whom my modern heroine could plausibly fall in love.