Yes, the IRS considers any second home an investment property if you spend less than two weeks staying in it, and then rent it for the remainder of the year. Depending on how long you have owned the home, the amount of capital gains taxes owed may differ.