I speculate that God would withdraw His Spirit from them upon their death, but I am just speculating. All I know is that Paul seemed to make it clear that one has Christ's Spirit until they become reprobate. Since the Scripture teaches, and experience shows us that someone may lose faith in God for a time, and not become reprobate, I would have to say that while they are living in sin away from God, they still have God's Spirit. Part of my answer is based off of my understanding of salvation. I do not see salvation as a one-time event or experience.
Salvation is a process.
There is a sense in which we can say that we were saved in the past, that we are saved right now, and there is another sense in which we can say that we have not yet been saved because we are not yet with Jesus. No matter what one's spiritual condition is at any one point in their life, what matters most is how they finish. We must finish this race with our faith in Jesus. It is my belief that there are many backsliders who are still filled with God's Spirit, some of whom will eventually come back because God is continually working with their spirit, and some who will die in unbelief, and ultimately go to hell, and some whom have become reprobate, in which case God has already withdrawn His Spirit from them. I believe this last group is a small minority, however.