The answer is nothing should happen for a little while, depending on the competitiveness of the niche or keyword. If you drop down a few positions we could analyse and tell you why should you re-start SEO.
If you have stopped SEO permanently, you may need to restart, especially if your competitors are doing SEO. Dropping 2-3 positions in Google can mean losing thousands in revenue!