Unfortunately, not. You are a resident of the country where you invested. However, if you have a Latvian residence permit, you can travel around the EU without restrictions and even stay in any of the Schengen countries for up to 90 days in any 6 months. But a residence permit does not give you the right to relocate and find a job in another EU country.
You can relocate to France or any other European country only if you have acquired Latvian citizenship which automatically means that you have become an EU citizen. However, you will need to live in the country for at least 10 years and learn the language in order to obtain a Latvian passport.