Yes, Selfoss city has many grocery stores; some of the popular ones include:
Bónus
Krónan
Samkaup Úrval Selfoss