Scalability in E-commerce websites is handled by designing a flexible architecture that can accommodate increased traffic and data. This includes using scalable hosting solutions, optimizing database performance, and implementing caching techniques. Additionally, regularly reviewing and updating the infrastructure can help manage growth effectively.