As the 1st Avenue area declined in the 1940s and 1950s, the Eitel Building lost its ground-level retail tenants and was sold to various investment groups. The building's office tenants slowly left, leaving the upper floors completely unoccupied by 1978, as renovation to modern building code standards became too costly for landlords. Real estate investor Richard Nimmer bought the property in 1975, and pursued several unsuccessful attempts at renovating and selling the Eitel Building for various uses, including apartments, offices, and a hotel. The Eitel Building became regarded as an "eyesore", owing to its highly trafficked location near Pike Place Market, with boarded windows, a rotting roof, and a pigeon infestation. At one point, the city of Seattle threatened to condemn the property as part of their efforts to clean up the area. The building also suffered damage during the 2001 Nisqually earthquake, including the separation of a brick wall from the roof, which was repaired but dissuaded potential buyers.