By the end of the 16th century, Portugal had lost its independence following the events of Alcácer Quibir. The region and village of Loulé was, once again, threatened by coastal attacks from pirates along the coast, yet the castle defenses had already become threatened and had fallen into ruin. After 1630, the economic vitality of the Algarve began to fall, its populations stagnating and political instability contributing to Barbary Coast piracy. The regions expansion suffered, even as new buildings were being constructed: in 1600, the Church of Portas do Ceu was constructed; in the three of the gates of the castle they constructed hermitages to Nossa Senhora da Conceição, Nossa Senhora do Pilar and Nossa Senhora do Carmo; in the courtyard, they constructed the Church of Espírito Santo (then recovered and expanded in 1693); expanded the nunnery of Convent of Nossa Senhora da Conceição; and in the east, D. Francisco Barreto, second bishop of the Algarve, laid the cornerstone of the Convent of Santo António (1675). After the Restoration, the castle took on a defensive role and lands surrounding the structure were occupied by new construction.