Olive trees can be affected by pests such as the olive fruit fly, scale insects, and aphids. These pests can cause damage to the leaves and fruit, which may lead to browning leaves.