Treatment with an insecticide for codling moth management may be necessary. If you decide to use an insecticide, choose a product labeled for controlling codling moth in apple trees and carefully follow the directions. It is important that you provide irrigation to the tree where the lawn was removed. The tree developed roots in the lawn area because there was water there to maintain the lawn. Although the pea gravel will serve as a mulch to reduce evaporation of water from the soil, irrigation will be necessary.
Insufficient water can cause fruit drop.
Honey bee colonies have been greatly reduced by mite infestations. This would result in the loss of large numbers of small fruit early in the growing season. Another factor may be the shade from nearby trees. Apple trees need several hours of direct sunlight each day. If the surrounding trees that are shading the gravel are also shading the tree, the competition for light could be causing the loss of fruit.