Initially, the were 180//3= <<180//3=60>>60 plums on the plum tree.
When Damien picked 3/5 of the apples from the apple tree, he picked 3/5*180 = <<3/5*180=108>>108 apples
The number of apples remaining on the apple tree is 180-108= <<180-108=72>>72
The number of plums that Damien picked from the plum tree is 3/5*60 = <<3/5*60=36>>36
The number of plums remaining on the plum tree is 60-36 = <<60-36=24>>24
The total number of fruits remaining on the trees is 72 apples+24 plums = <<72+24=96>>96
#### 96