No, they are still your neighbor's plants and responsibility (which means, of course, they can neglect them if they choose): they are on your neighbor's property and were planted by the home's previous owner. Someone cannot make part of their property your responsibility by simply excluding it outside a fence.