They hate cleaning up after them or fear they will fall. Listen respectfully to your neighbor because they might have a well founded request. Call a certified arborist and get good, solid information about your tree's health. Don't guess or be swayed by emotion alone. If they are threatening a law suit call your lawyer and find out your rights.
Laws pertaining to trees are different in each state.
Take this as an opportunity to become aquatinted with your local laws. But again, call in knowledgeable help. Don't depend on a friends advice unless they are a lawyer. Firewood is needed. People do take trees down for firewood.