1) If someone were to fall on property that is ostensibly our neighbor's but is actually yours and ownership were to be discovered, you could potentially be sued for their injury: a property owner can be held liable for injuries occuring on his/her property. While you might escape liability by showing you had no control over what happened, and therefore no fault, at the least you'd have to go to the trouble and expense of defending yourself--and there is no guaranty you'd win.2) If part of your house is on a neigbor's property, the neighbor could potentially force you to remove (demolish) that part of your house.3) You could force the neighbor to remove the shed and driveway.One way to resolve this situation would be for you and your neighbor to each deed each other the relevant pieces of land, so everyone owns the land with their property on it. You are advised to consult with a real estate attorney.