Besides the dangers of a potentially live bombs and weapons, and the dangers of getting a nasty scratch with rusty metal, there is also a danger of falling into the water. Always be super careful about it, especially if there are minors with you.
If you are fishing from a bridge, make sure the magnet does not attach itself to a metal beam or pole. You also do not want the magnet to attach itself to any moving object like watercraft or a propeller.
Remember that loadstones of such power can damage electronics, so keep them away from electronic devices.
Moreover, handle loadstones with caution, and never put them close to each other, because they can shatter from their pulling force. Never put your fingers or limbs between the loadstones for fishing.