Homemade tartar that has gone bad will have an off smell/taste/color. The relish, dill, and lemon juice will tinge the mayo a very light yellow. Any dark or orange/pink coloring is a good indication that something is off. When in doubt, check the inside of the lid of the storage container. If any bacterial growth is happening, it often first shows up on the lid. Some separation is natural in the tartar sauce, and that mainly comes from the lemon juice and onion. Stir prior to serving to combine.