If spirit bodies, necessary for the transmission of past life memories in reincarnation, can retain memory while the body dies and decomposes, they should be able to retain memory when the brain suffers trauma, and the body lives. In effect, with the spirit's memory as back-up for the brain's memory, there should never be any loss of memory from brain trauma, yet there is. The absence of memory after brain trauma seems to indicate there is not. Imagine, also, the memory of a past life being transcribed to a new being conceived of egg and sperm. At that point, the new being does not even have a brain with neurons that can create neuron relationships.
Complex information needs a complex vehicle.
An adult brain is needed to house adult memories. Some people feel that spirits of the dead exist and can be contacted by the living. Around 1848, two sisters, Kate and Margaret Fox, began to hold seances in which people sat around a table and the sisters acted as mediums between the visitors and spirits of the dead. There was money to be made in giving seances, connecting people with their ancestors, so this was like lighting a prairie fire. Soon, everyone was holding seances.