Late persistent Lyme disease It is the last and often the most serious stage of the disease. Symptoms at this stage may include: Arthritis that most often affects the knee.