Temporal lobe epilepsy (TLE) is the most prevalent form of drug-resistant epilepsy with concurrent cognitive impairment. Prevention, earlier diagnosis, and personalized management of cognitive deficits in TLE require more understanding of underlying structural and functional brain Ialterations. No study has evaluated the performance of TLE patients in different cognitive domains based on their structural brain lesions. In this study, 69 refractory TLE patients underwent magnetic resonance imaging (MRI) epilepsy protocol and several neuropsychological tests, consisting of the Wechsler adult intelligence scale-revised, Rey-Osterrieth complex figure test, verbal fluency test, digit span test, spatial span test, Wechsler memory scale-III, design fluency test, Rey visual design learning test, auditory-verbal learning test, and trail making test. MRI findings were classified into the following groups: Focal cortical dysplasia, gliosis, atrophy, mesial temporal sclerosis (MTS), tumor, vascular malformation, and other lesions or normal. Results of neuropsychological tests were compared between MRI groups using a generalized linear model with gamma distribution and log link. Patients with MTS showed better performance in general intellectual functioning, working memory, attentional span, and auditory-verbal learning compared to patients with non-MTS MRI lesions. Atrophy and focal cortical dysplasia had the largest differences from MTS. Cognitive performance of refractory TLE patients varies concerning structural brain alterations. Further neuroimaging studies of TLE lead to prevention and more accurate management of cognitive decline in clinical settings. Cognitive status in temporal lobe epilepsy (TLE) varies concerning structural brain alterations.Patients with mesial temporal sclerosis (MTS) show better cognitive performance than those with non-MTS lesions.Among non-MTS findings, patients with atrophy have more severe cognitive deficits. Temporal lobe epilepsy (TLE) is the most common form of epilepsy which does not respond to anti-seizure drugs and needs surgery of the brain lesions. One of the most important issues of TLE patients is their cognitive impairment. Cognition refers to the mental processes for thinking, understanding, and perception of the environment such as attention, memory, learning, language, etc. Prevention, earlier diagnosis, and treatment of cognitive deficits in TLE patients need more understanding of their brain changes. No study has evaluated the cognition of TLE patients in detail based on their brain lesions. In this study, 69 drug-resistant TLE patients have undergone brain magnetic resonance imaging (MRI) and several neuropsychological tests that assess cognition, consisting of the Wechsler adult intelligence scale-revised, Rey-Osterrieth complex figure test, verbal fluency test, digit span test, spatial span test, Wechsler memory scale-III, design fluency test, Rey visual design learning test, auditory-verbal learning test, and trail making test. MRI findings were classified into the following groups based on the type of brain lesion by an expert: Focal cortical dysplasia, gliosis, atrophy, mesial temporal sclerosis (MTS), tumor, vascular malformation, and other lesions or normal. Results of neuropsychological tests were compared between MRI groups using appropriate statistical methods. Patients with MTS, as the most common lesion in TLE, showed better results compared to patients with lesions other than MTS in intelligence, memory, attention, and learning tests. Patients with atrophy and focal cortical dysplasia had the largest differences from those with MTS. These results suggest that the cognitive performance of drug-resistant TLE patients is different based on their structural brain changes. As imaging, in particular brain MRI, is the most available technique in the clinic for the assessment of epilepsy, further brain imaging studies can lead to prevention and better management of cognitive decline in TLE.