No, having an accent will not going to be a problem for you in the IELTS Speaking test. The examiner will not evaluate you or judge you on the basis of your accent unless it causes problems with pronunciation and clarity of speech, making it difficult for the examiner to understand what you speak. If in any case, you have a strong accent, then that will be an issue and will probably result in a low IELTS Speaking band score.