If we speak statistically, then Japanese is ranked by the U.S. Foreign Services Institute as the most difficult language for native English speakers to learn. The institute uses the time it takes to learn a language to determine its difficulty 23-24 weeks for the easiest and 88 weeks for the hardest.
Though it purely depends on your will to learn and explore the language!