As a fluent Mandarin speaker that has only memorized a handful of these idioms, I would focus more on memorizing general vocabulary. You should think of knowing chengyu as a “nice to have” skill whereas expanding your vocabulary is essential.