To get to the Zodiac Village, you must be in Chapter 3. Go to the North Shore of the Bitter Lake Keeper’s Shrine. Follow the shoreline until you meet Chen Loong, a dragon NPC.Defeat Chen Loong in battle. Then, complete his quest to get the Ruyi Scroll. This scroll will teleport you to the Zodiac Village.