To make poison arrows, open a Crafting Table and place a Lingering Potion of Poison in the middle of the grid, then put arrows in the remaining boxes.