Use a chisel to carefully pry out the frame. Focus on one side of the window frame at a time. Make sure you don't damage your walls.