It is possible to make croutons without butter or oil, but they may not be as crispy or flavorful. You can try using a non-stick cooking spray or simply bake the bread cubes without any coating, but the texture and taste may be different.