Yes, you can connect solar panels directly to a battery. But this can be dangerous if the voltages don’t match. If a sudden surge of power runs through the circuit, your battery system might not have enough space for it, and it will overcharge.
Using an inverter is always a safer option to safely manage the flow of current.