The largest mapped sea cave found so far is Painted Cave on Santa Cruz Island. At 374 meters it is the largest known sea cave in the world. It is possible to kayak all the way back to the Inner Chamber which is home to a large colony of sea lions. Seal Canyon Cave and Catheral Cove caves are also good kayaking sea caves to explore. Remember that sea caves can be populated by marine mammals and many other organisms.
Sea caves can be quite 'sporting' when large sets of waves come rolling in.
These caves should only be entered in calm seas. Low tide is another factor to consider . Anacapa is probably the easiest island on which to do some kayaking. Best bet is to go out with the Park's concessionaire, Island Packers in Ventura. You can have them drop you off and then camp out at the island for up to 5 nights - you just need a permit from the park.