The issue at hand can be abstracted to flow in a network, and so maximizing utility is all about maximizing and exploiting bandwidth in the network. In an area with a complex geography like the Bay Area (and also in many if not most urban areas) rail-based systems will not get us to the sweet spot. Throughout my lifetime I have heard lots of rail good / automobiles bad moralising without any real thought behind it. (Below I am not going to argue any green issues. I think they are eventually solved and so irrelevant.) Almost all rail systems are direct descendants of 19th century technology when roads were too primitive and the mechanical motive systems bulky and most efficient at a relatively large scale. Autonomous vehicles (automobiles) became practical and ubiquitous with better roads.

Fixed systems never get you from where you are to where you want to be. Have you ever just happened to be at a BART station and decided I want to go to this other BART station? I doubt it. (OK, in high-density urban situations this can happen.) Changing transportation modes is a real joy killer.

The problem to solve with autonomous vehicles is a packing problem. Better packing while in motion (driving) and storage (parking). I submit this is a much easier set of problems to deal with given the technology we have and that which is on the horizon than getting fixed systems to transport you between arbitrary points A and B.

I'm a life long Bay Arean, and I'm familiar with all the major corridors and transportation modes. Here is my boiled down list of the some of the bigger issues:

1) BART is maxed-out. The bandwidth on the stretch from Millbrae to West Oakland effectively constrains the capacity of the entire system. The cars are too small and uncomfortable when overcrowded. The station size limits the number of cars to 10. Tunnel overhead prevents double-decker cars.

2) Bandwidth utilization on the Caltrain line is laughably small. There are still many grade crossings which snarl surface traffic at commute time when the occasional train does pass through.

3) The tie-ins of the east-west corridors to the north-south corridors are inefficient and cause some of the biggest back-ups. (237, 238, Dumbarton Bridge, San Mateo Bridge, Bay Bridge. The tie-in of the San Rafael Bridge is the happy exception, backups on 101 and 80 having more to do with maxing-out the local bandwidth. The Bay Bridge bandwidth itself is pretty much maxed-out.

So how do we solve the autonomous vehicle packing problems?

1) Self-driving (truly autonomous) cars will eventually allow platooning and so better packing while in motion. This should also lead to fewer fender-benders at commute time.

2) Self-driving should also lead to better packing in storage (parking).

3) Better monitoring of vehicle systems to prevent breakdowns from stalling traffic.

4) More capacity at critical network connection points (intersections). There is still room for auxillary lanes in many cases around the east-west connections I mentioned above and elsewhere. Caltrans has in some cases intentionally created choke points to meter traffic.

5) Can we hurry up with the current construction? I think it is a cost saving move to stretch out improvement projects, but is the savings worth the delay?

6) With self-driving and better monitoring of vehicle systems it is possible to make the rail corridors dual-purpose. In other words pack autonomous vehicles into the space between trains. That will require a sophisticated traffic control system and access ramps at stations (expensive to build at underground stations). Think of packing vehicles with the form factor of a Smart Car on a road surface build around the tracks. How much would that increase bandwidth? The Caltrain corridor alone could contain the equivalent of 2 lanes of traffic in each direction, possibly 3 if you really packed it. There are also underutilized Union Pacific / Amtrak lines in the East Bay, not to mention increasing the bandwidth of BART.

7) There is also room for better separation of vehicles and pedestrians and bicycles. This problem is still expensive, but solvable.