Sorry yeah this is fine what is happening.  I forgot that the spatialite provider creates its own connection so in memory db wouldn't work.