Sentence: I sang a song with Mary while you did so with Bill.

Linguistic integrity: acceptable

Sentence: That Jack sometimes slept is impossible.

Linguistic integrity: acceptable

Sentence: is out since the following is finite.
Linguistic integrity: acceptable