Implement fully-sparse inference (broken)

DO NOT MERGE
until tests pass.