Yes you can. You will want to use 4 mini marshmallows in place of 1 large marshmallow. Make sure you cut them in half so they stick to the graham crackers.