  Most existing set encoding algorithms operate under the assumption that all
the elements of the set are accessible during training and inference.
Additionally, it is assumed that there are enough computational resources
available for concurrently processing sets of large cardinality. However, both
assumptions fail when the cardinality of the set is prohibitively large such
that we cannot even load the set into memory. In more extreme cases, the set
size could be potentially unlimited, and the elements of the set could be given
in a streaming manner, where the model receives subsets of the full set data at
irregular intervals. To tackle such practical challenges in large-scale set
encoding, we go beyond the usual constraints of invariance and equivariance and
introduce a new property termed Mini-Batch Consistency that is required for
large scale mini-batch set encoding. We present a scalable and efficient set
encoding mechanism that is amenable to mini-batch processing with respect to
set elements and capable of updating set representations as more data arrives.
The proposed method respects the required symmetries of invariance and
equivariance as well as being Mini-Batch Consistent for random partitions of
the input set. We perform extensive experiments and show that our method is
computationally efficient and results in rich set encoding representations for
set-structured data.
