  Cross-modal retrieval is generally performed by projecting and aligning the
data from two different modalities onto a shared representation space. This
shared space often also acts as a bridge for translating the modalities. We
address the problem of learning such representation space by proposing and
exploiting the property of Discriminative Semantic Transitive Consistency --
ensuring that the data points are correctly classified even after being
transferred to the other modality. Along with semantic transitive consistency,
we also enforce the traditional distance minimizing constraint which makes the
projections of the corresponding data points from both the modalities to come
closer in the representation space. We analyze and compare the contribution of
both the loss terms and their interaction, for the task. In addition, we
incorporate semantic cycle-consistency for each of the modality. We empirically
demonstrate better performance owing to the different components with clear
ablation studies. We also provide qualitative results to support the proposals.
