  Result relevance scoring is critical to e-commerce search user experience.
Traditional information retrieval methods focus on keyword matching and
hand-crafted or counting-based numeric features, with limited understanding of
item semantic relevance. We describe a highly-scalable feed-forward neural
model to provide relevance score for (query, item) pairs, using only user query
and item title as features, and both user click feedback as well as limited
human ratings as labels. Several general enhancements were applied to further
optimize eval/test metrics, including Siamese pairwise architecture, random
batch negative co-training, and point-wise fine-tuning. We found significant
improvement over GBDT baseline as well as several off-the-shelf deep-learning
baselines on an independently constructed ratings dataset. The GBDT model
relies on 10 times more features. We also present metrics for select subset
combinations of techniques mentioned above.
