*= Equal Contributors
Advice programs in large-scale on-line marketplaces are important to aiding customers in discovering new content material. Nonetheless, state-of-the-art programs for item-to-item advice duties are sometimes based mostly on a shallow stage of contextual relevance, which might make the system inadequate for duties the place merchandise relationships are extra nuanced. Contextually related merchandise pairs can generally have problematic relationships which might be complicated and even controversial to finish customers, and so they may degrade person experiences and model notion when advisable to customers. For instance, the advice of a guide about one sports activities workforce to somebody studying a guide about that workforce’s largest rival might be a nasty expertise, regardless of the presumed similarities of the books. On this paper, we suggest a classifier to establish and forestall such problematic item-to-item suggestions and to reinforce general person experiences. The proposed strategy makes use of energetic studying to pattern onerous examples successfully throughout delicate merchandise classes and employs human raters for knowledge labeling. We additionally carry out offline experiments to exhibit the efficacy of this technique for figuring out and filtering problematic suggestions whereas sustaining advice high quality.