Modelling Relation Paths with Generative Adversarial for Representation Learning of knowledge Bases
Publised in Proceedings of KR2ML Workshop, NeurIPS 2019, 33rd Conference on Neural Information Processing Systems, 9th-14th December 2019, Vancouver, Canada.
Enabling neural networks to perform multi-hop (mh) reasoning over knowledge bases (KBs) is vital for tasks such as question-answering and query expansion. Typically, recurrent neural networks (RNNs) trained with explicit objectives are used to model mh relation paths (mh-RPs). In this work, we hypothesize that explicit objectives are not the most effective strategy effective for learning mh-RNN reasoning models, proposing instead a generative adversarial network (GAN) based approach. The proposed model – mh Relation GAN (mh-RGAN) – consists of two networks; a generator $G$, and discriminator $D$. $G$ is tasked with composing a mh-RP and $D$ with discriminating between real and fake paths. During training, $G$ and $D$ contest each other adversarially as follows: $G$ attempts to fool $D$ by composing an indistinguishably invalid mh-RP given a head entity and a relation, while $D$ attempts to discriminate between valid and invalid reasoning chains until convergence. The resulting model is tested on benchmarks WordNet and FreeBase datasets and evaluated on the link prediction task using MRR and HIT@ 10, achieving best-in-class performance in all cases.