Position-Aware Neural Attentive Graph Networks for Question Answering
Deep Learning Research Paper. We used Graph Neural Networks to solve a Multi-hop Question Answering task.
Paper
Position-Aware Neural Attentive Graph Networks for Question Answering (2021)
Authors: Ege Ersü, Yifu Qiu, Anda Zhou
Code
Abstract
Recently there has been considerable interest in applying Graph Neural Networks (GNN) to Multi-hop Question Answering (QA) tasks, as graph representations can explicitly express rich dependencies in language. However, graph representations suffer from the loss of sequential information and the difficulty of representing global semantic information. In this work, we propose the query-attention mechanism to enhance the GNN-QA system by utilizing both global and local contextual information. We also explore injecting the positional information into the graph to complement the sequential information. Our experiments are conducted on the WikiHop dataset to allow direction comparison with Entity Relational-Graph Convolutional Networks. Our contributions identify the existence of position bias in the dataset, and we further conduct ablation studies to confirm that our proposed modules improve the generalization accuracy by 1.43%.
Environment
- Python 3.6.13
- pytorch 1.7.1
- cudatookit 11.0.221
- scipy 1.5.2
- scikit-learn 0.24.1
- allennlp 0.9.0
- SpaCy 2.1.9
- tensorflow 1.13.1
- dgl 0.6.0
Datasets
Required Pretrained Models
How to run
- Step 1. Generation for the graph for train & dev set
- run
python entity_graph_gen.py --project-address --mode --number-of-data, e.g.,python entity_graph_gen.py --project-address=/path/to/file --graph-gen-mode=train --graph-gen-size=10 --dataset=medhopwill create 10 entity graphs from training set 1-10 samples in medhop.
- run
- Step 2. Train the model
- run
python train.py --project-address
- run