Wider Vision: Enriching Convolutional Neural Networks via Alignment to External Knowledge Bases

March 12, 2021

Cite:

Liu, X., Delany, S.J., & McKeever, S. (2021). Wider vision: enriching convolutional neural networks via alignment to external knowledge bases. The 1st International Workshop on Machine Reasoning Co-located with WSDM 2021, Virtual Event, March 12, 2021.

Paper link: Here.

Currently, Deep Learning neural network models of Computer Vision are considered black boxes. There have been many research works about revealing what is inside the black boxes. An explanation of CNNs that could fully un-boxes a CNN model and explains the CNN features detected is still missing. This explanation will help other real-world problems, such as Visual Question Answering and Zero-shot Learning.

The intuition is to build a graph from CNN using visual features and to use Entity Alignment methods to map these features to an external Knowledge Graph. The corresponding nodes for a feature provides an explanation for that feature.

Results