Lu et al: Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models
Finds that there is a structure they call “influence paths”, that exists between gates in a LSTM that incorporates how different words relate to one another, such as the number agreement task