So far all is well, but then in (Ref 1) it is claimed that a linear recurrent network will only realize processes similar to the AR process. The reference says "AR-like" but are we talking equivalence here?
Formally: For a given trained recurrent linear network and a bounded interval for the input, do coefficients exist for an AR process so that the AR process will always give the same output as the network?
Ref 1: Page 22 of H. Jaeger, The echo state approach to training an analysing recurrent neural networks. (http://www.faculty.iu-bremen.de/hjaege