shamebear (shamebear) wrote in ai_research,

Reccurent Neural networks and AR processes

I am working with a recurrent neural network with one input and one output, faced with the task of predicting a relatively simple process. I am using linear nodes because this works well and because the sigmoid (tanh) nodes tended to just max out.

So far all is well, but then in (Ref 1) it is claimed that a linear recurrent network will only realize processes similar to the AR process. The reference says "AR-like" but are we talking equivalence here?

Formally: For a given trained recurrent linear network and a bounded interval for the input, do coefficients exist for an AR process so that the AR process will always give the same output as the network?

Ref 1: Page 22 of H. Jaeger, The echo state approach to training an analysing recurrent neural networks. (
  • Post a new comment


    default userpic