Catalog Home Page

Connection between BPTT and RTRL

Hanselmann, T., Zaknich, A. and Attikiouzel, Y. (1999) Connection between BPTT and RTRL. Computational Intelligence and Applications . pp. 97-102.

Link to Open Access version:
*No subscription required


This paper shows the connection between the Backpropagation Through Time (BPTT) algorithm, its truncated forms with truncation depth h, and the Recurrent Real Time Learning (RTRL) algorithm. The comparison is done by looking at a fully connected recurrent network, which is based on the same error function and calculations, using exact ordered derivatives. Two kind of formulas, based on total ordered derivatives, for BPTT(h) are given and proven to be equivalent. Of the two formulae, the second one, can be interpreted by a target modification in the case of h. Further, a combination of BPTT and RTRL is proposed to account for possible instabilities caused by weight adaptation. An overview of BPTT and RTRL and their implementations as well as their interpretations and uses are outlined.

Publication Type: Journal Article
Publisher: World Scientific and Engineering Academy and Society
Item Control Page Item Control Page