Murdoch University Research Repository

Welcome to the Murdoch University Research Repository

The Murdoch University Research Repository is an open access digital collection of research
created by Murdoch University staff, researchers and postgraduate students.

Learn more

Connection between BPTT and RTRL

Hanselmann, T., Zaknich, A. and Attikiouzel, Y. (1999) Connection between BPTT and RTRL. Computational Intelligence and Applications . pp. 97-102.

Free to read:
*No subscription required


This paper shows the connection between the Backpropagation Through Time (BPTT) algorithm, its truncated forms with truncation depth h, and the Recurrent Real Time Learning (RTRL) algorithm. The comparison is done by looking at a fully connected recurrent network, which is based on the same error function and calculations, using exact ordered derivatives. Two kind of formulas, based on total ordered derivatives, for BPTT(h) are given and proven to be equivalent. Of the two formulae, the second one, can be interpreted by a target modification in the case of h. Further, a combination of BPTT and RTRL is proposed to account for possible instabilities caused by weight adaptation. An overview of BPTT and RTRL and their implementations as well as their interpretations and uses are outlined.

Item Type: Journal Article
Publisher: World Scientific and Engineering Academy and Society
Item Control Page Item Control Page