Murdoch University Research Repository

Welcome to the Murdoch University Research Repository

The Murdoch University Research Repository is an open access digital collection of research
created by Murdoch University staff, researchers and postgraduate students.

Learn more

Using an effective Boltzmann machine to learn context dependencies of a sequence

Bellgard, M.I. and Tsang, C.P. (1995) Using an effective Boltzmann machine to learn context dependencies of a sequence. In: 1995 IEEE International Conference on Neural Networks (ICNN '95), 27 November - 1 December 1995, Perth, W.A pp. 2841-2846.

PDF - Published Version
Download (515kB)
Link to Published Version:
*Subscription may be required


It has been recognised that current Artificial Neural Network (ANN) systems that employ windowing techniques to learn context dependencies in sequences have many deficiencies. One variant of this type of system that attempts to overcome many of these deficiencies is the Effective Boltzmann Machine(EBM). The EBM which is based on the Boltzmann Machine (BM) has the ability to perform completion and to provide an energy measure for the solution. In this paper, we extend the EBM and show that the system has many desirable properties. This paper reports two major improvements to the EBM First, whereas in the past, a BM is used to learn the local contexts of a sequence, we show that the EBM itself is an architecture suitable for learning local contexts. Our initial experiments show that it outperforms a window based BA4 Second, we demonstrate that, with this new training scheme, a multilayer EBM can be constructed to extend the effective window size. This means that learning contextual dependencies is not limited by the window size. This is done by utilising the state information of the first EBM hidden layer to train that of the second EBM layer. The effect of this long range effective window is demonstrated by experiments.

Item Type: Conference Paper
Murdoch Affiliation(s): School of Mathematical and Physical Sciences
Item Control Page Item Control Page


Downloads per month over past year