DSpace Repository

Explorations on chaotic behaviors of Recurrent Neural Networks

Show simple item record

dc.contributor.author Myrzakhmetov, Bagdat
dc.contributor.editor Assylbekov, Zhenisbek
dc.contributor.editor Takhanov, Rustem
dc.contributor.other Tourassis, Vassilios D.
dc.date.accessioned 2019-08-29T09:40:17Z
dc.date.available 2019-08-29T09:40:17Z
dc.date.issued 2019-04-29
dc.identifier.uri http://nur.nu.edu.kz/handle/123456789/4197
dc.description Submitted to the Department of Mathematics on Apr 29, 2019, in partial fulfillment of the requirements for the degree of Master of Science in Applied Mathematics en_US
dc.description.abstract In this thesis work we analyzed the dynamics of the Recurrent Neural Network architectures. We explored the chaotic nature of state-of-the-art Recurrent Neural Networks: Vanilla Recurrent Network, Recurrent Highway Networks and Structurally Constrained Recurrent Network. Our experiments showed that they exhibit chaotic behavior in the absence of input data. We also proposed a way of removing chaos chaos from Recurrent Neural Networks. Our findings show that initialization of the weight matrices during the training plays an important role, as initialization with the matrices whose norm is smaller than one will lead to the non-chaotic behavior of the Recurrent Neural Networks. The advantage of the non-chaotic cells is stable dynamics. At the end, we tested our chaos-free version of the Recurrent Highway Networks (RHN) in a real-world application. In a sequence-to-sequence modeling experiments, particularly in the language modeling task, chaos-free version of RHN perform on par with the original version by using the same hyperparameters. en_US
dc.language.iso en en_US
dc.publisher Nazarbayev University School of Science and Technology en_US
dc.rights Attribution-NonCommercial-ShareAlike 3.0 United States *
dc.rights.uri http://creativecommons.org/licenses/by-nc-sa/3.0/us/ *
dc.subject Research Subject Categories::MATHEMATICS::Applied mathematics en_US
dc.title Explorations on chaotic behaviors of Recurrent Neural Networks en_US
dc.type Master's thesis en_US
workflow.import.source science


Files in this item

The following license files are associated with this item:

This item appears in the following Collection(s)

Show simple item record

Attribution-NonCommercial-ShareAlike 3.0 United States Except where otherwise noted, this item's license is described as Attribution-NonCommercial-ShareAlike 3.0 United States

Video Guide

Submission guideSubmission guide

Submit your materials for publication to

NU Repository Drive

Browse

My Account

Statistics