Explorations on chaotic behaviors of Recurrent Neural Networks

Simple item page
dc.contributor.authorMyrzakhmetov, Bagdat
dc.contributor.editorAssylbekov, Zhenisbek
dc.contributor.editorTakhanov, Rustem
dc.contributor.otherTourassis, Vassilios D.
dc.date.accessioned2019-08-29T09:40:17Z
dc.date.available2019-08-29T09:40:17Z
dc.date.issued2019-04-29
dc.descriptionSubmitted to the Department of Mathematics on Apr 29, 2019, in partial fulfillment of the requirements for the degree of Master of Science in Applied Mathematicsen_US
dc.description.abstractIn this thesis work we analyzed the dynamics of the Recurrent Neural Network architectures. We explored the chaotic nature of state-of-the-art Recurrent Neural Networks: Vanilla Recurrent Network, Recurrent Highway Networks and Structurally Constrained Recurrent Network. Our experiments showed that they exhibit chaotic behavior in the absence of input data. We also proposed a way of removing chaos chaos from Recurrent Neural Networks. Our findings show that initialization of the weight matrices during the training plays an important role, as initialization with the matrices whose norm is smaller than one will lead to the non-chaotic behavior of the Recurrent Neural Networks. The advantage of the non-chaotic cells is stable dynamics. At the end, we tested our chaos-free version of the Recurrent Highway Networks (RHN) in a real-world application. In a sequence-to-sequence modeling experiments, particularly in the language modeling task, chaos-free version of RHN perform on par with the original version by using the same hyperparameters.en_US
dc.identifier.urihttp://nur.nu.edu.kz/handle/123456789/4197
dc.language.isoenen_US
dc.publisherNazarbayev University School of Science and Technologyen_US
dc.rightsAttribution-NonCommercial-ShareAlike 3.0 United States*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/3.0/us/*
dc.subjectResearch Subject Categories::MATHEMATICS::Applied mathematicsen_US
dc.titleExplorations on chaotic behaviors of Recurrent Neural Networksen_US
dc.typeMaster's thesisen_US
workflow.import.sourcescience

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
MYRZAKHMETOV, BAGDAT - Explorations on chaotic behaviors of Recurrent Neural Networks, 2019.pdf
Size:
1.86 MB
Format:
Adobe Portable Document Format
Description:
Thesis
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
6 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

02. Master's Thesis