Kaiyrbekov, Temirlan2024-08-062024-08-062024-04-28Kaiyrbekov, T. (2024). Analysis of Maximum Mean Discrepancy Generative Adversarial Networks (MMD GAN). Nazarbayev University School of Sciences and Humanitieshttps://nur.nu.edu.kz/handle/123456789/8189Deep neural networks can be used to generate new data by sampling from the data distribution without explicitly defining the distribution. These nets heavy rely on optimization for efficient learning, and hence, they need mathematical guarantees for feasibility of learning. Generative Adversarial Networks (GAN) were proposed to generate images by the use of a mini-max objective function that is ”played” among two agents - a generator and a discriminator network. Later, Generative Moment Matching Networks (GMMN) were proposed to use a two-sample test instead of a discriminator network. GMMN uses Maximum Mean Discrepancy metric for distinguishing between real and generated images, but it only trains the generator network, and was implemented inefficiently. Lastly, Maximum Mean Discrepancy Generative Adversarial Networks (MMD GAN) were introduced that use adversarial kernel learning that has a mini-max objective function, efficient learning and mathematical guarantees that justify its improved performance. In this work, the mathematical reasoning behind the idea of MMD GAN was analyzed and experiments were made to tweak the parameters of the network. The loss function of MMD GAN is said to enjoy a weak topology - that MMD should tend to zero as two probability distributions converge to each other - and it will be shown empirically. Also, since the network has a loss function that is locally Lipschitz and continuous everywhere, and almost everywhere differentiable, the network was able to learn efficiently. Finally, MMD GAN with changed bandwidth parameters will be introduced that showed improved convergence with less MMD loss during training, although the loss was less smooth over epochs.enCC0 1.0 Universaldeep neural networksgeneratordiscriminatorkernelbandwidthmaximum mean discrepancyloss functionType of access: OpenCapstone project