Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings

Abstract

This project presents that using a modified version of Skip-gram with negative sampling with weight tying proposed by Assylbekov and Takhanov (2019) word embedding of each word can be decomposed into two subvectors: x and y. We empirically showed that xsubvector is more related to the semantic role of the word and y-subvector is more related to the syntactic role of the word.