Assuming that when predicting a certain central word, we happen to use the i-th vocabulary, then the (n, 1) dim matrix obtained by mapping this word through V is the embedding result of this word.
This matrix is called the input word matrix, we record it as V, shape=(n,|V|), where n represents the dimension of the word vector (embedding), and |V| represents the total number of vocabulary in the corpus. Then take these 2m words and map them through a parameter matrix. The first thing we need is the one-hot encoding of these 2m words. Suppose we need m words before and after the head word to predict the head word t. In general, CBOW predicts the head word by the words surrounding a head word. At the same time, it should be emphasized that no matter which model, although they all use neural networks, they can not be regarded as a deep neural network model, because in addition to input and output layers, they only use a hidden layer. The two main models are CBOW and skip-gram. By definition, word2vec is a combination of related models used to generate word vectors. The specific method used is Doc2vec, does it sound a bit like word2vec? This is indeed the case, because word2vec was also invented by these two, and the Doc2vec method also borrows word2vec to a large extent. In fact, they use a low-dimensional dense vector to represent a paragraph or sentence. Paragraph Vectorīased on the various drawbacks of BOW, the two big guys of Google proposed the Paragraph Vector in the 2014 paper–Distributed Representations of Sentences and Documents. The semantics of the first group of words are similar, and the second group is completely opposite, but through one-hot encoding. The semantic similarity of words cannot be reflected, such as'good' and'nice', and'good' and'bad'. Of course, the bag-of-n-gram-word model can moderately alleviate this problem, but at the same time it brings back high-dimensional solving problems and the feature matrix is too sparse (very many 0).Ģ. For example, the vectors of "I love you" and "You love me" in BOW are exactly the same, which is obviously not reasonable. The word order in the text cannot be reflected. It is undeniable that this is a very direct form of presentation, but this This method has two very big problems.ġ. We know that one of the most direct ways of presenting words is one-hot encoding, and the bag of words model (BOW) based on this in text modeling.
0 kommentar(er)
