Four Strategies of Personalized Recommendation Algorithm 05

The first two terms are the logistic regression (LR) model, and the third term is the second-order feature combination, which means th

Four Strategies of Personalized Recommendation Algorithm 05

The first two terms are the logistic regression (LR) model, and the third term is the second-order feature combination, which means that the model needs to learn a low-dimensional hidden vector representation V (that is, the vector representation in the deep neural network) for each input feature.

Replace the original w{ij} with the dot product of V{i} and V{j}, where V{i} represents the hidden vector expression of feature x{i} and V{j} represents the hidden vector expression of feature x{j}, and the weight of feature combination is reflected by the hidden vector inner product of each single feature.

However, what's wrong with this feature intersection? Take the users of 58 city APP local service car rental business as an example. According to the portrait of platform users, there will be characteristics such as gender, preferred model and preferred brand.

However, when a feature is combined with other features in the second order, it may share the same feature vector * * * For example, gender and preferred car combination, gender and preferred brand combination * * * share the same feature vector, which makes the expression ability of the model very limited.

Based on the factorization machine (FM) model, we introduce the concept of field, thus forming a new model-FFM (field factorization machine).

Compared with FM, FFM introduces the concept of field, which makes the model produce more auxiliary information and has stronger expressive ability. At this point, FM can be regarded as FFM with only one domain.

The main manual combination features of logistic regression (LR) model. In order to solve the shortage of artificial experience, effective features can be automatically found and combined, thus shortening the experimental period of feature mining, so high-dimensional complex models such as factor decomposition machine (FM) are derived.

FM model mainly looks for the combination relationship between pairwise features by means of hidden variables, but this combination of features is limited to pairwise features. Thus, a more complex combination model-GBDT model was born, which is mainly used to solve the problem of high-dimensional feature combination.

Gradient lifting decision tree (GBDT) model is one of the most powerful algorithms in traditional machine learning. It uses linear combination based on function to realize multiple iterations, and generates a weak classifier after each iteration. Finally, each classifier is trained according to the residual of the previous classifier.

Generally speaking, ensemble learning will choose Categorical Regression Tree (CART) as a weak classifier, and the depth of each Categorical Regression Tree will not be very deep. Finally, the general classifier will sum the weak classifiers obtained from each round of training to get a result, and the specific expression is as follows:

The GBDT+LR model uses GBDT to automatically screen and combine features, and then generates a new discrete feature vector, which is then used as the input of the logistic regression (LR) model, and finally produces a prediction result.

Generally speaking, the logistic regression (LR) model itself is more suitable for processing linear separable data than combining nonlinear data.

If we want the logistic regression (LR) model to process the fitted nonlinear data, we can use GBDT to generate the combination of features. Because the model has the overall fitting ability to nonlinear distribution data, it can automatically and efficiently find effective feature combinations through the algorithm, resulting in more than third-order feature combination results, while FM and FFM obviously do not have such strong feature combination ability.