Please refer to previous blog for data acquisition and processing as well as sever and DBMS configuration.
For the recommender developing, vector space model is initially applied to calculate similarity between any two movies based on movie overview. We also introduce movie popularity to refine the result ranking. Html/CSS, php and python are used for developing the recommender system. Procedure are listed as following,
Since this recommender is developed depended on LAMP, runing python directly on Apache is not a good choice. We use an intermediate php script to call the python script running which is similiar to run python through bash.
Since phase one have developed tf-idf algorithm by php, in this phase, we build the tf-idf algorithm by python and retrieve top 10 movies which are similar to the movie user input. We adjust the top 10 movie rank by introducing popularity, this is because we think a similar movie might not be the most of interested to the user, a popular and at the same time a similar movie may be a better choice compare to similar one solely.
Scripts on Github: GitHub
Content-based recommender suggest similar items based on a particular item. This system uses item features, such as genre, director, description, actors, etc. for movies, to make recommendations. The general idea behind these recommender systems is that if a person likes a particular item, he or she will also like an item that is similar to it. In our dataset, we have overview to describe each of movies, we will compute TF-IDF vectors for each movie based on overview. This will give us a matrix where each column represents a word in the overview vocabulary and each row represents a movie. By the vectors we calculated, we can have pairwise similarity scores for all movies and recommend movies based on that similarity score. Mathematically, the similarity scores is defined as follows: The tf-idf algorithm explanation in detail can be found in phase 1 blog. We can further refined the recommender system by introducing more features to get a better ranking mechanism. Considering users may be not only interested in similar movies, but a popular movie along with similarity may be the movie they are most interested in. Take this into consideration, we refine the ranking mechanism by introducing popularity feature to re-rank the top ten movie which are retrieved by similarity scores of tf-idf. By observing distribution of popularity value, the popularity value take down as 1% of their original values. And contribution to 20% of the whole score, the similarity scores recalculate as 80% accordingly. The final ranking scores is defined as follows: The movie display order is re-ranked by the final score. For example, a user want to find recommendations by input “The Godfather”, the original recommender will return the results by similarity solely, However, by introducing popularity, the movie re-orders as, We intuitively say the “The Godfather: Part 3” is more of interested to user who finds suggestion based on “The Godfather” than “Blood Ties”.
This is the main page for user input. After user type in movie title, the ten most similar movies adjusted by popularity values will return.