This blog post explains the basic concepts of recommendation systems, focusing on the principles and application examples of collaborative recommendations.

Overview of Recommendation Systems and Data Mining

You may recall seeing phrases like “Customers who bought this item also bought” on the U.S. online bookstore amazon.com. This is a service that suggests to users while they are shopping, “How about this book?” Additionally, when a user enters a search term incorrectly on a portal site, some search engines automatically correct typos so that the user can find the search term they originally intended. Ultimately, it is the computer that plays the role of recommending products and correcting misspelled words.
Recommender systems are a subfield of data mining, one of the major research areas in computer science. The term “data mining”—a combination of “data” (meaning information) and “mine” (referring to the extraction of minerals from a mine)—refers to the process of discovering new data models previously unknown from given data, extracting actionable information for the future, and using that information to inform decision-making.
The purpose of a recommendation system is to predict user behavior based on existing data and, accordingly, extract and provide meaningful information. So, how can such predictions be made? Basic and representative approaches to recommendation systems include collaborative recommendation, content-based recommendation, and knowledge-based recommendation. In this article, we will examine what collaborative recommendation is and how it works.

Principles and Examples of Collaborative Recommendation

Suppose a clerk at a large bookstore is asked by a customer to recommend a new book, citing Book A as one they found particularly memorable. The clerk can examine the subsequent behavior of other customers who purchased the same Book A. If, upon checking sales records, the clerk finds that most customers who previously bought Book A went on to purchase Book B, the clerk can recommend Book B to the current customer. This is a classic example of collaborative recommendation.
Collaborative recommendation is a method that links users who have behaved similarly in the past through a “collaborative” relationship and uses that information to predict a user’s next action or preference. One of the key features of collaborative filtering is that it can predict outcomes based solely on the past behavior patterns of users in the collaborative network, even without information about the target item. Therefore, the clerk mentioned above can recommend Book B even without knowing specifically what Book A is.
One of the core processes in collaborative filtering is establishing the collaborative network. From a computer science perspective, this involves quantifying the similarity between users who have exhibited similar behavior in the past to determine the degree of similarity. One basic method is to use the Pearson correlation coefficient.
The Pearson correlation coefficient is a metric that indicates the similarity of choices between two users, with values ranging from -1 (strong negative correlation) to 1 (strong positive correlation). Let P be a set of specific items, r_i,j be the ratings given by user i for item j, and r̄_a be the average rating of user a. Then, the similarity sim(a, b) between users a and b can be calculated using the Pearson correlation coefficient. (The original text included a formula and an example user-rating table, but here we provide a conceptual explanation.)
Suppose that, in the example user-rating table, the similarity between Alice and User1, calculated using the Pearson correlation coefficient, is 0.85. Since 0.85 is close to 1, we can conclude that the ratings of Alice and User1 are positively correlated, and this can be regarded as a collaborative relationship.
If there is a metric indicating the similarity of user ratings, it can be used to predict a specific user’s rating for an item. First, let N (neighbors) denote the set of other users whose Pearson correlation coefficients with a specific user meet a threshold (specified arbitrarily). Conceptually, the predicted rating pred(a, p) for item p by user a is calculated by taking a weighted average of the ratings and similarities of these neighbors.
For example, in the table above, if User1 and User2—whose Pearson correlation coefficients are 0.85 and 0.7, respectively—are set as Alice’s neighbors, Alice’s rating for Item5 can be predicted by reflecting the similarity to the ratings of these two neighbors.
Computer science has evolved to pursue human convenience, and recommendation systems—as part of these research achievements—provide convenience to modern people without them even realizing it. People tend to consult the opinions of others when making choices; while in the past this process had to be done manually, today computer-based recommendation systems allow us to find suitable choices more easily and quickly. The fact that computers automatically recommend choices tailored to individuals prompts us to reconsider just how useful technological advancements are to humanity.