TBA2105: Smartphones with Internet Connectivity are Probably be Considered More Essential Items: Web Mining Assignment, NUS, Singapore

Learning Objectives

This final assessment is meant to be an open-ended individual take-home assessment. It aims to assess your ability to think critically and design solutions to tackle real-world problems. As the time given for this assessment is very short compared to the amount of time a data analyst/scientist will spend on similar projects, you will only be focusing on a few deliverables.

Opening Narrative

Smartphones with internet connectivity are probably be considered more essential items in modern-day contexts. Even developing countries are catching up on this aspect and the number of smartphones has even surpassed the number of computer devices. One would agree that the value of a smartphone has shifted from focusing just on providing superior hardware to enhancing the user experience with software support.

Google Pixel phone is probably one good example of this. While the whole world was chasing over bezel-less design and impressive hardware specifications, Google Pixel 2 launch seems to be rather disappointing if we were to compare based on the form factor. However, Google seems to be able to entice consumers by providing more superior software support (such as a camera with Al processing, first to receive the latest Android updates, free unlimited full-quality image uploads to Google Photos, etc).

While different smartphone manufacturers offer various proprietary software, there is still a generous selection of mobile apps found on the various App Stores. In this assessment, you will be designing and implementing 3 Mobile App Recommender systems.

Types of Recommender Systems

As suggested in the lecture, there are different ways to build a recommender system. In this assessment, we will focus on the following 3 types of recommender systems:

• Aggregate-based recommender systems
• Content-based recommender systems
• Collaborative Filtering recommender systems

For each of the above 3 types of recommender systems, you will research and discuss how you can go about building such a recommender system. The typical process of building recommender systems often involves

1. Mining data (potentially from multiple sources)
2. Preparing the data (e.g. merging the data from multiple sources into a single dataset)
3. Building the recommender system

Mining Data and Data Preparation

[Discussion]

For each of the 3 types of recommender systems, you should research and discuss what sort of data, fields, and web source(s) by which you will be collecting the data to build the recommender system(s). Please discuss the motivation for why the chosen data will help build the recommender system. You should also focus on data that can be mine publicly from the web rather than internal data (e.g. Google/Apple has the app download data of its user but this information is unlikely to be available online). You should avoid choosing dataset(s) that can be downloaded off data repository websites as the data tend to be outdated but this assessment aims to provide the design of recommender systems that can be built and enhanced with live data.

[Implementation]

Provide the R codes that can be used to scrape the above-discussed data. To assist in the marking, you should comment on your codes to describe what different sections of the codes are doing.

[Discussion]

For each type of recommender systems, discuss the structure of the dataset (columns and its description) used to train the system. In addition, you should also discuss the data preparation process (e.g. how to merge the data, what sort of data cleaning has to be performed, etc).

Embed screenshots of how the data would look like after it has been mined by the scraper and when the data preparation has been performed. Note that you will only need to build and ensure that the scraper(s) work. You do not have to mine the full set of data and you do not have to show the data preparation codes in R (i.e. you are free to do the data preparation using a text editor or with Excel) but you should at least describe the steps to perform the data preparation.

Building the Recommender Systems

[Discussion]

For each of the 3 tunes of recommender systems, discuss how each type of recommender system works (e.g. idea behind how the recommendations are made). You should have 3 separate discussions (one for each type of recommender system). As mentioned earlier, you do not need to do the full web scraping process and do not need to provide the full data that is used to build the systems. However, you should provide some examples of how the data might look like to explain how they work.