Data Science

This question is designed to help you understand topic modeling  better as well as how to visualize topic modeling results, aims to  collect the human meanings of documents. Based on the yelp review data  (only the review text will be used for this question), which can be  download from Dropbox: https://www.dropbox.com/s/59hsrk56sfwh9u2/Assignment%20four%20data%20Yelp%20%28question%201%20and%202%29.zip?dl=0, select two models and write a python program to identify the top 20 topics (with 15 words for each topic) in the dataset.  Before answering this question, please review the materials in lesson  8, as well as the introduction of these models by the links provided.(1)   Labeled LDA (LLDA): https://github.com/JoeZJH/Labeled-LDA-Python(2)   Biterm Topic Model (BTM): https://github.com/markoarnauto/biterm(3)   HMM-LDA: https://github.com/dongwookim-ml/python-topic-model(4)   SupervisedLDA: https://github.com/dongwookim-ml/python-topic-model/tree/master/notebook(5)   Relational Topic Model: https://github.com/dongwookim-ml/python-topic-model/tree/master/notebook(6)   LDA2VEC: https://github.com/cemoody/lda2vec(7)   BERTopic: https://github.com/MaartenGr/BERTopic(8)   LDA+BERT Topic Modeling: https://www.kaggle.com/dskswu/topic-modeling-bert-lda(9)   Clustering for Topic models: (paper: https://arxiv.org/abs/2004.14914), (code: https://github.com/adalmia96/Cluster-Analysis)The following information should be reported:(1) Top 20 clusters for topic modeling.(2) Summarize and describe the topic for each cluster. (3) Visualize the topic modeling reasults by using pyLDAVis: https://www.machinelearningplus.com/nlp/topic-modeling-visualization-how-to-present-results-lda-models/#14.-pyLDAVis Question 2: Yelp Review Sentiment Analysis**
Sentiment analysis also known as opinion mining is a sub field  within Natural Language Processing (NLP) that builds machine learning  algorithms to classify a text according to the sentimental polarities of  opinions it contains, e.g., positive, negative, neutral. The purpose of  this question is to develop a machine learning classifier for sentiment  analysis. Based on the dataset from assignment three, write a python  program to implement a sentiment classifier and evaluate its  performance. Notice: 80% data for training and 20% data for testing.The data can be download from Dropbox: https://www.dropbox.com/s/59hsrk56sfwh9u2/Assignment%20four%20data%20Yelp%20%28question%201%20and%202%29.zip?dl=0
The sentiment of can be accessed based on the star rating, if  no star information avaliable for a record, just remove that record.  Detail star and sentiment level can be matched blew:Very positive = 5 starsPositive = 4 starsNeutral = 3 starsNegative = 2 starsVery negative = 1 starHere is code for yelp data preprocessing: https://github.com/Yelp/dataset-examples. Answer the following questions:(1) Features used for sentiment classification and explain why you  select these features (tf-idf, sentiment lexicon, word2vec, etc).  Considering achieve the best performance as you can. (2) Select two of the supervised learning algorithm from scikit-learn library: https://scikit-learn.org/stable/supervised_learning.html#supervised-learning, to build a sentiment classifier respectively. (3) Compare the performance over accuracy, precision, recall, and F1  score for the two algorithms you selected. Here is the reference of how  to calculate these metrics: https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9.

Question 3: House price prediction

You are required to build a regression model to  predict the house price with 79 explanatory variables describing  (almost) every aspect of residential homes. The purpose of this question  is to practice regression analysis, an supervised learning model. The  training data, testing data, and data description files can be download  from Dropbox: https://www.dropbox.com/s/52j9hpxppfo921o/assignment4-question3-data.zip?dl=0. Here is an axample for the implementation: https://towardsdatascience.com/linear-regression-in-python-predict-the-bay-areas-home-price-5c91c8378878.

Solution: