A Survey Paper on Approaches & Tools used in Sentiment Analysis Monu Bhagat 1Dr. Dilip Kumar2,
1 NIT Jamshedpur, Computer Science & Engineering Department, Jamshedpur, India
2 NIT Jamshedpur, Computer Science & Engineering Department, Jamshedpur, India
Abstract. Appearance of Web 2.0 and evaluation of social media platform (like Facebook, Twitter, LinkedIn etc.), product based sites (like Amazon, Flipkart, EBay etc.) and Service based sites (like OYO, Trip Advisor etc.) attracted large number of people to share their views or opinions with other through the Internet. So it become very urgent to analyze and judge the underlying meaning or tendency of emotion expressed in the text which faces a huge number of unstructured comments from these different sources by Natural Language Processing. Natural Language Processing (NLP) has many applications in which Sentiment analysis is one of them of great importance. The task to perform Sentiment analysis involves various works like subjectivity detection, sentiment classification, Aspect term extraction, feature extraction etc. In this paper, Survey of different methods and works of sentiment, its applications ,research scope and different Tools used in Sentiment analysis are described in detailed.
Keywords: Sentiment analysis, classi?cation, feature selection, NLP, machine learning, polarity, SVM, opinion mining.
Sentiment analysis is also known by some other names like emotion reorganization or opinion mining etc. It is very well known field of Research which falls in the area of Text Mining. The basic idea behind the Sentiment analysis is to find the polarity of the given text and accordingly classify it into positive or negative or neutral. In recent years sentiment analysis or opinion mining is one of the biggest ?eld of research in Natural language Processing (NLP) or Text mining where researcher has the duty to train a system in such a way so that it can classify the sense of a sentence. By sense we mean its underlying attitude whether positive or negative or neutral. The language spoken by a people can easily be read through a paragraph and quickly identify whether the writer had an overall positive or negative impression of the topic at hand. However, for a computer (machine), which has no concept of natural spoken language, this problem must be reduced to mathematics. Without any context of what words actually mean, it cannot simply deduce whether a piece of text conveys joy, anger, frustration, or otherwise. Sentiment analysis seeks to solve this problem by using natural language processing to recognize keywords within a document and thus classify the emotional status of the piece of information. It helps in decision making of human being. The process of sentiment analysis is shown below in fig.1.
Fig.1, Sentiment Analysis Process
Important Terms in Sentiment Analysis Technique
A) Subjectivity/Objectivity- In order to perform sentiment analysis first it is needed to identify the subjective and objective text. Only subjective text holds the sentiments. Objective text contains only factual information. Example-
1.) Subjective: Lagaan is a superb movie. This sentence has a sentiment (superb), thus it is subjective.
2.) Objective: James Cameron is the director of titanic. This sentence has no sentiment, it is a fact, and thus it is objective 2.
B) Polarity- Further subjective text can be classified into 3 categories based on the sentiments conveyed in the text.
1.) Positive: I love Lenovo mobile.
2.) Negative: The picture quality of camera was awful.
3.) Neutral: I usually get hungry by noon. This sentence has user’s views, feelings hence it is subjective but as it does not have any positive or negative polarity so it is neutral. 2.
Sentiment level- sentiment analysis can be performed at various levels –
Document Level- In it the whole document is given a single polarity positive, negative or objective 1.
Sentence Level – In it document is classified at sentence level. Each sentence is analyzed separately and classified as negative, positive or objective. Thus overall document has a number of sentences where each sentence has its own polarity.
Phrase Level- It involves much deeper analysis of text and deals with identification of the phrases or aspects in a sentence and analyzing the phrases and classifies them as positive, negative or objective. It is also called aspect based analysis 2.
2 Literature Survey
In this section, a literature survey on the basis of literature and research papers of past year’s carried out by several researchers in the sentiment analysis domain has been presented. A few works has been discussed below in tabular form where different methods, algorithm, challenges etc. have shown.
TABLE.1: Previous works on Sentiment Analysis
S.no Title of Paper & publication Year Author Name Issues Addressed Technique/ Method
used Sources/ Data Set Used Proposed Solution Reference
1 Sentiment Analysis using Product Review Data
Xing Fang and Justin Zhan
(2015) Sentiment polarity categorization
Naïve Bayes, Random forest, SVM, Sentence level categorization, and Review level categorization
E-commerce Product review (Amazon)
Naïve Bayesian, Random forest, and Support vector machine 6
2 Mining affective text to improve social media item recommendation Jianshan Sun et al.
(2015) Large volume of information like Data Sparsity problem
Ensemble learning (SentiWordNet) Implicit feedback, inferred sentiment feedback
Talk comments on Social media website
Extend OCCF to SA_OCCF, and improve recommendation performance
3 Multi-aspect opinion polling from textual reviews Zhu et al.
(2009) Mixed opinion problem
Unsupervised approach, Multiaspect bootstrapping method, Predominant polarity, Aspect-sentiment extraction
Aspect based opinion polling from raw textual reviews
4 Opinion Miner: A novel machine learning system for web opinion mining and extraction Jin et al.
(2009) Problem of find out potential review into huge amount of reviews
Lexicalizes HMMs approach, Sentiment classification, Aspect extraction, Bootstrapping approach and LHMM
Online product (Camera) reviews
The opinion miner system mine customer reviews of a product and extract high detailed product entities on which reviewers express their opinion
5 Red Opal: Product feature scoring from reviews Scaffidi et al
(2007) Opinion extraction is a hard problem Red Opal model, Aspect detection, Feature extraction, Product feature and Product score Product rating and reviews on Amazon Red Opal that enables users to search best product with good features 10
6 Assessing Antidepressants Using Intelligent Data Monitoring and Mining of Online Fora Altug Akay et al.
(2016) Depression is a global health concern. Social networks allow the affected population based on observations derived from user behavior in depression related social networks.
Network based analysis, TF-IDF score, k-means clustering
User behavior in Depression Forum
It allows all parties to participate in improving future health solutions of patients suffering from depression.
7 Systematical Approach for Detecting the Intention and Intensity of Feelings on Social Network
Chih-Hua et al.
(2016) The problem of detecting how peoples feel about their daily life through their online post about mental illness.
FeD (Feeling Distinguisher system) based on sLDA, LDA, and SentiWordNet
User feeling on social network
Developed new method FeD for detecting a person’s intention and intensity of feelings through the analysis of his/her online posts.
8 Dual Sentiment Analysis: Considering Two Sides of One Review Rui Xia et al.
(2015) Limited performance of BOW and Polarity shift problem Lexicon based antonym and Corpus based Pseudo-Antonym dictionary Polarity shift and Reversed reviews E-commerce and Social media Bag of Words (Reviews)
Propose Dual training algorithm, Dual Sentiment analysis, and extend polarity of DSA framework from 2-class to 3-class.
9 Sentiment Analysis of Textual Reviews. V.K. Singh, R. Piryani, A.Uddin, P. Waila Polarity shift and
Accuracy Lexicon based
Approach Movie Review
Dataset SentiWordNet approach 22
10 Ontology-Guided Approach to Feature-Based Opinion Mining. Isidro Peñalver-
Valencia Garcia(2014) Binary
problem, Ontology at feature
extraction stage Twitter Dataset Feature based opinion mining by employing ontologies improves performance. 23
4 Sentiment Analysis Methods
The following figure will show different types of techniques so far used for sentiment analysis by researcher.
Fig.2, Methods for Sentiment Analysis.5 Tools Used In Sentiment Analysis
Now day’s several sentiment analysis tools are available in the market. The (open-source text analytics tools) have been used for natural language processing (NLP), such as information extraction and classification which can also be applied for web sentiment analysis, In this section we have been explored some of the popular tools used for sentiment analysis. Some of them as are follows:
TABLE.2: Different Tools used in Sentiment Analysis
SL. No. Sentiment Analysis Tool Application/Purpose Reference
1 NTLK A Natural Language Toolkit for text processing, cataloging, tokenization, stopping tagging, parsing etc. 14
2 Opinion Finder Tool for identifying individual sentences and to create different parts of subjectivity in these sentences. 15
3 Open NLP Toolkit based on Machine learning technique. 16
4 Web Fountain Sentiment analysis tool that completes the requirements of analysis agent such as text gathering, storing, indexing and querying 17
5 WEKA Tool based on ML techniques. JAVA programming language is used to implement it. It has its GUI to show data. Used many algorithm and techniques such as classification, clustering, preprocessing and linear regression. 18
6 Ling Pipe Tool used for linguistic processing for text including clustering, cataloging and extraction. 19
7 Stanford Parser This tool is used as a POS tagger and sentence parsing from the NLP group. 20
8 SentiWorldNet Lexical dictionary and scores obtained by
semi-machine learning approaches 21
9 SenticNet Natural language processing approach for
inferring the polarity at semantic level 21
10 LIWC Dictionary and sentiment classified
6 Applications of Sentiment Analysis Technique
As we know that the Opinion based or feedback based application are more valuable, now a days, the natural language processing researchers show more interest in Sentiment Analysis and Opinion Mining system. With the growth of internet people’s life style has changed, now they are more expressive on their views and opinions 1, and this tendency helped the researchers in getting user-generated content easily. The followings are major applications of sentiment analysis.
1) In Detection of flame: The taking care of forums, blogs and social media is easily possible by sentiment analysis. It can analyze over heated words or hatred language used in emails or forum entries or tweets on various internet sources.
2) In Decision Making: People’s opinion and experience are very useful element in decision making process. Opinion mining and Sentiment analysis gives analyzed people’s opinion that can be effectively used for decision making.
3) In Policy Making: Through Sentiment analysis, policy makers can take citizen’s point of view towards some policy and they can utilize this information in creating new citizen friendly policy.
4) In Opinion spam detection: People may write spam content to mislead the people. Opinion mining and sentiment analysis can classify the internet content into? spam? content and not spam content 1.
5) In Purchasing Product or Service: This technique is useful for evaluation of people’s opinion and experience about any product or service and also he/she can easily compare the competing brands.
6) ) In Recommendation Systems: By classifying the people’s opinion into positive and negative, the system can say which one should get recommended and which one should not get recommended 5.
7) In Marketing research: The result of sentiment analysis techniques can be utilized in marketing research 3. By sentiment analysis techniques, the recent trend of consumers about some product or services can be analyzed. Similarly the recent attitude of general public towards some new government policy can also be easily analyzed. These all result can be contributed to collective intelligent research 4.
8) For Quality Improvement in Product or service: By sentiment analysis the manufactures can collect the critic’s opinion about their products or services and hence they can improve the quality of their products or services.
7 Research Scope in Sentiment Analysis
The followings are the major research scope in sentiment analysis:
Rich content lexicon database generation;
Development of fully automatic analyzing tool;
Good Handling of bi polar sentiments;
Developing good sentiment identification algorithms;
Sentiment analysis of short messages;
From the Survey, It is clear that Sentiment analysis has wide area of applications and it also facing many research challenges. With the explosion of internet and internet based applications, the Sentiment Analysis has taken a most interesting place in research area among natural language processing researchers. A more appropriate and workable techniques required to be discovered which should overcome the current challenges faced by Opinion Mining and Sentiment Analysis community. Some hybrid techniques should be used to get more accurate results.
1. Liu, B. (2010), “Sentiment Analysis and Subjectivity”. Appeared in Handbook of Natural Language Processing, Indurkhya, N. ; Damerau, F.J. Eds..
2. Hu, M. ; Lui, B. (2004), “Mining and Summarizing Customer Reviews”. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2004 (KDD-2004), p. 168–177.Vol-3 Issue-2 2017 IJARIIE-ISSN (O)-2395-43964871 www.ijariie.com 5078
3. Boiy, E., Hens, P., Deschacht, K. ; Moens, M.-F. (2007), “Automatic Sentiment Analysis in OnLine Text”. In Proceedings of the Conference on Electronic Publishing (ELPUB-2007), p. 349-360.
4. Segaran, T. (2007), Programming Collective Intelligence. Sebastopol: O’Reilly Media, Inc.
5. Pang, B. & Lee, L. (2008), “Opinion Mining and Sentiment Analysis”. In Foundations and Trends in Information
Retrieval 2 (1-2), p. 1–135.
6. Xing Fang and Justine Zhan, Sentiment Analysis using Product Review Data, Journal of Big Data (Springer), 2:5, (2015)
7. David Griol, Jose Manuel Molina, and Garcia-Herrero, Fusion of Sentiment Analysis and Emotion Recognition to Model the User’s Emotional State, 18th International Conference on Information Fusion, Washington, DC, July 6-9, (2015).
8. J. Zhu, H. Wang, B. K. Tsou, and M. Zhu, Multi-aspect opinion polling from textual reviews, in Proceedings of the 18th conference on information and knowledge management, A CM (2009) pp. 1799-1802.
9. W. Jin, H. H. Ho, and R. K. Srihari, Opinion Miner: A novel machine learning system for web opinion mining and extraction, in Proceedings of the 15th international conference on knowledge discovery and data mining, ACM (2009) pp. 1195-1204.
10. C. Scaffidi, K. Bierhoff, E.Chang, M. Felker, H. Ng, and C. Jin, Red Opal: Product feature scoring from reviews, in Proceedings of the 8th conference on electronic commerce , A CM (2007) pp. 182-191.
11. Altug Akay, Andrei, and Bjorn –Erik Erlandsson, Assessing Antidepressants Using Intelligent Data Monitoring and Mining of Online Fora, IEEE Journal of Bio medical and Health Informatics, Vo l. 20, No. 4, July 2016.
12. Chih-Hua Tai, Zheng-Han Tan, and Yue-Shan Chang, Systematical Approach for Detecting the Intention and Intensity of Feelings on Social Network, IEEE Journal of Bio medical and Health Informatics, Vo l. 20, No. 4, July 2016.
13. Rui Xia, FengXu, ChengqingZong, Qianmu Li, Yong Qi, and Tao Li, Dual Sentiment Analysis: Considering Two Sides of One Review, IEEE Transactions on Knowledge and data Engineering, Vo l. 27, No. 8, (2015).
14. Online Available: http://Ntlk.logo
15. Online Available: http://opinion finder. Logo/
16. Online Available: http://open nlp.biolab.si/
17. Online Available: http://webfountain.logo/ 18. Onlinehttp://www.cs.waikato.ac.nz/ml/weka
19. Online http://alias-i.com/lingpipe.
20. Dr. Arvind K Sharma et al, “Web Opinion Mining Techniques and Tools for Finding User’s Opinion”, in the 5th International Conference on Innovative Research in Engineering Science and Management (ICIRESM-16) at India International Centre, Max Mueller MargS, New Delhi, India.
21. Vijayaraj, R. Saravanan, P. Victer Paul, R. Raju, “A Comprehensive Survey on Big Data Analytics Tools”, IEEE Third International conference on Innovations in Information, Embedded and Communication Systems (ICIIECS’16), Coimbatore, Volume 2, March 2016, pp. 32-40. ISSN: 978-1-4673-8203-3
22. V. K. Singh, P. Waila, Marisha, R. Piryani & A. Uddin, “Sentiment Analysis of Textual Reviews: Evaluating Machine Learning, Unsupervised and SentiWordNet Approaches”, In Proceedings of 5th International Conference of Knowledge and Smart Technologies, Burapha University, Thailand, Jan. 2013.
23. Peñalver-Martínez I., Valencia-García R., García-Sánchez F. (2011) Ontology-Guided Approach to Feature- Based Opinion Mining. In: Muñoz R., Montoyo A., Métais E. (eds) Natural Language Processing and Information Systems. NLDB 2011. Lecture Notes in Computer Science, vol 6716. Springer, Berlin, Heidelberg.