Text Analytics

The Text Analytics page will provide you with a convenient perspective of the free format comment-type responses to open-ended survey questions - either sorted by keyword or by theme or topic.

Selecting & Filtering Survey Results

The survey results on the Text Analytics page will be displayed according to the (1) survey model and campaign as selected in the Datasets toolbox in the side panel (see Survey Models/Datasets for more), and the (2) current active filter that has been set for business units, demographics and/or engagement level (see Filtering Survey Responses for more).

2 select dataset

Click on the Print/PDF icon to print or generate a PDF report of the current content as displayed, and on the How-to icon to view video of the analytics dashboard.

Expanding Constructs to view Open-ended Responses

The survey constructs/topics that have open-ended survey questions will be shown, and will by default be in a collapsed state. A text analytics speech bubble icon next to the survey construct/topic description will be displayed, with a number underneath to indicate the number of open-ended responses that have been submitted.

7 questions 2

Click on the down arrow next to the survey construct/topic description to expand the construct and to show the open-ended survey questions that were included in the survey construct/topic. Open-ended questions will be highlighted in blue and will show the number of open-ended comment-type responses. Click on the open-ended question to open a text analytics panel with the survey question's open-ended responses.

The text analytics panel will allow you to view and to filter by unit/sub-unit and/or demographics qualitative comments and responses to open-ended questions – if such questions have been included in the survey – and to search or display the comments according to either keywords or themes.

Please note that no open-ended comments will be shown if a business unit received fewer survey responses than the anonymity threshold for open-ended responses; the default minimum setting for open-ended responses is 4.

Text Analytics by Topic or Sentiment

The Thematics option will be selected by default, and will provide you with a (1) list of themes or topics for the open-ended comments, (2) a graph with the sentiment scores for the selected topic, and (3) the selected topic’s open-ended comments. Comments can be sorted either according to their probability of belonging to the selected topic, or according to their sentiment scores, and can in addition be filtered according to keywords.

The Thematics function applies Natural Language Processing (NLP) techniques to open-ended comments to gain insights into the meaning of the comment and to identify topics and common themes. NLP techniques are also used to determine the sentiment, or emotional tone, of an open-ended comment, and to categorize it as either a positive, negative or neutral sentiment.

Topic analysis requires some level of human intervention. The Thematics function will analyse all the responses to an open-ended survey question, and will group them in relatable topics or themes according to their keyword relevance, but a human operator with a better understanding of the actual work environment will then have to review the topic’s keywords and define a more relatable or descriptive name for the topic. Since the editing of topic names can only be done by System Administrators (managers and other analytics dashboard users will not be able to edit any of the topic names), it will make sense to review and edit the topic names before allowing anyone else to view the thematic results of their business unit’s open-ended responses – generic names such as “Topic 3” will not add much value.

7 questions 4A

1.	Display comments sorted by themes – select the Thematics option to display the comments sorted according to topics (themes).
2.	Select a topic – click on a topic in the topic list to view the topic’s open-ended comments, sorted by default according to relevancy (i.e. how well the comment fits into the topic).
3.	Edit the topic name (System Admin only) – click on the Edit icon to enter a more descriptive and relatable name for the topic. The Keyword relevance graph and percentages (see item 6. Keyword relevance for the topic below) will provide you with a good indication of the most dominant and relevant keywords and should be used as the basis for the formulation of an appropriate name for the topic.
4.	Probability % per topic – display the overall probability score for the topic as a percentage.
5.	Number of comments per topic – display the number of open-ended comments that have been grouped with the topic.
6.	Keyword relevance for the topic – when a topic has been selected, the bar graph will display a list of the topic's keywords, sorted according to their relevance with respect to the topic. When the All Topics option at the top of the topic list has been selected, the bar graph will display a list of all the topics, sorted according to their probability scores.
7.	Help content – click to access more information and Help content regarding the text analytics function.
8.	List of comments – a list of the selected topic’s open-ended comments, based on the current active filter that has been set for business units, demographics and/or engagement level, will be displayed. The list will be sorted according to either their relevance or sentiment scores for the topic. Use the vertical scroll bar to scroll down.
9.	Sort comments according to relevance – sort the list according to their relevance scores for the topic.
10.	Sort comments according to sentiment – select the Most Positive or Most Negative options to sort the selected topic’s open-ended comments according to their sentiment scores.
11.	Search text – filter the responses according to one or more words that have been entered into the search field. Searches can include more than one word, separated by spaces (e.g. performance bonus), and can also include parts of a word (e.g. perform will find perform, performs, performed, performance and more). Toggle the Exact match option to do a partial search, e.g. for all open-ended responses that contain the word 'perform'.
12.	Distribution of sentiment scores – line graph with the number of comments per sentiment score. Hover with the mouse over the graph’s data points to view the sentiment scores and number of comments.

Making sense of Sentiment Analysis

To measure the sentiment polarity (emotional tone) in open-ended comments, another open-source library, Valence Aware Dictionary for Sentiment Reasoning (VADER), is used. The technique in effect compares the words of an open-ended comment to a pre-trained dictionary of lexical features, and then label them according to their semantic orientation. The overall sentiment score for each open-ended comment is then represented by a number in the range from -1.0 (extremely negative) to +1.0 (extremely positive).

Some open-ended comments will have large variations and extremes in sentiment polarity, which may result in a neutral sentiment score when averaged. The system will, however, account for negation words. To illustrate, taken individually, a negative word such as "rude" will get a negative sentiment score of -0.4588. Other negative words such as "aggressive" and "impatient" will likewise get negative scores of -0.1531 and -0.296 respectively. However, taken in the context of a sentence, the statement "My manager has never been rude, aggressive, or impatient with me" will get an overall positive sentiment score of +0.1838.

The system will also consider punctuation. For example, the word "enjoy" will get a sentiment score of +0.4939, but " enjoy!" with an exclamation mark will get a higher positive sentiment score of +0.5431, and "enjoy!!" with two exclamation marks will get an even higher positive sentiment score of +0.6219.

Please keep in mind that both topic analysis and sentiment analysis are intended to facilitate further manual review and interpretation of the text feedback.

Text Analytics by Keywords

The Keywords option will display a list the keywords, sorted from high to low frequency, as well as a word cloud where the keywords are shown in different sizes according to their frequency. An English dictionary is used as the basis for keyword sorting and to identify and exclude stop words in the English language, e.g. 'and' or 'the'.