The Text Analytics page provides a convenient perspective of the free format, comment-type responses to open-ended survey or assessment questions which can be sorted by either theme (topic) or keyword.
Selecting & Filtering Survey or Assessment Results
The survey or assessment results on the Text Analytics page will be displayed according to the (1) model and campaign/project as selected in the Datasets toolbox in the side panel (see Models/Datasets for more), and the (2) current active filter that has been set for business units, demographics and/or engagement level (see Filtering Survey Responses for more).
Select the model, and the campaign/project if necessary, from the dropdowns in the Datasets toolbox in the side panel.
Click on the Print/PDF icon to print or generate a PDF report of the current content as displayed, and on the How-to icon to view a video of the analytics dashboard.
Expanding Constructs to view Text Analytics
The survey constructs/topics that contain open-ended questions will be shown, and will, by default, be in a collapsed state. A text analytics speech bubble icon will be displayed next to the construct/topic description, the number beneath it indicating the number of open-ended responses that have been submitted
Click on the down arrow above the construct’s text analytics speech bubble icon to expand the construct and show the open-ended questions that were included in the survey construct/topic. Open-ended questions will be highlighted in blue and the number of open-ended comment-type responses will be displayed. Click on the open-ended question, or its down arrow, to open a text analytics panel displaying the question's open-ended responses.
The text analytics panel will allow you to view and filter, by unit/sub-unit and/or demographics, qualitative comments and responses to open-ended questions – if such questions were included in the survey. You can also search or display the comments according to either keywords or themes.
Please note that no open-ended comments will be shown if a business unit received fewer survey responses than the anonymity threshold for open-ended responses; the default minimum setting for open-ended responses is 4.
Text Analytics by Topic or Sentiment
The Thematics option will be selected by default and will provide you with (1) a list of themes or topics for the open-ended comments, (2) a graph with the sentiment scores for the selected topic, and (3) the selected topic’s open-ended comments. Comments can be sorted by their sentiment scores, and can also be searched for keywords.
The Thematics function uses Artificial Intelligence (AI) techniques to provide insights into the meanings of the comments entered as responses to the open-ended questions and to identify topics and common themes. AI techniques are also used to determine the sentiment, or emotional tone, of open-ended responses, and to categorize them as either a positive, negative or neutral sentiment.
The Thematics function will analyze all the responses to an open-ended survey or assessment question, grouping them into relatable themes. Analyzing these responses can be a lengthy operation and the We are currently analysing the survey responses. This may take a few minutes. message may be displayed. If the message is not replaced with thematics output within 15 minutes please contact your System Administrator.
Themes can be edited by a System Administrator as required to better reflect a client’s culture or HR practices. Only System Administrators who have been granted Edit Topics rights in Manage Participants will be able to edit the topic names.
| 1. | Display comments sorted by themes – select the Thematics option to display the comments sorted according to topics (themes). |
| 2. | Display comments sorted by keywords – select the Keywords option to display the comments sorted according to keywords (see Text Analytics by Keywords for more). |
| 3 | Select a topic – click on a topic in the topic list to view the topic’s open-ended comments sorted, by default, according to relevancy (i.e. how well the comment fits into the topic). |
| 4. | Edit the topic name (System Admin only) – click on the Edit icon should you wish to make the topic name more descriptive. |
| 5. | Number of comments per topic – the number of open-ended comments that have been grouped for this topic. |
| 6. | Topic prevalence - the frequency with which the various topics were cited by the respondents. |
| 7. | Help content – click to access more information and Help content regarding the text analytics function. |
| 8. | List of comments – a list of the selected topic’s open-ended comments, based on the currently active filter settings for business units, demographics and/or engagement level, will be displayed. The list can be sorted by sentiment or searched for keywords (see points 9. and 10. below for details). Use the vertical scroll bar to traverse the list. |
| 9. | Sort comments according to sentiment – select the Most Positive or Most Negative options to sort the selected topic’s open-ended comments according to their sentiment scores. |
| 10. | Distribution of sentiment scores – line graph with the number of comments per sentiment score. Hover with the mouse over the graph’s data points to view the sentiment scores and number of comments. |
| 11. | Search text – filter the responses according to one or more words that have been entered into the search field. Searches can include more than one word, separated by spaces (e.g. performance bonus), and can also include parts of a word (e.g. perform will find perform, performs, performed, performance and more). Toggle the Exact match option to do a partial search, e.g. for all open-ended responses that contain the word 'perform'. |
More about Topic Modelling
Topic modelling is a Machine Learning (ML) method that uses unsupervised learning as a means to identify hidden, abstract structure and meaning in free-format text, or in the case of Engage Analytics, open-ended comments. The approach assumes that each comment can be aligned with one or more topics, with each topic comprising a group of keywords.
For the more technically inclined: the application uses Anthropic, the OpenAI spinoff, and their Claude large language model for topic analysis. Claude excels in in-depth textual analysis, advanced language capabilities, and contextual understanding of complex conversations, making it ideal for analyzing open-ended responses.”
Making Sense of Sentiment Analysis
To measure the sentiment polarity (emotional tone) in open-ended comments, another open-source library, Valence Aware Dictionary for Sentiment Reasoning (VADER), is used. The technique in effect compares the words of an open-ended comment to a pre-trained dictionary of lexical features, and then label them according to their semantic orientation. The overall sentiment score for each open-ended comment is then represented by a number in the range from -1.0 (extremely negative) to +1.0 (extremely positive).
Some open-ended comments will have large variations and extremes in sentiment polarity, which may result in a neutral sentiment score when averaged. The system will, however, account for negation words. To illustrate, taken individually, a negative word such as "rude" will get a negative sentiment score of -0.4588. Other negative words such as "aggressive" and "impatient" will likewise get negative scores of -0.1531 and -0.296 respectively. However, taken in the context of a sentence, the statement "My manager has never been rude, aggressive, or impatient with me" will get an overall positive sentiment score of +0.1838.
The system will also consider punctuation. For example, the word "enjoy" will get a sentiment score of +0.4939, but " enjoy!" with an exclamation mark will get a higher positive sentiment score of +0.5431, and "enjoy!!" with two exclamation marks will get an even higher positive sentiment score of +0.6219.
Text Analytics by Keywords
The Keywords option will display a list of keywords, sorted from high to low frequency, as well as a word cloud where the keywords are shown in different sizes according to their frequency. An English dictionary is used as the basis for keyword sorting.
| 1. | Keyword list – list of all the keywords used in the open-ended comments, sorted according to frequency. |
| 2. | Display comments sorted by keywords – select the Keywords option to display the comments sorted according to keyword frequency. |
| 3. | Keyword frequency – the number of times the keyword appeared in the open-ended comments. |
| 4. | Select a keyword – to select a keyword, click on it in the keyword list (on the left) or the word cloud. The list of comments at the bottom of the panel will be updated with all the open-ended responses that contain the selected keyword sorted from high to low frequency. |
| 5. | Word cloud – a visual display of all the keywords, sized according to the keyword’s frequency. |
| 6. | Help content – click to access more information and Help content regarding the text analytics function. |
| 7. | List of comments – list of all the open-ended comment-type responses, based on the current active filter settings for business units, demographics and/or engagement level and the keyword that has been selected (if any). Use the vertical scroll bar to scroll through the list. |
| 8. | Search text – filter the responses according to one or more words that have been entered into the search field. Searches can include more than one word, separated by spaces (e.g. performance bonus), and can also include parts of a word (e.g. perform will find perform, performs, performed, performance and more). Toggle the Exact match option to do a partial search, e.g. for all open-ended responses that contain the word 'perform'. |
Text Analytics by Employee Select
The Employee select question type utilizes the Keywords type output (see section above) to display the list of employees selected via the survey or assessment, sorted from high to low frequency, as well as a word cloud where the employees' names are shown in different sizes according to their frequency of selection.
| For more information, click on a sub-menu item at the top of the page⇑ or in the right margin⇒ |