Transforming Federal Agency’s Complex Survey Data With AI Text Analytics

A large government agency needed to analyze a high volume of responses to a biennial customer satisfaction survey conducted among more than 10,000 annuitants. The survey collects both closed- and open-ended responses on satisfaction with various retirement benefits and services. The agency wanted to understand how its full breadth of service offerings were performing and also uncover opportunities for improvement.

ORI analyzed nearly 6,000 text responses to open-ended questions—delivering insights that may not have been unearthed through manual coding alone.

Challenge

The previous survey questionnaire lacked detailed and focused questions, making it difficult to collect responses with the depth of feedback needed to produce high-value findings and identify areas for improvement. While the previous vendor’s analysis of open-ended responses provided a broad view of customers’ experiences and perceptions related to retirement benefits, it lacked robust details, interpretation of results, and actionable insights into the driving factors behind them.

Solution

ORI overhauled the data collection and analysis methods, starting with redesigning the survey questionnaire to reduce survey fatigue and encourage more completions. The revamped survey also helped ensure that the data collected on customers’ experiences and satisfaction levels was granular and specific enough to guide decision making.

Next ORI’s AI-powered text analytics solution was used to analyze nearly 6,000 text responses to open-ended questions—delivering insights that may not have been unearthed through manual coding alone. Open-ended survey responses were categorized efficiently and effectively into detailed themes to provide a deeper understanding of annuitants’ experiences and perceptions of the agency’s retirement services. Finally, ORI improved the usefulness, quality, and readability of the report to deliver actionable information to guide program improvements.

Outcome

The data analysis identified satisfaction levels at various points throughout the retirement process, delivering a nuanced understanding of annuitants’ experiences and perceptions of retirement services—including specific areas of concern—to help inform improvements and increase satisfaction. A valuable bonus: The updated survey questionnaire is easily repeatable and will enable consistent tracking of satisfaction measures to evaluate the effectiveness of changes made to retirement services over time.