Success story

AI-Powered Customer Experience Analytics

Unstructured reviews turned into brand reputation insights

About the collaboration

Client

Brand reputation software provider

Request

Extend the platform with AI-powered analytics that process millions of reviews and show how your brand reputation evolves over time

Solution

Topic-based customer review analytics and conversational AI agent

The solution in a nutshell

Impact we delivered

Millions of reviews

Analyzed at scale

≤15 seconds

Response latency

4 months

AI chat agent development

Identifying key brand trends across millions of reviews

Client:

“We had already collected a huge amount of customer feedback, ratings, and sentiment data. But when we looked at our dashboard, we could see the numbers without really understanding the insights behind them.

What we really needed was to hear the customer’s voice to understand what people were actually talking about and what was shaping their perception of our brands.”

Oxagile's AI Engineer:

“Initially, it seemed straightforward: feed the reviews into an LLM and ask it to summarize them. But once we dug deeper, we realized that approach wouldn’t work at the scale we needed.

Millions of reviews meant dealing with context limits, response latency, inconsistent outputs, and the challenge of comparing any time period reliably. The real engineering obstacle became: how do we turn millions of unstructured reviews into something an LLM can analyze quickly, consistently, and at scale?”

Building the custom topic-modeling pipeline

We first prototyped topic extraction with BERTopic. However, the off-the-shelf approach had limitations with large datasets, new-data inference, performance, and parameter tuning.

To make the solution production-ready, we replaced the prototype with a custom topic-modeling pipeline. Each review was assigned to a topic and combined with its existing sentiment data, creating a structured layer that could be efficiently aggregated by topic, sentiment, volume, and period of time.

Step 1

Text embeddings

We converted review data into semantic vector representations using Sentence Transformers.

Step 2

Sampling at scale

We used MiniBatch K-Means for initial clustering, followed by stratified sampling to build a representative training set of roughly one million reviews.

Step 3

Hierarchical clustering

We applied it to build a flexible topic hierarchy, allowing the system to operate at different levels of granularity depending on the required number of topics.

Step 4

Topic quality evaluation

We developed a custom metric to measure how distinct each cluster was from its nearest semantic neighbors, helping evaluate and refine topic quality.

Step 5

Topic naming

Finally, we used c-TF-IDF to extract the most characteristic words and phrases for each cluster, turning abstract vector-space groups into human-readable topics.

Customer feedback you can actually use case study

How the analytics dashboard looks like

Select any time period, from a week to a year, and analyze ten to thousands of business locations
Filter by platform and business location to customize your report
Track how sentiment, review volume, and key topics change over time
Get an AI-generated summary of the most common issues and customer concerns
See review volume, average rating, response rate, and sentiment distribution at a glance
Compare top- and bottom-performing businesses across selected locations and periods

Integrating a conversational AI agent

The AI agent helps explore customer feedback through natural conversation, transforming complex review data into answers, insights, and evidence.

Ask questions naturally

Ask business questions in your own words
Explore customer pain points, trends, and reputation
Follow up with additional questions to go deeper

Explore insights interactively

Ask why a topic became significant or changed over time
Move from broad trends to specific business areas
Build on previous answers to explore the data further

Go from answers to evidence

Drill down from an insight to the topics and reviews behind it
See the customer feedback supporting each conclusion
Validate insights without manually searching through thousands of reviews

Infrastructure and deployment

“The existing infrastructure was a relatively lightweight, self-managed Kubernetes cluster, so we didn’t want to add the operational overhead of running heavy ML workloads. That's why we used AWS SageMaker to manage the ML infrastructure and pipelines. SageMaker Pipelines handled the data and model workflow, and we deployed the models for asynchronous inference.

This also allowed topic extraction to run alongside sentiment extraction as part of the existing data-processing pipeline. The goal was to use managed ML infrastructure without increasing the operational burden of the platform.”

— Oxagile's AI Engineer

The business impact of our approach and architecture

Better scalability

The solution operates on much larger datasets than the initial LLM-based approach.

Faster analysis

Large analytical workloads are processed within the required dashboard response window.

More granular temporal analysis

Users are no longer constrained to monthly summaries and can select arbitrary periods.

Fewer inconsistencies

The topic-based architecture reduces the mismatch between aggregated numbers and generates textual explanations.

Traceable AI insights

Users can trace generated insights back to the underlying reviews.

Production-ready topic modeling

The initial BERTopic prototype was replaced with a custom scalable pipeline.

Tech stack and practices

Clustering and sampling methods

MiniBatch K-Means clustering • Stratified sampling • Hierarchical clustering • Metric-based threshold selection

Model deployment and infrastructure

Amazon SageMaker Pipelines • Asynchronous endpoints

Model evaluation metrics

Mean KNN diversity • Calinski–Harabasz score • Davies–Bouldin score

Ready to scale AI beyond the obvious?

Scaling AI doesn’t mean processing more data. We create the data structure it before it reaches the model. Talk to our AI experts to build a more efficient solution.

Request a call