ThoughtSpot Cuts Its CDP Costs by 30% with Revefi-Backed Data Governance and Management

‍

“Revefi is an insurance policy for your data team. It notifies you of problems so your data team can act in real time. That helps you to sleep peacefully at night.”
– Aby Jerin, Principal Data Engineer, ThoughtSpot.

“It was up and running in almost no time, and immediately we were able to get into the insights. Over time, we got our workflows aligned in terms of how it makes sense for us, in terms of our use case and the problems we're trying to address.” – Kaushik Patel, SVP Business Operations, ThoughtSpot.

ThoughtSpot is an AI-driven analytics platform that provides companies with insights. The company harnessed cutting-edge data processing technologies to help digital business supervise their data assets and run full-fledged self-service analytics.

The main challenge of keeping ThoughtSpot’s data stack robust and clockwork is the highly intense, heavy-loaded processing of business users' queries ensuring stable workflows for 600-700 active users.

Working out the weak spots was critical to preserve ThoughtSpot’s output data reliability. Dataset inconsistency and errors could affect end users, who fully trust insights based on the platform’s data. Therefore, ThoughtSpot’s data ops team knew they needed to enhance data observability to address insufficient data governance and management.

This case study based on feedback from ThoughtSpot’s Aby Jerin, Principal Data Engineer, and Kaushik Patel, SVP Business Operations, details how Revefi helped ThoughtSpot build data trust with business users and reduce cloud data platform costs (CDP) by 30% while increasing CDP usage by 35%, resulting in a six-figure savings.

The Lowdown

ThoughtSpot’s prime concerns arose from general awareness of gaps in data pipelines and processing. Their data ecosystem has been expanding and growing more complex, which inevitably led to the growing number of data artifacts that could skew metrics calculation. Each new data source integration led to a more complex calculation of corresponding metrics. Those transformations required data integrity and coherence throughout its evolution in ETL and reverse ETL pipelines. 

Pinpointing the origin of those artifacts and outliers was ThoughtSpot’s primary objective in order to keep the outputs consistent and reliable for business users decision-making.

“We found that there are gaps in this data stack in terms of a solution that will solve the data governance and management problem. We identified Revefi as one that would solve a number of different issues on that front for us, hence the choice of selecting Revefi.” – said Kaushik Patel. 

Therefore ThoughtSpot’s main goals were to:

• Reduce time and cost of data issues debugging process
• Automate manual coding to unburden data engineers
• Observe who exactly uses particular data assets and what for
• Identify which missing data or broken pipelines affect downstream users
• Ensure data integrity and cleanliness to get clear and actionable insights from data mining
• Strengthen the trust between the data team and stakeholders from those meaningful insights.
  ‍

The Deep Dive

The Roadblocks

The key challenge for ThoughtSpot data engineers was to layer data governance and management interface upon the complex data ecosystem they have been operating for years. So, they needed a data monitoring product that would seamlessly fall into the existing architecture and provide informative exploration dashboards.

While researching available data governance solutions, the ThoughtSpot team set their minds on finding the data co-pilot that integrates into the existing data stack instantly. They prioritized the ability to identify and fix data issues without a lengthy debug process. Thus, they chose Revefi as the best-of-breed Data Operations Cloud transforming data observability and ensuring ultimate data governance and management efficiency. This way, ThoughtSpot became an early adopter of Revefi. It’s been using Revefi for about six months now.

Before embracing Revefi, ThoughtSpot:

• faced a lot of flagged data issues and broken data pipelines
• spent a long time manually finding and fixing these problems
• found that the ROI on such efforts was very long.
  ‍

Revefi offered the solution for all ThoughtSpot’s concerns and also helped with the following:

• Data freshness issues
• Ambiguity in understanding the role of data artifacts in the outputs
• Excessive or anomalous data to tackle first-place
• Estimate the ROI of putting the team’s efforts into data issue fixes
• Over-consumption of data warehouse resources.

‍
The Specifics

Revefi-powered top-to-bottom data observability and data governance yielded the ThoughtSpot data ops team tons of benefits. The most note-worthy are:

• Fast and Simple Setup. When ThoughtSpot looked at all the other enterprise tools out there in the data space, it saw that such tools involved a lot of upfront configuration work. The small data team at ThoughtSpot didn’t have the time to contend with that, said Jerin. But with Revefi, Jerin notes, there is no learning curve. ThoughtSpot just went to the setup screen, input its credentials, and clicked connect. Revefi took care of everything else, and it was all automated. The setup process of the Revefi Data Operations Сloud was insanely easy, and the data team received the first data health monitoring alerts right after Revefi consumed metadata.
• Insightful Data Governance and Management. ThoughtSpot’s team obtained a full-fledged tool for identifying the root causes of data issues. They also discovered what unnecessary access permissions provoked the overuse of data artifacts and plugged the loopholes that could endanger data security.
• Enhanced Data Integrity. It became fairly simple for the data quality team to detect and correct outdated and misplaced data points. They finally managed to track the use scenarios of particular assets and decide whether some of them should be terminated, updated, or have discoverability problems.
• Data Freshness. Revefi ensured real-time updates on the freshness of ThoughtSpot’s data. With Revefi, ThoughtSpot’s team gets alerted when something is wrong, such as when artifacts are affected by missing data or a broken pipeline. AI-powered data governance and management eliminated tons of manual effort.
• Automatic, Instant Access to Data Lineage. Within moments, Revefi had all lineage details ready for ThoughtSpot. Before adopting Revefi, it could have taken ThoughtSpot about 24 hours to get that information and solve a data issue. But now they can solve issues in just 15 minutes.
• Improved Trust from the Board Members. Adopting Revefi for data governance and management strategy brought better visibility into all cloud data-related processes and unique insights into data performance & use, thus fostering enterprise-wide acceptance of data strategies alongside with truly data-driven decision making.
• Cost and Spend Management Ensuring Higher Data Warehouse ROI. Owing to optimized data access and data cleansing practices, the company cut 30% of cloud data platform costs while boosting its usage by 35%. Kaushik acknowledges that Revefi has enhanced his awareness of spending patterns, providing insights into processes and workflows that consume the budget in vain.
  ‍

The Outcome

Revefi is a one-stop solution for data governance, data quality, and cost management. It empowers ThoughtSpot to be more strategic in how it works with and plans for data.  

Revefi automates ThoughtSpot’s systems and makes them work much faster and easier. If there are unwanted users, Revefi points it out. ThoughtSpot experienced such an incident just recently, and after receiving the alert from Revefi, it revoked that user’s permission right away. A bulletin from Revefi recently revealed that ThoughtSpot had exhausted its credits. After receiving this news, ThoughtSpot was able to act to address that issue within five minutes. Revefi also helps companies like ThoughtSpot avoid running out of credits and stay on course. And the 30% in savings that Revefi enabled yielded ThoughtSpot a six-figure cost savings!

With Revefi, Jerin says, he and everybody else at ThoughtSpot sleep peacefully at night.

“Now that the cost management is on track with Revefi, I think it has given us much better confidence in terms of putting together a trend line of how cost will evolve over time. And that has helped us with all of our budgeting and also making sure that we are not running into any surprises.” – Kaushik concludes by looking back on the significant business outcomes achieved with Revefi’s automated data observability and, as a result – data governance and management improvements.

On top of that, ThoughtSpot’s top executive is elated with the way Revefi skyrocketed the productivity of the data team. The 3X less time spent on data issues debugging is an incredible payoff and definitely worth investing in a monthly subscription.

Contact the Revefi team today and get started with the top benefits of data quality monitoring.