Data Warehouse Automation & Real-time Data – Reducing Time to Value in a Distributed Analytical Environment
Creating Data Products in a Data Mesh, Data Lake or Lakehouse for use in Analytics (24-25 November 2022 – Amsterdam)
Centralised Data Governance of a Distributed Data Landscape (28-29 November 2022 – Live Streaming Event)
Centralised Data Governance of a Distributed Data Landscape (19-20 October 2022 – Live Streaming Event)
Smart Infrastructure & Smart Applications for the Smart Business – Infrastructure & Application Performance Monitoring
Centralised Data Governance of a Distributed Data Landscape (24-25 October 2022 – Live Streaming Event)
Creating Data Products in a Data Mesh, Data Lake or Lakehouse for use in Analytics (17-18 October 2022 – Live Streaming Event)
I attended the ThoughtSpot Beyond conference recently in Las Vegas, Nevada where the growing search-based BI platform vendor made a series of announcements to both solidify and strengthen its position in a mature BI platform market. The following summarises what they announced together with my analysis of these announcements recognising that we are in a market where analytics adoption continues to grow and where integration and cloud are increasingly important.
To start with they announced 2 new ThoughtSpot SaaS editions to add to their Enterprise Edition. These are:
- ThoughtSpot Team Edition
- ThoughtSpot Pro Edition
ThoughtSpot Team Edition is $95 per month for a single user-group, unlimited number of users and up to 5m rows of data connected to ThoughtSpot. ThoughtSpot Pro Edition is $2500 per month for up to 5 user groups with an unlimited number of users. This is a consumption edition based on actual queries and up to 100 m rows of data connected to ThoughtSpot.
The announcements of these two new editions provides a much lower cost entry point for prospective customers wanting to make use of ThoughtSpot while also opening it up to the small and medium size business (SMBs) market. It is also increases ThoughtSpot’s ability to ‘land and expand’ in larger enterprises. Overall, this addresses the complaint levelled at ThoughtSpot that Enterprise Edition sets a “high bar” for those organisations wanting to start using the product.
ThoughtSpot also announced broader connectivity with support for:
- AWS Redshift Serverless
- Snowflake data marketplace
- Databricks partner connect
- Starburst Galaxy
The focus of these new connectivity announcements is on cloud-based data sources. Partnerships with Databricks and Snowflake have also been strengthened. Companies that have subscribed to Snowflake data marketplace can now access live external marketplace data that is already loaded into Snowflake tables directly from ThoughtSpot. In addition, the Databricks partner connect announcement allows Databricks customers to launch ThoughtSpot from within the Databricks console. Setting these aside, connectivity to Dremio and Starburst Galaxy caught my eye as both these technologies provide data virtualisation capability and can simplify access to multiple data sources. This allows companies to break the dependency between physical schema in underlying data stores such as data warehouses and lakehouses and create virtual views in Dremio or Starburst that can be accessed by ThoughtSpot. Therefore, if the structure changes in underlying data sources, business users would not be impacted by any such changes. Also, if companies are using Starburst to create virtual data products (as per their Data Mesh positioning) then these data products can effectively be consumed by ThoughtSpot with this announcement.
ThoughtSpot also made a number of developer-oriented announcements at Beyond 2022. These include:
- CodeSpot – a searchable repository of reusable blocks of ThoughtSpot Modelling Language (TML) code
- ELT live analytics templates for SpotApps that run on Matillion
- The ability to access Matillion jobs from Matillion Exchange
- New 3rd party data blocks from 3rd party data providers
- Integration with dbt – to automatically translate dbt models into ThoughtSpot TML (i.e., automated TML generation)
- A number of new SpotApps
These announcements are clearly geared towards improving developer productivity with reusable TML code blocks, pre-defined ELT templates, 3rd party data blocks and integration with dbt. Deeper integration with Matillion and dbt helps customers that have those technologies to shorten time to value with ThoughtSpot. It also fits with the trend towards component-based development which started in software engineering some years ago and now finds its way into data and analytics. Nevertheless, while this is all good, component-based development can fall apart if it is not well organised and managed. The promise is agility, but productivity is as much about organisation of components and making things easy to find as it is about new technology capabilities. If companies are to maximise sharing and reuse then clear communication and organisation of all these code blocks, templates and data blocks is critical. This is not something any vendor can give you out-of-the -box. What rings in my ears is a question I was asked a few years ago by an executive which was “if everyone is coding, how exactly is that agile?”. Organisation of blocks and templates in a catalogue or marketplace and making them easy to find and reuse are critical to shortening time to value. This is something that should not be overlooked. Furthermore, version management as well as aligning and tagging code blocks, data blocks and templates with strategic business goals will also help understand what is being used where and for what business purpose. The last thing you want is a “wild west” leading to a component and template “graveyard”.
Operational analytics and decision management
ThoughtSpot also announced a new ThoughtSpot Sync capability. This is the integration of their SeekWell acquisition folded into ThoughtSpot.
For me, despite the other new capabilities, this was the announcement that really stood out for me at the conference. They announced the ability to automatically propagate insights from ThoughtSpot Liveboards and answers into other technologies such as Slack and other applications. The reason this is important is that it really enables companies to act on analytics. It is particularly exciting when you combine ThoughtSpot’s SpotIQ automatic analysis engine with ThoughtSpot Sync . The combination of automatically identifying insights that need to be acted upon and then providing the capability to automatically act on them takes ThoughtSpot into the world of continuous intelligent business. It goes beyond embedding analytics with ThoughtSpot Everywhere and opens up the opportunity for ThoughtSpot to proactively participate in automated decisions to continually improve business performance. Although the combination of SpotIQ and ThoughtSpot Sync is just in its infancy, this is something that could open up the way for customers to make use of ThoughtSpot to really shorten time to value. The more integration of SpotIQ and ThoughtSpot Sync with other applications the better. In particular I would loke to see integration with Microsoft Power Automate and equivalents. That would be extremely powerful. It is clear more is to come here which no doubt will be grabbed with both hands by customers. Looking at this, it is clear that customers should explore how to best utilise SpotIQ and ThoughtSpot Sync, ThoughtSpot Everywhere and ThoughtSpot Monitor capabilities (see below) to get maximise business value from the platform using automation.
Finally, ThoughtSpot announced:
- Bring Your own Charts
- Monitor – users get automated insights that alert them as changes occur
Bring Your own Charts plays to ThoughtSpot extensibility which is important for flexibility. The use of Monitor with custom designed insights and SpotIQ identified insights allows customers to use ThoughtSpot to keep a finger on the pulse of business while proactively driving alerts to optimise business performance and everyday operations.
Overall, this was a solid set of announcements by ThoughtSpot that provides a lower cost entry point, greater connectivity, new developer-oriented capabilities, greater integration with complementary technologies and actionable insights via embedding and automation