Cutting costs with the new QUERY_ATTRIBUTION_HISTORY view in Snowflake.

I follow the Snowflake Data Superheroes on LinkedIn to get useful insights and updates about Snowflake. Today I found this post of Tomáš Sobotík introducing a new metric: QUERY_ATTRIBUTION_HISTORY.

I like following domain experts in the data world on LinkedIn. I can learn new things while indulging in the bad habit of scrolling through my LinkedIn feed.

If you're a Snowflake optimization enthusiast, you know how crucial it is to track and manage costs effectively. Whether you’re trying to optimize a query’s performance or reduce the financial footprint of your data processes, having clear visibility into query costs has always been a challenge. Until now.

The QUERY_ATTRIBUTION_HISTORY view as a game changer

Recently, Snowflake introduced a new system view called QUERY_ATTRIBUTION_HISTORY. This new view provides detailed credit consumption for individual queries, significantly simplifying cost analysis for users.

With this powerful view, you can now:

• Identify user-specific spend over time
• Calculate the cost of a single transformation pipeline
• Track the expense of repeated queries

Key Benefits

This system view can offer substantial benefits to teams looking to optimize their Snowflake usage. Here’s how:

1. Simplified Cost Tracking: You no longer need complex models to calculate query costs. The data is now available at a granular level.
2. Uncovering Hidden Expenses: Whether it's a rogue query or a poorly optimized pipeline, this tool helps you identify unexpected high spend, enabling quick fixes.
3. Customizable Dashboards: Integrate this data into your custom cost dashboards for real-time monitoring and management of expenses.

My test

I tried running the query below to spot the 5 most expensive queries I ran using the COMPUTE_WH warehouse:

This was the result:

From the QUERY_ID, I can dig deeper into the queries and see if there's something I can optimiz to reduce spending.

Important Considerations

While this new view is a fantastic addition, there are some caveats:

• There's still need to manually account for warehouse idle time, as it's not included.
• Serverless features, storage costs, and AI services are also excluded from this view.

Despite these limitations, QUERY_ATTRIBUTION_HISTORY is a critical asset for anyone looking to enhance cost efficiency in Snowflake.

Final Thoughts

Snowflake’s QUERY_ATTRIBUTION_HISTORY view provides a clearer window into query-level spending, giving you the power to optimize your data operations and prevent costly inefficiencies. By integrating this data into your analytics, you can gain better control over resource allocation and streamline operations for more cost-effective data management.

You can find out more by reading the official documentation page.

Banner Photo by Behnam Norouzi on Unsplash