Cross-platform data cost optimization: Snowflake, Databricks, BigQuery in one pane

The Cross-Platform Cost Question Single-Platform Tools Cannot Answer

On April 16, 2026, SELECT — recently acquired by DoiT — announced the Early Access Program for its automated BigQuery cost optimization platform. The announcement closes the loop on a cross-platform data-platform cost-optimization roadmap that has been clear since the acquisition: Snowflake live today, Databricks coming June 2026, BigQuery generally available in Q3. SELECT’s automation engine has been proven in production across more than $250M in Snowflake spend.

The product picture is interesting on its own. The category picture is the bigger story. Modern data teams in 2026 do not run a single warehouse. The same enterprise typically runs Snowflake for the analytics warehouse layer, Databricks for the lakehouse and ML feature-store workloads, and BigQuery for the marketing / product-analytics workloads tied to GCP’s downstream services. A second tier of teams runs ClickHouse, DuckDB, or StarRocks for specific high-cardinality or low-latency cases. The cost question is no longer “how do I optimize Snowflake?” — it is “how do I balance spend across three or four engines that share a workload graph?”

Single-platform cost tools cannot answer that question. They report, optimize, and right-size inside their own engine, which leaves the cross-platform routing decisions on a platform-engineering team that doesn’t have time to make them. SELECT by DoiT’s roadmap is a strong signal that the cost-optimization category is converging on cross-platform — but the convergence is still a roadmap, not a finished product.

What the 2026 Spend Picture Looks Like

Three observations frame the 2026 cost picture across the three engines:

• Snowflake’s pricing model rewards careful auto-suspend tuning, materialized-view selection, and clustering keys. A typical mid-market warehouse runs 40–60% wasted compute against the available knob set. The SELECT base of $250M+ in optimized Snowflake spend translates that into $100M+ in identified savings before any human review.
• Databricks’ pricing model rewards cluster autoscaling tuning, pool reuse, photon-vs-classic compute selection, and serverless-vs-classic routing. A typical lakehouse runs another 30–50% wasted compute against the same knob set. Databricks’ acquisition of Tabular and the unification of the Iceberg / Delta storage layer in 2025 means the cost-optimization knob surface area is wider in 2026 than it was twelve months ago.
• BigQuery’s pricing model rewards slot reservation tuning, materialized-view selection, partition-and-cluster table layout, and the routing of ad-hoc query traffic between on-demand and capacity slots. Slot reservations changed shape across 2025 with the introduction of autoscaling reservations, which makes the cross-platform comparison materially harder than it was when reservations were a flat commitment.

Across all three engines, the same workload — a 90-day rolling user-cohort aggregation, for example — can run at materially different cost depending on which engine is asked to run it, which storage layer the data sits in, and what time of day the query runs. The cross-platform cost question is real, and it is large enough to justify a dedicated agent.

The Workload-Class Decision Matrix

In 2026, the cross-platform routing decision falls into a small set of workload classes:

• Interactive analytical queries (BI dashboards, ad-hoc analyst sessions). Snowflake and BigQuery are both strong; the choice depends on which engine the data already sits in and the query-concurrency pattern. BigQuery slot reservations win for predictable concurrency; Snowflake’s per-warehouse caching wins for workloads that hit the same hot tables repeatedly.
• Batch ETL and data-engineering pipelines. Databricks lakehouse compute or Snowflake compute, depending on whether the pipeline is open-table-format-native (Iceberg / Delta) or warehouse-native. Databricks photon clusters win for heavy-Spark workloads; Snowflake task-and-stream pipelines win when the pipeline is already SQL-shaped.
• ML feature engineering and feature-store materialization. Databricks lakehouse plus the Databricks feature store wins for ML-native workflows; Snowflake’s Snowpark wins when the team is already SQL-first and the ML overhead is low.
• Embedding generation and vector search. Lakehouse compute (Databricks) for the generation step; specialized vector indices (pgvector, Pinecone, Weaviate, OpenSearch) for the serving step. The cross-platform question here is whether to write the embeddings back into the warehouse or only into the vector index.
• Marketing and product analytics tied to ad-platform feeds. BigQuery wins by default — the GA4 / GAds / Display & Video 360 export targets are BigQuery-native. The cross-platform optimization question is when to copy or replicate the BigQuery output into Snowflake or Databricks for downstream cross-platform joins.

A cross-platform cost agent runs the routing decision continuously, against the actual workload’s recent spend and runtime profile, rather than against the team’s internal lore about which engine “should” run a given workload class. That is the structural lift SELECT by DoiT is targeting and the same lift IAN’s cost agent runs across the broader cloud footprint.

What SELECT by DoiT Adds — And Where the Gap Stays

SELECT by DoiT’s product is purpose-built for the data-platform layer and benefits from the DoiT acquisition’s broader cloud-cost expertise. The April 16 announcement makes three things clear:

• The automation engine — the scoring and remediation primitive that ranked Snowflake spend across $250M+ in customer base — is portable across engines. The Snowflake-to-Databricks-to-BigQuery sequencing is a deliberate productization choice, not a fundamental architectural rewrite.
• The cost-observability layer is the entry point. Customers see the cross-platform spend picture before any optimization action lands. That is the right wedge — it removes the “which engine is this from?” classification problem before any decision has to be made.
• The product remains data-platform-only. Compute spend on the application layer (Kubernetes, EC2, Lambda, container services), object storage spend (S3, GCS, ADLS), egress fees, and AI-inference spend (the inference-engine layer covered in batch-twenty’s self-hosted-llm-inference-engine-cost-optimization post) are out of scope.

The gap that stays is the cross-layer correlation: data-platform spend goes up because an upstream Lambda function is logging too much, because an embedding pipeline is duplicating writes into both the warehouse and the vector index, or because an ML feature store is materializing features the downstream model has stopped using. Those answers do not live inside the data-platform layer — they live in the operational workflow that crosses application, warehouse, and AI-inference spend.

See the IAN team run on your cloud. We connect to your AWS account via a scoped read-only role, run the Observe-tier agents, and leave you with a concrete audit report — cost waste, security exposure, compliance gaps, and a labor-offset estimate. You keep the findings regardless of next steps. Get a free infrastructure audit →

Cross-Platform Patterns Worth Codifying

Independent of any single tool, three cross-platform patterns are worth codifying inside the platform team’s runbook:

• A daily cross-platform spend digest. Snowflake credit consumption, Databricks DBU consumption, and BigQuery slot consumption, all normalized to a dollar figure, alongside the equivalent week-prior numbers. The digest catches drift before it compounds. SELECT, Vantage, Cloudability, and the FOCUS-conformant data feeds (covered in this batch’s third post) are the input layer.
• Workload-class tagging at write time. Tag every workload at runtime with its class (bi_interactive, etl_batch, feature_engineering, embedding, marketing_analytics). Tag enforcement at write time is the only reliable way to keep the workload-class data model accurate over a multi-month window. Without it, the cross-platform routing decision degrades to gut feel within a quarter.
• A quarterly cross-platform routing review. Once a quarter, walk the workload graph and ask: which workloads are running on the engine they would have been routed to if the team were starting fresh today? The answer is rarely “all of them.” The review is also the natural venue to decommission the long tail of dashboards and pipelines that nobody is consuming any more.

All three are mechanical; none of them require a specific vendor. SELECT by DoiT, IAN’s cost agent, and the in-house equivalent each do parts of the work.

Capability Tiers Mapped to Cross-Platform Cost Optimization

The Observe / Operate / Administer capability-tier model maps cleanly to cross-platform data-platform cost optimization:

• Observe. Continuous spend ingestion across Snowflake, Databricks, BigQuery, ClickHouse, and the application-layer compute / storage / egress feeds. FOCUS-conformant feeds where available. Workload-class classification, slow-query detection, materialized-view candidate detection, slot-reservation utilization, cluster-autoscaling-ratio tracking. Auto-execute, fully audited.
• Operate. Auto-suspend tuning against pre-approved windows, materialized-view creation against pre-approved tables, clustering key application against pre-approved partitions, photon-vs-classic compute routing for Databricks, BigQuery slot-reservation rebalancing, and ad-hoc-vs-reserved query routing. Auto-execute for reversible / scoped actions; gated for organization-wide commitments.
• Administer. Reserved-capacity commitments, workload re-platforming (moving a workload from Snowflake to Databricks or vice versa), and any spend-control policy that affects organizational FinOps governance. Always requires explicit approval.

The mapping is the difference between a data team that reviews the warehouse bill once a quarter (and finds 40% waste each time) and one that runs continuous optimization on the active operational layer.

How IAN Helps: The Cost Agent on the Active Operational Layer

IAN is the AI DevOps team for cloud infrastructure, delivered as a coordinated team of specialized agents on the active operational layer. The cost agent runs continuous spend monitoring across every connected cloud account, including the data-platform layer (Snowflake, Databricks, BigQuery via FOCUS-conformant feeds and direct connectors), the application-compute layer, the storage layer, the egress layer, and the AI-inference layer.

Workload-class classification lands in the audit trail. Cross-platform routing decisions surface as proposals scored against the actual workload’s recent runtime and spend profile. Reversible Operate-tier actions execute automatically when policy permits — auto-suspend tuning against pre-approved windows, materialized-view creation against pre-approved tables, slot-reservation rebalancing, ad-hoc-vs-reserved routing. Irreversible or organization-wide actions escalate to the Administer tier with separation-of-duties enforced.

Pricing is BYOK and usage-based with a monthly minimum. Customers bring their own model keys (Claude, OpenAI, or another provider) and pay inference cost directly to their model vendor. IAN charges for the orchestration layer, per agent action, per cloud account, per operation class. Usage scales with the volume of cost-optimization actions taken, which is itself a strong signal of where the team is recovering material spend.

The Three-Phase Rollout

Phase 1 — Stand up the cross-platform Observe layer. Wire continuous spend ingestion across Snowflake, Databricks, BigQuery, the application-compute layer, the storage layer, and the AI-inference layer into the cost agent’s read path. Two-to-four weeks for a team with reasonable platform-engineering capacity.

Phase 2 — Codify the workload-class taxonomy and the routing decision rules. Tag every workload at write time with its class. Pre-authorize the auto-suspend / materialized-view / slot-reservation / clustering-key scopes the cost agent can act on. Run a tabletop exercise against the team’s actual workload graph and measure recovered spend over the first 90 days. Two-to-three months of pattern tuning.

Phase 3 — Cross the cost / resource / deployment agent loop. Cross-platform cost is also a resource-agent question (tagging, lifecycle, quota) and a deployment-agent question (which workloads are scheduled where). Once the cost agent’s Observe layer is live, its signals feed the resource-agent’s tagging-and-quota loop and the deployment-agent’s release-orchestration loop. Each agent’s outputs make the others more accurate.

The combined effect is the same the platform team would get from hiring a senior data-platform engineer and a senior FinOps analyst, except the agents run continuously, surface drift before the next monthly bill lands, and produce an immutable audit trail that satisfies internal audit. For most mid-market data teams, that is materially better than the current state.

SELECT by DoiT is one signal among several that the data-platform cost-optimization category is converging on cross-platform. The category is right; the next phase is binding the data-platform layer into the broader cross-cloud cost picture.

Get a free infrastructure audit → | See pricing →