Databricks connects local IDEs directly to its compute

Databricks has updated its developer experience by introducing a secure shell (SSH) tunnel that links local integrated development environments (IDEs) and command-line interfaces directly to Databricks compute resources. This update allows engineers to interactively run and debug Python and SQL workloads on real cluster infrastructure, including Serverless, AI Runtime, and dedicated clusters, without leaving their preferred local setups.

To establish the connection, developers can use the Databricks CLI. Running the command "databricks ssh connect" targets serverless compute, while appending the cluster ID flag targets dedicated clusters. For AI Runtime environments, developers can specify GPU accelerators using the accelerator flag, supporting options like GPU_1xA10 or GPU_8xH100. The SSH tunnel can also be initiated directly from the Databricks IDE extension or by appending IDE-specific flags for VS Code or Cursor.

This integration addresses long-standing pain points around managing non-Spark workloads and keeping local dependencies synchronized with the Databricks Runtime. The new system keeps files and project dependencies in sync and allows developers to browse data assets in the Unity Catalog directly from their IDE. Additionally, local AI coding assistants such as GitHub Copilot, Cursor, and Claude Code can now operate with full workspace context through the active tunnel.

The update also introduces administrative controls, such as the ability to attach a serverless usage policy via a specific flag to track costs by user or project. Looking ahead, Databricks plans to automatically configure its Unity AI Gateway for SSH tunnel users to govern access and spend. The company also intends to support custom Docker images and non-Python dependencies at startup, giving developers complete control over their remote environments.