Lead Databricks Engineer (Contract)

Location: Houston, TX - The Galleria (100% Onsite)

Contract Length: 12 months (extensions likely)

Experience: 8 years

Overview

We are seeking a Lead Databricks Engineer to support large-scale data platform initiatives focused on data modernization, cloud migration, and advanced analytics. This is a hands-on senior role requiring deep Databricks expertise, strong AWS experience, and the ability to partner closely with business and technical stakeholders.

The ideal consultant has led complex Databricks implementations end-to-end, can code at a high level, and has experience modernizing legacy data platforms into scalable, cloud-native architectures.

Key Responsibilities

Lead the design and implementation of enterprise-scale Databricks solutions on AWS
Drive data modernization initiatives, including lift-and-shift and re-architecture of legacy data platforms
Build and optimize data pipelines using Python, PySpark, and SQL
Design and manage Delta Lake architectures and implement Unity Catalog for governance and access control
Develop and support streaming and batch data workloads
Configure and optimize Databricks clusters and serverless compute for performance and cost
Integrate Databricks with upstream and downstream systems (APIs, data sources, analytics tools)
Partner with stakeholders to gather user and business requirements and translate them into technical solutions
Implement observability, monitoring, and cost controls, including usage, volume, and pricing metrics
Support data domains related to billing, accounting, pricing, and volume metrics
Provide technical leadership, best practices, and mentorship to engineering teams

Required Skills & Experience

8 years of experience in data engineering, with recent hands-on focus on Databricks
Strong experience deploying Databricks on AWS
Advanced coding skills in Python, PySpark, and SQL
Deep knowledge of Delta Lake, Unity Catalog, and Databricks workspace governance
Experience with streaming data (Structured Streaming, Kafka, or similar)
Strong understanding of Databricks cluster management, serverless compute, and performance tuning
Databricks certifications
Experience integrating Databricks with enterprise systems and data sources
Proven ability to work directly with business and technical stakeholders
Experience supporting financial data domains (billing, accounting, pricing, usage metrics) is highly preferred
Strong communication skills and ability to lead technical discussions
Experience with AKS / Kubernetes environments

Nice to Have

AWS certifications
Consulting or contracting background in large enterprise environments

Work Requirements

100% onsite role in Houston, TX
Must be authorized to work in the United States

Principal Databricks Engineer

Job Description

Recent Jobs.