Senior Principal Data Platform Engineer

What is this role?

The Senior Principal Data Platform Engineer is a strategic technical leader responsible for shaping the enterprise data platform vision and driving modernization initiatives. This role leads complex, cross-functional projects, including the migration from legacy Microsoft BI technologies to modern cloud-native platforms such as Databricks, while ensuring governance, compliance, and operational excellence. The position demands deep technical expertise, strong leadership, and a passion for building scalable, secure, and future-ready data solutions.

What will I be doing?

Platform Modernization & Migration

• Architect and lead large-scale migrations from MS BI stack (SSIS, SSAS, Power BI, SQL Server) to Databricks and other modern data platforms.
• Develop migration strategies that minimize downtime, ensure data integrity, and meet compliance requirements.
• Evaluate and implement tools and frameworks for seamless data movement and transformation.

Data Architecture & Engineering

• Design and maintain scalable, resilient data pipelines and ETL/ELT processes using Spark-based platforms (Databricks, Microsoft Fabric).
• Define reusable patterns and frameworks for efficient data transformation and integration.
• Own critical components of the enterprise data architecture, ensuring alignment with business objectives and scalability.

Governance, Security & Compliance

• Establish and enforce robust data governance practices, including data quality, lineage, privacy, and security.
• Ensure compliance with regulatory standards (e.g., GDPR, HIPAA) and internal policies.
• Advocate for best practices in data stewardship across teams.

DevOps, Source Control & CI/CD

• Implement and champion best practices for source control (GitHub, Azure DevOps) and CI/CD pipelines in data engineering.
• Integrate automated testing, code reviews, and deployment strategies to ensure reliability and maintainability.
• Drive adoption of Infrastructure-as-Code and automated provisioning for data environments.

Leadership & Collaboration

• Provide technical leadership and mentorship to data engineers and cross-functional teams.
• Partner with product managers, analysts, and engineering teams to deliver solutions that solve real business problems.
• Act as an evangelist for modern data strategies and continuous improvement.

Innovation & Continuous Learning

• Stay ahead of emerging technologies in distributed computing, data architecture, and cloud platforms.
• Lead evaluations and adoption of new tools and methodologies to keep the data platform future ready.

We'd love to hear from you if:

• Experience: 15+ years in data engineering or related fields, with at least 5 years in a senior/principal role.
• Migration Expertise: Proven track record of migrating legacy data warehouses and BI solutions to Databricks or similar platforms.
• Technical Skills:
• Strong SQL and dimensional modeling (Kimball, star/snowflake schemas).
• Expert-level MS SQL Server experience (including performance tuning, query optimization, and schema design.
• Hands-on experience with Spark, Databricks, and cloud data services.
• Familiarity with Microsoft BI stack (SSIS, SSAS, Power BI).
• Governance & Compliance: Deep understanding of data governance frameworks and regulatory compliance.
• DevOps: Expertise in source control, CI/CD, and automated testing in data engineering contexts.
• Leadership: Exceptional communication, problem-solving, and mentoring skills.
• Preferred: Experience supporting Data Science and ML initiatives.