We are building a next-generation Cloud Data Platform to unify data from Policy, Claims, Billing, and Administration systems into a single source of truth. We are seeking a Lead Cloud Data Engineer who will be 75% hand-on and play a critical role in designing, building, and optimizing our modern data ecosystem leveraging Medallion architecture, Delta Lake, and modern data warehouse technologies such as Snowflake, Synapse, or Redshift.
As a technical leader, the Lead Data Engineer will define and execute the end-to-end data engineering strategy from data ingestion and modeling to governance and performance optimization enabling scalable, high-quality, and analytics-ready data assets. This role requires hands on deep technical expertise in cloud-native data engineering, automation, and architecture design, coupled with strong leadership to mentor teams and align solutions with business goals.
Responsibilities:
Data Platform Design & Architecture
- Define the strategic roadmap for the enterprise data platform, ensuring scalability, performance, and interoperability across business domains.
- Architect and implement cloud-native, Medallion-based data architectures (Bronze–Silver–Gold layers) for unified and governed data delivery.
- Drive standardization of data models, pipelines, and quality frameworks across Policy, Claims, Billing, and Administrative data assets.
- Evaluate and implement emerging data technologies to strengthen the platform’s performance, cost efficiency, and resilience
Data Integration & Ingestion
- Design, build, and optimize high-performance ingestion pipelines, using AWS Glue, Databricks, or custom Spark applications.
- Automate ingestion of structured, semi-structured, and unstructured data from APIs databases, and external data feeds.
- Tune and monitor ingestion pipelines for throughput, cost control, and reliability across dev/test/prod environments.
Data Transformation & Modeling
- Hands on Development of ETL/ELT pipelines using Databricks or similar frameworks to transform raw data into curated and consumption-ready datasets.
- Design and develop relational, Vault, and dimensional data models to support analytics, BI, and AI/ML workloads.
- Define and enforce data quality standards, validation frameworks, and enrichment rules to ensure trusted business data.
- Apply data quality, cleansing, and enrichment logic to ensure accuracy and completeness of business-critical data.
Cloud Infrastructure, Automation and Performance
- Collaborate with DevOps and Cloud Engineering teams to design automated, infrastructure-as-code environments using Terraform, CloudFormation, or equivalent tools.
- Implement CI/CD pipelines for data pipeline deployment, versioning, and testing.
- Lead performance tuning and scalability optimization to ensure highly available, cost-efficient data platform.
Governance, Security & Compliance
- Implement and enforce data governance, cataloging, and lineage practices using tools such as Purview, Alation, or Collibra.
- Partner with InfoSec to implement data privacy, access control, and compliance frameworks aligned with regulatory standards.
- Drive consistency and accountability in data stewardship across business and IT teams.
Leadership, Collaboration & Mentorship
- Lead a team of data engineers, providing technical guidance, coaching, and performance mentorship.
- Collaborate with Data Architects, Analysts, and Business Leaders to align data solutions with enterprise strategy.
- Promote a culture of engineering excellence, reusability, and knowledge sharing across data organization.
- Influence enterprise-wide standards for data engineering, automation, and governance.
