Senior Big Data Engineer

Join H1 as a Senior Big Data Engineer and play a pivotal role in shaping the data infrastructure that powers global healthcare insights. Our mission is to deliver accurate, actionable medical data, fostering health equity and supporting transformative outcomes in patient care and drug development. As part of a collaborative, remote-first environment, you will design, scale, and optimize data systems that impact millions worldwide.

Job Overview

As a Senior Data Engineer, you will lead the development and maintenance of scalable data pipelines, ensuring data quality, reliability, and performance. You will work cross-functionally with product managers, engineers, and stakeholders to evolve our data platform, mentor team members, and drive engineering excellence.

Key Responsibilities

• Design, develop, and maintain scalable data extraction frameworks for structured and unstructured sources.
• Build and optimize robust ETL/ELT pipelines using big data technologies, particularly Apache Spark on AWS EMR.
• Transform, clean, and normalize complex datasets to ensure data quality and consistency.
• Lead integration efforts and collaborate with cross-functional teams to align technical solutions with business objectives.
• Monitor and troubleshoot data flows, proactively resolving performance and reliability issues.
• Maintain thorough documentation of systems, workflows, and processes.
• Participate in code reviews, mentor colleagues, and promote continuous improvement and best practices.

Required Skills & Qualifications

• 6+ years of experience in data engineering with large-scale data systems and pipelines.
• Proficiency in Python, Java, or similar programming languages.
• Strong SQL skills, including advanced queries, window functions, and complex joins.
• Hands-on experience with big data tools such as Apache Spark, preferably on AWS EMR.
• Familiarity with Docker or other containerization technologies.
• Understanding of Large Language Models (LLMs) and their applications.
• Basic knowledge of network, security, and encryption protocols like HTTP/HTTPS/TLS.
• Demonstrated ability to collaborate across teams and communicate effectively with technical and non-technical stakeholders.
• Strong analytical and problem-solving skills with a focus on data quality and system performance.
• Passion for clean, efficient code and adherence to engineering best practices.