AI Platform Lead Engineer

New

Skills

Airflow BigQuery Dataflow Pyspark Python Sql

Join AssemblyAI as a Lead AI Platform Engineer and drive the development of scalable, future-proof data platforms optimized for cutting-edge AI research. This fully remote position is available to candidates based in the United States. You will design and implement efficient data pipelines, manage large-scale ML infrastructure, and lead the adoption of advanced tooling to accelerate team performance in a fast-paced startup environment.

Key Responsibilities
  • Design and build scalable data platforms optimized for AI research workloads.
  • Develop efficient data pipelines using advanced GCP services.
  • Implement cost-effective storage and monitoring solutions for large-scale ML operations.
  • Optimize resource allocation and management for maximum training efficiency.
  • Lead adoption of cutting-edge ML tools and streamline workflows to reduce complexity.
  • Enhance tooling and documentation to improve team velocity.
  • Implement guardrails for cost, quality, and performance.
  • Participate in on-call rotation to ensure system reliability.
  • Identify and eliminate technical bottlenecks in training pipelines.
Required Skills & Qualifications
  • 8+ years of experience in AI/ML infrastructure or research platform engineering.
  • 3+ years in an AI data and infrastructure or similar role.
  • Strong proficiency in Python and SQL.
  • Expertise with GCP services including BigTable, BigQuery, Dataproc, and Dataflow.
  • Experience with distributed processing frameworks (e.g., Apache Beam, PySpark).
  • Familiarity with workflow orchestration tools (e.g., Airflow, Composer, Astronomer).
  • Understanding of distributed training systems and data loading optimization.
  • Experience with experiment tracking and training tooling.
  • Ability to thrive in a dynamic, fast-paced startup environment.
  • Excellent problem-solving and communication skills.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: 12 Months

Share this job:

Similar Jobs

Developer Advocate

Posted 9 days ago

Support developers in the DataHub Slack community.

Create compelling technical content to educate users about DataHub.

Airflow Apache Kafka Communication Community engagement

Developer Advocate

Posted 9 days ago

Empower developers through technical communication and content creation.

Engage actively in the DataHub community to support users.

Airflow Apache Kafka Communication Community engagement

Lead Data Engineer Role

Posted 12 days ago

Architect scalable, secure data platforms.

Implement modern software engineering practices.

Airflow Devops Docker Engineer

Python Kubernetes Engineer

Posted 23 days ago

Build open source AI/ML and analytics solutions

Develop and maintain scalable data platforms

Airflow Analytics Docker Engineer

Crypto Data Engineer Platform

Posted 25 days ago

Architect scalable data pipelines and infrastructure.

Enable real-time, reliable, and high-quality data access.

Airflow Big Data Engineer Kafka

Senior Solutions Architect Role

Posted 31 days ago

Design and implement scalable data architectures

Lead and mentor engineering teams

Airflow AWS Azure Databricks

Developer Advocate, DataHub

Posted 42 days ago

Empower and support DataHub developers

Create educational technical content

Airflow Apache Kafka Communication Community engagement

Senior Solutions Architect Role

Posted 44 days ago

Design and implement scalable data solutions

Lead and mentor engineering teams

Airflow Azure Databricks Hadoop

Senior Solutions Architect Role

Posted 45 days ago

Architect and deliver scalable data solutions

Lead and mentor engineering teams

Airflow AWS Databricks Hadoop

Senior Data Engineer Role

Posted 47 days ago

Architect scalable and reliable data infrastructure

Empower data-driven decision making and analytics

Airflow AWS Big Data Engineer

Lead Data Engineer Role

Posted 47 days ago

Architect scalable data platforms and pipelines

Drive innovation in cloud-native product development

Airflow Aws glue Docker Engineer

Data Governance Engineering Lead

Posted 48 days ago

Lead data governance strategy and execution

Build and manage governance-aware data pipelines

Airflow Architecture Computer science Databricks

Generative ML Engineer Role

Posted 56 days ago

Design and scale generative AI infrastructure

Develop and fine-tune generative video and visual models

Airflow Engineer Machine Learning Prompt Engineering

Data Engineer Crypto Platform

Posted 57 days ago

Design scalable data infrastructure

Build and maintain high-quality data pipelines

Airflow Big Data Engineer Kafka

Data Engineer, Solar Solutions

Posted 59 days ago

Enable data-informed decision-making organization-wide

Design and implement scalable cloud-based ETL/ELT solutions

Airflow Lambda NoSQL Pyspark

Remote Staff Data Engineer

Posted 60 days ago

Hire a remote data engineer

Build automated communication systems

Airflow AWS Data Data Warehousing

Data Governance Engineering Lead

Posted 66 days ago

Lead global data governance strategy and execution

Build and maintain governance-aware data pipelines

Airflow Architecture Databricks Infosec

Wikimedia Data Engineer Role

Posted 67 days ago

Design and maintain scalable data pipelines

Ensure data quality and reliability

Airflow CI/CD Hadoop hive

Senior ML Engineer, Ads

Posted 68 days ago

Lead end-to-end ML ad targeting product development

Drive technical research and strategic roadmap

Airflow BigQuery Deep Learning Machine Learning

Compliance Data Analyst Role

Posted 75 days ago

Develop and automate compliance dashboards and reports

Support regulatory reporting and audit readiness

Airflow AWS Data Analysis Data Analyst

BI Analyst – Sporty Group

Posted 77 days ago

Deliver actionable business insights

Develop and optimize data pipelines

A/b Testing Ab testing Airflow AWS

Senior Solutions Architect Role

Posted 77 days ago

Design and implement scalable data solutions

Lead and mentor engineering teams

Airflow AWS Azure Hadoop

Senior Data Engineer Role

Posted 89 days ago

Design and build scalable data architectures

Lead customer-facing technical engagements

Airflow Big Data Engineer Google Cloud Platform

Senior Data Engineer Role

Posted 89 days ago

Architect scalable and reliable data pipelines

Develop and launch self-serve analytics products

Airflow AWS Databricks Engineer

Senior Data Engineer Wikimedia

Posted 92 days ago

Design and maintain scalable data pipelines

Ensure data quality and governance

Airflow Docker Hadoop hive

Senior Data Engineer Wikimedia

Posted 93 days ago

Develop and maintain scalable data pipelines

Ensure data quality and robust monitoring

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 93 days ago

Design and build scalable data pipelines

Ensure data quality and system reliability

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 93 days ago

Develop robust, scalable data pipelines

Ensure high data quality and governance

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 93 days ago

Build scalable and robust data pipelines

Enhance data quality and governance

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 93 days ago

Develop scalable and robust data pipelines

Ensure high data quality and system reliability

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 98 days ago

Design and build scalable data pipelines

Ensure data quality and robust monitoring

Airflow Docker Engineer Hadoop

Wikimedia Data Engineer Role

Posted 99 days ago

Develop scalable and robust data pipelines

Enhance data quality through monitoring and alerting

Airflow CI/CD Engineer hive

Lead Data Engineer Role

Posted 100 days ago

Lead and architect scalable data platforms and pipelines

Apply modern software engineering and cloud-native principles

Airflow Aws glue Devops Docker

Wikimedia Data Engineer Role

Posted 100 days ago

Build and maintain scalable data pipelines

Ensure data quality and governance across systems

Airflow CI/CD Engineer Hadoop

Staff ML Engineer, Apollo

Posted 104 days ago

Lead development of scalable ML systems

Advance Apollo's AI-native product features

Airflow Architecture Engineer Machine Learning

BI Analyst – Sports Betting

Posted 109 days ago

Mine and analyze large-scale business data

Develop and maintain dashboards and reports

A/b Testing Ab testing Airflow Analyst

Staff ML Engineer, Apollo

Posted 136 days ago

Lead and scale ML-driven product features

Develop and optimize AI-first user experiences

Airflow Databricks Engineer Machine Learning

Senior Data Science Manager

Posted 146 days ago

Lead and mentor a data science team

Integrate analytics into business strategy

Airflow Amplitude AWS Databricks

Cloud Data Solutions Architect

Posted 147 days ago

Lead and manage modern cloud data platforms for clients.

Provide architectural guidance and operational support.

Airflow AWS Azure CI/CD

Cloud Data Solutions Architect

Posted 150 days ago

Lead operation and management of cloud data platforms

Provide architectural guidance and technical leadership

Airflow AWS Azure CI/CD

Senior Data Engineer Wikimedia

Posted 152 days ago

Design and build scalable data pipelines

Ensure data quality and robust monitoring

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 152 days ago

Design and build scalable data pipelines

Ensure and monitor data quality

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 152 days ago

Develop and maintain scalable data pipelines

Ensure data quality, governance, and lineage

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 152 days ago

Design and develop scalable data pipelines

Enhance data quality, governance, and lineage

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 152 days ago

Design and maintain scalable data pipelines

Ensure data quality, governance, and lineage

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 152 days ago

Design and scale data infrastructure

Ensure data quality and governance

Airflow Docker Engineer Hadoop

Junior Data Scientist Germany

Posted 153 days ago

Deliver actionable business insights

Collaborate across cross-functional teams

Airflow AWS Looker Pandas

Lead Data Engineer Role

Posted 189 days ago

Design and implement scalable data solutions

Lead and mentor engineering teams

Airflow AWS Databricks Java

Senior Data Engineer Africa

Posted 197 days ago

Ensure high-quality, reliable data management.

Automate data quality assurance processes.

Airflow AWS Lambda Postgresql

Junior Data Engineer Addepto

Posted 200 days ago

Develop scalable data processing platforms

Design and optimize data pipelines

Airflow AWS Docker Java
overtime