AI Platform Lead Engineer

New

Skills

Airflow BigQuery Dataflow Pyspark Python Sql

Join AssemblyAI as a Lead AI Platform Engineer and drive the development of scalable, future-proof data platforms optimized for cutting-edge AI research. This fully remote position is available to candidates based in the United States. You will design and implement efficient data pipelines, manage large-scale ML infrastructure, and lead the adoption of advanced tooling to accelerate team performance in a fast-paced startup environment.

Key Responsibilities
  • Design and build scalable data platforms optimized for AI research workloads.
  • Develop efficient data pipelines using advanced GCP services.
  • Implement cost-effective storage and monitoring solutions for large-scale ML operations.
  • Optimize resource allocation and management for maximum training efficiency.
  • Lead adoption of cutting-edge ML tools and streamline workflows to reduce complexity.
  • Enhance tooling and documentation to improve team velocity.
  • Implement guardrails for cost, quality, and performance.
  • Participate in on-call rotation to ensure system reliability.
  • Identify and eliminate technical bottlenecks in training pipelines.
Required Skills & Qualifications
  • 8+ years of experience in AI/ML infrastructure or research platform engineering.
  • 3+ years in an AI data and infrastructure or similar role.
  • Strong proficiency in Python and SQL.
  • Expertise with GCP services including BigTable, BigQuery, Dataproc, and Dataflow.
  • Experience with distributed processing frameworks (e.g., Apache Beam, PySpark).
  • Familiarity with workflow orchestration tools (e.g., Airflow, Composer, Astronomer).
  • Understanding of distributed training systems and data loading optimization.
  • Experience with experiment tracking and training tooling.
  • Ability to thrive in a dynamic, fast-paced startup environment.
  • Excellent problem-solving and communication skills.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: 12 Months

Share this job:

Similar Jobs

Generative ML Engineer Role

New

Design and scale generative AI infrastructure

Develop and fine-tune generative video and visual models

Airflow Engineer Machine Learning Prompt Engineering

Data Engineer Crypto Platform

New

Design scalable data infrastructure

Build and maintain high-quality data pipelines

Airflow Big Data Engineer Kafka

Data Engineer, Solar Solutions

Posted 3 days ago

Enable data-informed decision-making organization-wide

Design and implement scalable cloud-based ETL/ELT solutions

Airflow Lambda NoSQL Pyspark

Remote Staff Data Engineer

Posted 4 days ago

Hire a remote data engineer

Build automated communication systems

Airflow AWS Data Data Warehousing

Data Governance Engineering Lead

Posted 10 days ago

Lead global data governance strategy and execution

Build and maintain governance-aware data pipelines

Airflow Architecture Databricks Infosec

Wikimedia Data Engineer Role

Posted 11 days ago

Design and maintain scalable data pipelines

Ensure data quality and reliability

Airflow CI/CD Hadoop hive

Senior ML Engineer, Ads

Posted 13 days ago

Lead end-to-end ML ad targeting product development

Drive technical research and strategic roadmap

Airflow BigQuery Deep Learning Machine Learning

Compliance Data Analyst Role

Posted 20 days ago

Develop and automate compliance dashboards and reports

Support regulatory reporting and audit readiness

Airflow AWS Data Analysis Data Analyst

BI Analyst – Sporty Group

Posted 21 days ago

Deliver actionable business insights

Develop and optimize data pipelines

A/b Testing Ab testing Airflow AWS

Senior Solutions Architect Role

Posted 22 days ago

Design and implement scalable data solutions

Lead and mentor engineering teams

Airflow AWS Azure Hadoop

Senior Data Engineer Role

Posted 33 days ago

Design and build scalable data architectures

Lead customer-facing technical engagements

Airflow Big Data Engineer Google Cloud Platform

Senior Data Engineer Role

Posted 34 days ago

Architect scalable and reliable data pipelines

Develop and launch self-serve analytics products

Airflow AWS Databricks Engineer

Senior Data Engineer Wikimedia

Posted 37 days ago

Design and maintain scalable data pipelines

Ensure data quality and governance

Airflow Docker Hadoop hive

Senior Data Engineer Wikimedia

Posted 38 days ago

Develop and maintain scalable data pipelines

Ensure data quality and robust monitoring

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 38 days ago

Design and build scalable data pipelines

Ensure data quality and system reliability

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 38 days ago

Develop robust, scalable data pipelines

Ensure high data quality and governance

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 38 days ago

Build scalable and robust data pipelines

Enhance data quality and governance

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 38 days ago

Develop scalable and robust data pipelines

Ensure high data quality and system reliability

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 43 days ago

Design and build scalable data pipelines

Ensure data quality and robust monitoring

Airflow Docker Engineer Hadoop

Wikimedia Data Engineer Role

Posted 43 days ago

Develop scalable and robust data pipelines

Enhance data quality through monitoring and alerting

Airflow CI/CD Engineer hive

Lead Data Engineer Role

Posted 44 days ago

Lead and architect scalable data platforms and pipelines

Apply modern software engineering and cloud-native principles

Airflow Aws glue Devops Docker

Wikimedia Data Engineer Role

Posted 44 days ago

Build and maintain scalable data pipelines

Ensure data quality and governance across systems

Airflow CI/CD Engineer Hadoop

Staff ML Engineer, Apollo

Posted 49 days ago

Lead development of scalable ML systems

Advance Apollo's AI-native product features

Airflow Architecture Engineer Machine Learning

BI Analyst – Sports Betting

Posted 53 days ago

Mine and analyze large-scale business data

Develop and maintain dashboards and reports

A/b Testing Ab testing Airflow Analyst

Staff ML Engineer, Apollo

Posted 80 days ago

Lead and scale ML-driven product features

Develop and optimize AI-first user experiences

Airflow Databricks Engineer Machine Learning

Senior Data Science Manager

Posted 90 days ago

Lead and mentor a data science team

Integrate analytics into business strategy

Airflow Amplitude AWS Databricks

Cloud Data Solutions Architect

Posted 92 days ago

Lead and manage modern cloud data platforms for clients.

Provide architectural guidance and operational support.

Airflow AWS Azure CI/CD

Cloud Data Solutions Architect

Posted 95 days ago

Lead operation and management of cloud data platforms

Provide architectural guidance and technical leadership

Airflow AWS Azure CI/CD

Senior Data Engineer Wikimedia

Posted 97 days ago

Design and build scalable data pipelines

Ensure data quality and robust monitoring

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 97 days ago

Design and build scalable data pipelines

Ensure and monitor data quality

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 97 days ago

Develop and maintain scalable data pipelines

Ensure data quality, governance, and lineage

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 97 days ago

Design and develop scalable data pipelines

Enhance data quality, governance, and lineage

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 97 days ago

Design and maintain scalable data pipelines

Ensure data quality, governance, and lineage

Airflow Docker Engineer Hadoop

Senior Data Engineer Wikimedia

Posted 97 days ago

Design and scale data infrastructure

Ensure data quality and governance

Airflow Docker Engineer Hadoop

Junior Data Scientist Germany

Posted 97 days ago

Deliver actionable business insights

Collaborate across cross-functional teams

Airflow AWS Looker Pandas

Lead Data Engineer Role

Posted 134 days ago

Design and implement scalable data solutions

Lead and mentor engineering teams

Airflow AWS Databricks Java

Senior Data Engineer Africa

Posted 141 days ago

Ensure high-quality, reliable data management.

Automate data quality assurance processes.

Airflow AWS Lambda Postgresql

Junior Data Engineer Addepto

Posted 144 days ago

Develop scalable data processing platforms

Design and optimize data pipelines

Airflow AWS Docker Java

Senior Data Engineer

Posted 166 days ago

Architect scalable data models and ETL pipelines

Design and launch self-serve analytics products

Airflow AWS Big Data Databricks

Senior ML Engineer II at Apollo

Posted 168 days ago

Build and productionize Machine Learning models for Apollo products

Optimize users' experience at all stages of their product journey

Ai Systems Airflow Cloud Computer science

Staff Data Engineer

Posted 171 days ago

Design and build efficient data pipelines for large volumes of data

Optimize transformation models and data pipelines

Airflow Python Redshift Sql

Technical Delivery Manager

Posted 188 days ago

Lead and motivate client technical teams for modern data platforms

Maintain knowledge of modern data technology for best practices

Airflow Architecture AWS Azure

Data Sales Automation Platform

Posted 198 days ago

Enhancing media sales innovation through automation and intelligent proposals

Driving growth and optimization for media companies and agencies

Airflow Data Modeling Data Warehousing Etl Processes

ML Engineer on Apollo Team

Posted 198 days ago

Build and deploy ML models for Apollo products.

Enhance user experience through data-driven insights.

Airflow Cloud Computer science Databricks

Staff Analytics Engineer

Posted 203 days ago

Enable informed decision-making through accessible data

Lead data vision and architecture for impactful insights

Airflow Data Engineering Data Modeling Etl

Senior Data Engineer Project

Posted 206 days ago

Develop data infrastructure and systems for various business functions

Implement data observability and monitoring

Airflow Big Data Data Security Devops

Data Lead: ETL & Analytics

Posted 213 days ago

Improve data infrastructure Optimize performance and accessibility Enable data-driven

g Collaborate with cross-functional teams Mentor and lead data

Airflow Analytics AWS BigQuery

Data Engineering Lead

Posted 213 days ago

- Manage platform APIs and AI capabilities - Oversee data system scalability and performance -

borate with data science and product teams - Implement AI and ML models into the platform - Ensure

Ai/ml Airflow AWS BigQuery

OpenSC Remote Jobs

Posted 214 days ago

- Enhance data solutions for sustainable food systems - Lead customer onboarding and supply chain

implementations - Transform sustainability goals into actionable solutions - Drive product

Airflow AWS Cloud Docker
overtime