Senior Software Engineer, Inference Project

New

Skills

AWS Cloud infrastructure Distributed systems Gcp Kubernetes Python Rust

Build and maintain large-scale distributed inference systems. Develop routing and fleet orchestration across accelerators. Improve compute efficiency for growth and research. Collaborate across teams to ship models to users.

Key Responsibilities
  • Build and maintain large-scale distributed inference systems.
  • Develop routing and fleet orchestration across accelerators.
  • Improve compute efficiency for growth and research.
  • Collaborate across teams to ship models to users.
Required Skills & Qualifications
  • Significant software engineering experience, particularly with distributed systems.
  • Experience with Kubernetes and cloud infrastructure (AWS, GCP).
  • Proficiency in Python or Rust.
  • Familiarity with LLM inference, batching, caching strategies.
  • Ability to design load balancing and routing systems.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: 12 Months

Share this job:

Similar Jobs

Senior Security Manager

New

Define and execute a product security strategy with AI capabilities

Recruit, develop, and retain top security talent

Ai AWS Azure Cloud Security

Applied AI Manager - Partnerships

New

Lead and mentor team of Partner Solutions Architects

Serve as senior tech partner to international GTM teams

Ai technologies AWS Google Cloud Platform Partnership Building

Platform Engineer Job

New

Evaluate cloud infrastructure strategy

Design and implement infrastructure-as-code

Ai AWS CI/CD Datadog

Sr. Director Customer Security

New

Serve as primary security representative in customer engagements

Drive technical security discussions

AWS Azure Cloud Security Compliance

CDP Implementation Architect

New

Promote information security practices

Ensure compliance with data privacy standards

AWS Azure Google Cloud Snowflake

Principal AI Engineer

New

Lead design and development of Java-based backend services

Optimize platform components for high throughput and low latency

AWS Distributed systems Elasticsearch Java

Senior Penetration Tester

New

Lead and conduct advanced penetration testing

Define offensive security roadmap

AWS Azure Cloud infrastructure Kubernetes

CX Operations Lead

New

Drive financial fluency and scalability within Customer Experience operations

Enhance high-touch customer experiences through AI and data utilization

Ai AWS Data Analysis Financial Management

Strategic Partnerships & AI Ecosystem Lead

New

Define and lead ISV ecosystem strategy

Expand AWS co-sell partnerships with key players

Ai Anthropic AWS google

Principal Software Engineer - FGA Core

New

Design and develop scalable authorization solutions

Lead optimization of data models and query paths

Analytical Skills AWS Debugging Problem-solving Skills

Senior Software Engineer

New

Drive design and implementation of new features

Break down complex problems into elegant designs

AWS Azure C# Gcp

AI Product Manager

New

Lead product management for cloud cost visibility product

Define vision, roadmap, and strategy for cloud cost features

Analytics Apis AWS Azure

Security Architect Remote Role

New

Serve as primary security representative in customer engagements

Articulate security architecture and controls

AWS Azure Cloud Security Compliance

AI Model Serving Engineer

New

Efficient implementation of AI models

Prioritizing tasks based on urgency and impact

AWS Azure Backend Development Docker

Senior Software Engineer, Infrastructure

New

Shape technical roadmap

Architect and design deployment pipelines

AWS Docker Gcp Go

Director, Strategic Alliances

New

Lead and execute BD & strategic alliances strategy in the Americas region.

Build and manage a cross-functional partnerships team.

AWS Communication E-commerce Fraud Prevention

Senior Consultant, Back Office

New

Design and deploy workflows for investment lifecycle

Optimize platform including data migration

Apis AWS Confluence Jira

Senior Software Engineer

New

Develop and deliver full-stack code

Ensure testing, documentation, and observability standards

AWS Docker Full-stack Development Github

Senior ML Engineer

New

Lead ML projects and client discussions

Optimize ML models for performance and scalability

AWS Azure CI/CD Distributed systems

Backend Engineer

New

Develop and scale Python scripts for data analysis

Ensure accurate financial and regulatory reporting

Automated Testing AWS CI/CD Data Analysis

Data Engineer, Growth Platforms

New

Build and maintain reliable and scalable data pipelines for Growth use cases

Improve data models and schemas to meet evolving needs

Airflow AWS Data Warehousing Etl Processes

Principal Software Engineer, AI Platform

New

Lead design, development, and scalability of backend services

Architect and optimize platform components

AWS Elasticsearch Java Kubernetes

CDP Architect

New

Promote and enforce awareness of key information security practices

Identify, assess, and report security risks

AWS Azure Google Cloud Snowflake

Senior Penetration Tester

New

Lead advanced penetration testing of web applications and APIs.

Perform security assessments on Cloud Infrastructure.

Apis AWS Azure Cloud infrastructure

Sr. Director Customer Security

New

Serve as the primary security representative in customer and prospect engagements

Drive technical security discussions during pre-procurement stages

AWS Azure Cloud Security Compliance

AI Model Serving Engineer

New

Develop state-of-the-art AI model serving solutions

Prioritize tasks based on urgency and impact

AWS Azure Backend Development Docker

Cloud Security Engineer

New

Review system designs and identify security issues

Develop and implement cloud and infrastructure security architecture

AWS Cloud Security devsecops Github/gitlab

CDP Architect (Contract)

New

Promote and enforce information security practices

Identify and report security risks

AWS Azure Google Cloud Snowflake

AI Model Serving Engineer

New

Work on state-of-the-art voice cloning and low latency text-to-speech technologies

Proven experience in Python backend development and cloud services

AWS Azure Backend Development Docker

Customer Security Director

New

Serve as the primary security representative in customer engagements.

Drive technical security discussions during pre-procurement stages.

AWS Azure Cloud Security Compliance

Senior Penetration Tester

New

Lead and conduct advanced penetration testing

Define offensive security roadmap

AWS Azure Cloud Security Containerization

CDP Architect

New

Promote information security practices

Identify and report security risks

AWS Azure Google Cloud Snowflake

AI Platform Software Engineer

New

Lead design and development of Java-based backend services

Optimize platform components for high-throughput workflows

AWS Distributed systems Elasticsearch Java

AI Model Serving Engineer

New

Developing and deploying AI models for voice cloning

Implementing efficient text to speech services

AWS Azure Backend Development Docker

Sr. Director Customer Security

New

Serve as primary security representative in customer engagements

Articulate security architecture and risk management approach

AWS Azure Cloud Security Compliance

CDP Architect

New

Promote and enforce information security practices.

Identify and report security risks.

AWS Azure Google Cloud PowerBI

AI Model Serving Engineer

New

Develop and maintain AI model serving infrastructure

Implement backend solutions using Python

AWS Azure Backend Development Docker

Principal AI Software Engineer

Posted 3 days ago

Lead design and development of Java backend services.

Optimize platform components for high-throughput workflows.

AWS Devops Elasticsearch Java

AI Model Serving Engineer

Posted 3 days ago

Developing AI models for voice cloning & text to speech

Creating low-latency and cost-effective solutions

AWS Azure Backend Development Cloud Computing

Sr. Director Customer Security

Posted 3 days ago

Representing the organization in customer and prospect interactions with regard to security.

Driving technical security discussions during pre-procurement stages.

AWS Azure Cloud Security Compliance

Staff Software Engineer

Posted 3 days ago

Own long-term technical health for a business unit

Tackle complex, ambiguous problems at Carta

AWS gRPC Postgresql React

Senior Software Engineer, Software Delivery

Posted 3 days ago

Design, implement, and maintain CI/CD platform

Collaborate with users and partner teams

Agile Methodologies AWS Github Actions Google Cloud

Senior ML Engineer

Posted 3 days ago

Design and develop high-performance backend services with Python and FastAPI

Translate ambiguous business and technical requirements into concrete designs

AWS Ci/cd Pipelines Data Security FastAPI

Software Dev Manager, Engage

Posted 3 days ago

Build systems for Twitch ecosystem connectivity

Operate consumer-scale services for VODs, Clips, and Stories

AWS Golang Java Mentoring

Senior Data Engineer

Posted 3 days ago

Architect scalable data pipelines for healthcare data

Lead technical decision-making for data infrastructure

Airflow AWS BigQuery Gcp

Data Scientist Project

Posted 3 days ago

Applying analytics, statistics, and machine learning for decision-making

Developing predictive models for pricing, underwriting, and claims

AWS Azure Etl Gcp

Engineering Manager for Guides & Surveys

Posted 3 days ago

Lead and own the Guides & Surveys product

Collaborate with cross-functional teams

Ai Architecture AWS Code Quality

Cloud Platform Engineer

Posted 3 days ago

Manage and scale self-hosted cloud deployments

Implement third-party DB integrations on big data warehouses

AWS Azure Docker Gcp

AI Engineer II Project

Posted 3 days ago

Design, develop, and deploy AI Agents and ML models

Collaborate with cross-functional teams

Ai agents AWS Golang Java

Detection Engineer

Posted 3 days ago

Design security alerts and response playbooks

Monitor and defend corporate infrastructure

Automated Testing AWS Git Infrastructure as code
overtime