Software Data Engineer

Join our team as a Software Data Engineer and work on exciting document mining challenges in collaboration with ML and full-stack teams. Scale data pipelines efficiently and apply best practices for cloud-based data platforms. Architect data pipelines using LLMs for high-fidelity entity extraction and implement evaluation frameworks for monitoring accuracy and drift.

Key Responsibilities

• Collaborate with ML and full-stack teams on document mining challenges
• Scale data pipelines to move data quickly from research to platform
• Define best practices for a cloud-based data platform
• Architect data pipelines using LLMs for high-fidelity entity extraction
• Implement evaluation frameworks to monitor accuracy, drift, and hallucination

Requirements & Qualifications

• Degree in Computer Science/Engineering or related field
• 3+ years experience as a software developer
• Proficient with Python
• Proficient with SQL
• Experience using LLMs for structured data extraction
• Experience with event-driven architecture with Pub/Sub