Build your future with Sovos.
At Sovos, we're transforming tax compliance through innovative technology and AI-powered solutions used by customers worldwide. We're looking for an LLM Ops Lead with 2-3 years of experience to help scale, govern, and operationalize Generative AI across the business while ensuring quality, safety, observability, and continuous improvement.
What You'll Do
- Own AI quality standards, evaluation frameworks, and release gates across Ask Sovi products
- Lead prompt and model versioning, regression detection, and AI governance initiatives
- Coordinate Safety, Guardrails, and Red Team testing programs
- Build observability practices across latency, cost, quality, and safety metrics using tools such as Langfuse
- Manage Jira epics, ticket triage, and cross-functional coordination across Product and Engineering teams
- Develop LLMOps playbooks, best practices, and operational standards
- Lead AI incident response, troubleshooting, and root-cause analysis
What We're Looking For
- 2-3 years of experience in Applied AI, LLMOps, AI Quality Engineering, AI Operations, Test Automation, or related fields
- Hands-on experience with Langfuse, LangSmith, or similar AI observability and evaluation tools
- Strong understanding of AI quality, evaluation pipelines, prompt versioning, and AI governance
- Experience with AI safety, guardrails, content moderation, audit logging, and release management
- Strong stakeholder management and communication skills
- Ability to support teams operating across UK business hours
Ideal Candidate Backgrounds
- Applied AI Engineers from AI-native or AI-first organizations with hands-on Langfuse/LangSmith experience
- Product-minded Engineers who combine strong technical skills with customer and business impact awareness
- QA/Test Automation Engineers who have transitioned into AI evaluation, quality, or LLM testing
- AI Quality Analysts with expertise in hallucination detection, model drift monitoring, LLM-as-a-Judge methodologies, and AI quality governance
Why Join Sovos?
- Work on cutting-edge Generative AI products and platforms
- Collaborate with global AI, Product, and Engineering leaders
- Bi-Weekly Meeting Free Days
- Mentoring and career development programs
- Globally recognized training opportunities
- Comprehensive health and wellness benefits
- Flexible and collaborative work environment
Recruitment Process
CCAT & EPP Assessment → Automated Video Interview → Hiring Manager Interview → Final Interview
#Hiring #LLMOps #AIOps #GenerativeAI #AIEngineering #LangSmith #Langfuse #SovosCareers