Prompt Engineer/LLM Specialist - Support & Innovation

Job Summary

The Prompt Engineer/LLM Specialist - Support & Innovation is responsible for building production-grade large language model (LLM) systems that go far beyond prompt writing.

This role focuses on designing and evaluating prompts, developing testing pipelines, orchestrating LLM-based workflows, ensuring system reliability, and supporting innovation initiatives.

The specialist will work with evaluation frameworks, RAG architectures, and Python-based automation to maintain high-quality LLM performance in production environments.

Key Responsibilities

• Design and optimize prompts for LLM systems using systematic evaluation methods.
• Build Python pipelines to support prompt testing, scoring, and evaluation.
• Architect, refine, and optimize Retrieval-Augmented Generation (RAG) pipelines.
• Implement orchestration for LLM workflows, including tool usage and agent-based systems.
• Monitor and improve LLM performance, quality, drift behavior, and failure modes.
• Document best practices, evaluation outcomes, and system improvements.
• Support innovation initiatives involving prompt engineering, LLM usage, and applied AI development.

Required Qualifications

• 4+ years of experience in AI, machine learning, natural language processing, or related fields.
• Strong proficiency in Python development.
• Hands-on experience working with LLMs, embeddings, and vector databases.
• Experience deploying LLM systems into production environments.

Preferred Qualifications

• Experience with agentic AI frameworks.
• Strong background in LLM evaluation, observability, and quality monitoring.
• Experience with GenAI solutions in financial services or similar regulated industries.