Senior AI Trainer
Senior AI Trainer
We are looking for a detail-oriented and technically curious AI Trainer to support the development, training, and evaluation of LLM-based systems.
Project info: The project is an AI-powered virtual assistant for a telecom customer support. It uses a hybrid architecture combining Dialogflow CX, Generative AI, and advanced NLU with RAG tools to deliver accurate, contextual responses across voice and chat channels.
This is a hybrid role combining content expertise, linguistic precision, and a basic understanding of model behavior, particularly in the context of NLP and generative AI systems.
Employment type: Full-time
Location: USA, Canada, or Mexico
Language: English C1 (Advanced)
Key Responsibilities
Design and optimize prompt templates for various use cases (e.g., summarization, classification, entity extraction, reasoning, question answering, dialogue).
Evaluate model responses (using similarity comparison, criteria ranking, or pass/fail) and provide actionable feedback to fine-tune performance.
Create high-quality input-output examples, instruction sets, and single-turn/multi-turn test cases for model behavior validation.
Support adversarial testing and edge-case coverage to ensure the robustness of AI models.
Analyze and troubleshoot model errors to identify failure patterns, inconsistencies, and hallucination risks.
Perform post-deployment human evaluations to identify edge cases, model misalignments, and opportunities for improvement.
Monitor live systems for intent recognition accuracy, fallback behavior, and GenAI output consistency; troubleshoot anomalies and implement fixes.
Implement, assess and enhance the consistency and relevance of outputs from Retrieval-Augmented Generation (RAG) tools and Knowledge Bases/Datastores.
Annotate, curate, and clean datasets for supervised and reinforcement learning purposes.
Ensure all data handling complies with privacy, security, and ethical AI guidelines.
Participate in the design and feasibility review of conversation flows, ensuring the AI system is well-equipped to handle real-world scenarios.
Collaborate with engineering, QA, and product teams to ensure timely delivery of NLU and GenAI components as per the roadmap.
Requirements:
Strong written English skills (C1+), with the ability to create clear, structured, and instruction-based content.
Experience with prompt engineering and optimization for tasks such as summarization, classification, entity extraction, reasoning, Q&A, and dialogue.
Understanding of LLM behavior and hands-on experience with models like OpenAI GPT, Anthropic Claude, and Google Gemini.
Knowledge of RAG frameworks, vectorbases, optimized chunking techniques.
Hands-on skills in data annotation and evaluation workflows, including ranking, scoring, and linguistic analysis.
Familiarity with GenAI safety, including prompt injection mitigation and red teaming practices.
Familiarity with conversational AI platforms such as Dialogflow, RASA, LUIS.ai, IBM Watson, Infobip, Amazon Lex.
Proficiency in Python (pandas, numpy, spacy, nltk, scikit-learn, transformers, tensorflow, pytorch, matplotlib) for working with datasets and prototyping evaluations.
Ability to work with structured data using SQL (PostgreSQL, MySQL, MongoDB) and tools like Databricks, Athena, Power BI, Looker etc.
Basic familiarity with DevOps/testing tools (Docker, CI/CD, Postman).
Use of collaborative tools such as Figma, Miro, Lucidchart for coordinating with design and product teams.
Process Flow:
HR pre-screen + English check (0.5 h)
Professional interview (1 h)
Client's Technical interview (1 h)