Responsibilities

• Design and implement scalable model deployment pipelines for LLMs, ensuring high availability and low latency.
• Build and maintain CI/CD workflows for model training, evaluation, and release.
• Monitor and optimize model performance, drift, and resource utilization in production.
• Manage cloud infrastructure (e.g., AWS, GCP, Azure) and container orchestration (e.g., Kubernetes, Docker) for AI workloads.
• Implement observability tools to track system health, token usage, and user feedback loops.
• Ensure security, compliance, and governance of AI systems, including access control and audit logging.
• Collaborate with cross-functional teams to align infrastructure with product goals and user needs.
• Stay current with the latest in MLOps and GenAI tooling and drive continuous improvement in deployment practices. Define and evolve the architecture for GenAI systems, ensuring alignment with business goals and scalability requirements

Qualifications

• Bachelor’s or master’s degree in computer science, Software Engineering, Data Science, or a related technical field.
• 5 to 7 years of experience in software engineering, DevOps, and 3+ years in machine learning infrastructure roles.
• Hands-on experience deploying and maintaining machine learning models in production, ideally including LLMs or other deep learning models.
• Proven experience with cloud platforms (AWS, GCP, Azure) and container orchestration (Docker, Kubernetes).
• Strong programming skills in Python, with experience in ML libraries (e.g., TensorFlow, PyTorch, Hugging Face).
• Proficiency in CI/CD pipelines for ML workflows
• Experience with MLOps tools: MLflow, Kubeflow, DVC, Airflow, Weights & Biases.
  • Knowledge of monitoring and observability tools