LLM Infrastructure Engineer

Gainwell is hiring for the role of LLM Infrastructure Engineer!

Responsibilities of the Candidate:

• Develop and manage scalable deployment strategies specifically tailored for LLMs (GPT, Llama, Claude, etc.).
• Optimize LLM inference performance, including model parallelization, quantization, pruning, and fine-tuning pipelines.
• Integrate prompt management, version control, and retrieval-augmented generation (RAG) pipelines.
• Manage vector databases, embedding stores, and document stores used in conjunction with LLMs.
• Monitor hallucination rates, token usage, and overall cost optimization for LLM APIs or on-prem deployments.
• Continuously monitor models for its performance and ensure alert system in place.
• Ensure compliance with ethical AI practices, privacy regulations, and responsible AI guidelines in LLM workflows.

Requirements:

• Bachelor's/Master’s degree in computer science, Engineering, or related fields.
• Strong experience with ML Ops tools (Kubeflow, MLflow, TFX, SageMaker, etc.).
• Experience with LLM-specific tools and frameworks (LangChain,Lang Graph,  LlamaIndex, Hugging Face, OpenAI APIs, Vector DBs like Pinecone, FAISS, Weavite, Chroma DB etc.).
• Solid experience in deploying models in cloud (AWS, Azure, GCP) and on-prem environments.
• Proficient in containerization (Docker, Kubernetes) and CI/CD practices.
• Familiarity with monitoring tools like Prometheus, Grafana, and ML observability platforms.
• Strong coding skills in Python, Bash, and familiarity with infrastructure-as-code tools (Terraform, Helm, etc.).Knowledge of healthcare AI applications and regulatory compliance (HIPAA, CMS) is a plus. 
• Strong skills in Giskard, Deepeval etc.