AI Data Engineer

About the Role:
We are seeking an experienced AI Data Engineer based in Hyderabad to design, build, and maintain scalable data pipelines and machine learning workflows. The ideal candidate will leverage advanced data engineering and AI/ML frameworks to enable retrieval-augmented generation (RAG) solutions and support large-scale analytics on Azure.

Responsibilities:
Design, develop, and optimize end-to-end data pipelines using Python, SQL, Databricks, and Snowflake
Implement and maintain RAG solutions and embeddings pipelines for large language models
Develop, train, and deploy machine learning models using PyTorch, TensorFlow, and scikit-learn
Collaborate with data scientists, architects, and stakeholders to translate business requirements into technical solutions
Ensure data quality, governance, and compliance in all pipelines and workflows
Monitor performance, troubleshoot issues, and optimize resource utilization in Azure environments
Document architecture, code, and best practices; conduct knowledge-sharing sessions
Mentor and guide junior engineers on best practices in data engineering and MLOps

Required Qualifications:
Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
8–12 years of hands-on experience in data engineering and ML engineering
Expertise in Python programming and SQL for data processing
Strong experience with Databricks and Snowflake for large-scale data analytics
Proficiency in Azure data services including Data Factory, Databricks, and Storage
Experience building retrieval-augmented generation (RAG) solutions
Production experience with PyTorch, TensorFlow, and scikit-learn
Solid understanding of data modeling, ETL/ELT processes, and distributed systems

Preferred Qualifications:
Azure Data Engineer or Snowflake certifications
Experience with Docker, Kubernetes, and CI/CD pipelines
Familiarity with real-time data streaming (Kafka, Event Hubs)
Knowledge of MLOps frameworks (MLflow, Azure ML)
Advanced degree (MS or PhD) in a quantitative discipline
Strong communication skills and experience leading cross-functional teams
Experience with large language model fine-tuning and deployment