AI Data Engineer

About the Role:
We are seeking an experienced AI Data Engineer to join our Hyderabad team. In this role, you will design, build, and maintain scalable data pipelines and architectures to support our AI and analytics initiatives. You will work closely with data scientists, analysts, and stakeholders to ensure high-quality, robust data solutions on Azure and Snowflake platforms.

Responsibilities:
Design, develop, and maintain end-to-end data pipelines and ETL processes using Python and Spark
Implement and optimize data ingestion, transformation, and storage solutions on Snowflake
Leverage Azure services (Data Factory, Databricks, Synapse) to build scalable data architectures
Collaborate with data science teams to prepare and manage datasets for AI/ML model training
Monitor data workflows, troubleshoot performance issues, and ensure data reliability
Establish and enforce data governance, security, and compliance best practices
Document data models, pipelines, and processes for knowledge sharing and maintenance
Mentor and guide junior data engineers and contribute to technical reviews
Participate in agile ceremonies and contribute to sprint planning and delivery

Required Qualifications:
9+ years of experience in data engineering or related roles
Proficiency in Python for data processing and pipeline development
Strong expertise with Apache Spark for large-scale data processing
Hands-on experience with Snowflake data warehousing
Practical experience with Azure data services (Data Factory, Databricks, Synapse)
Solid SQL skills and understanding of data modeling and warehousing concepts
Experience with version control systems such as Git
Excellent problem-solving skills and attention to detail
Strong communication and collaboration skills to work with cross-functional teams

Preferred Qualifications:
Microsoft Certified: Azure Data Engineer or SnowPro Advanced certification
Experience with containerization and orchestration tools (Docker, Kubernetes)
Familiarity with CI/CD pipelines and DevOps practices for data solutions
Knowledge of machine learning frameworks (TensorFlow, PyTorch, scikit-learn)
Exposure to data governance and metadata management tools
Proven track record of mentoring or leading technical teams