AI Data Engineer

About the Role:
We are seeking a Senior AI Data Engineer to join our Hyderabad team. In this role, you will design, build, and maintain scalable data architectures and pipelines to support AI and machine learning initiatives, ensuring seamless data integration, processing, and governance.

Responsibilities:
Design, develop, and optimize end-to-end data pipelines for AI/ML workloads using Python and Apache Spark
Build and manage data warehousing solutions on Snowflake, ensuring high performance and scalability
Implement and maintain data ingestion frameworks in Azure, including Azure Data Factory and Databricks
Collaborate with data scientists and ML engineers to provision and curate datasets for model training and deployment
Ensure data quality, reliability, and security by applying best practices in data governance and ETL testing
Monitor and troubleshoot data workflows, optimizing for performance and cost efficiency
Document architecture, data flows, and operational procedures, and mentor junior team members

Required Qualifications:
9+ years of experience in data engineering or related roles
Expert proficiency in Python for data processing and automation
Hands-on experience with Apache Spark for large-scale data analytics
Strong experience with Snowflake data warehousing platform
Proven experience with Azure services (Data Factory, Databricks, SQL Data Warehouse)
Deep understanding of SQL, data modeling, and ETL best practices
Excellent analytical, problem-solving, and communication skills

Preferred Qualifications:
Experience with workflow orchestration tools such as Apache Airflow
Familiarity with CI/CD and DevOps practices for data pipelines
Knowledge of containerization technologies (Docker, Kubernetes)
Experience building real-time data streaming pipelines
Exposure to additional cloud platforms (AWS, GCP)
Background in machine learning data pipeline design and feature engineering