Data Engineer

We are seeking an experienced and strategic Data to design, build, and optimize scalable, secure, and high-performance data solutions. You will play a pivotal role in shaping our data infrastructure, working with technologies such as Databricks, Azure Data Factory, Unity Catalog, and Spark, while aligning with best practices in data governance, pipeline automation, and performance optimization. Key Responsibilities:

• Design and develop scalable data pipelines using Databricks and Medallion (Bronze, Silver, Gold layers).
• Architect and implement data governance frameworks using Unity Catalog and related tools.
• Write efficient PySpark and SQL code for data transformation, cleansing, and enrichment.
• Build and manage data workflows in Azure Data Factory (ADF) including triggers, linked services, and integration runtimes.
• Optimize queries and data structures for performance and cost-efficiency.
• Develop and maintain CI/CD pipelines using GitHub for automated deployment and version control.
• Collaborate with cross-functional teams to define data strategies and drive data quality initiatives.
• Implement best practices for DevOps, CI/CD, and infrastructure-as-code in data engineering.
• Troubleshoot and resolve performance bottlenecks across Spark, ADF, and Databricks pipelines.
• Maintain comprehensive documentation of architecture, processes, and workflows.

Requirements:

• Bachelors or masters degree in computer science, Information Systems, or related field.
• Proven experience as a Data Architect or Senior Data Engineer.
• Strong knowledge of Databricks, Azure Data Factory, Spark (PySpark), and SQL.
• Hands-on experience with data governance, security frameworks, and catalog management.
• Proficiency in cloud platforms (preferably Azure).
• Experience with CI/CD tools and version control systems like GitHub.
• Strong communication and collaboration skills.

Information :

• Company : LTIMindtree
• Position : Data Engineer
• Location : Bangalore, Karnātaka
• Country : IN