Data Engineer Databricks + Flink

Job Title: Senior Data Engineer / Big Data Engineer

Experience: 10 to 12 Years

Location: Mumbai, Pune, Chennai, Bangalore, Hyderabad

Job Description-

We are seeking an experienced Senior Data Engineer / Big Data Engineer with strong expertise in Databricks and Apache Flink to design, develop, and optimize large-scale data processing systems and real-time streaming pipelines. The ideal candidate should possess hands-on experience in big data ecosystems, distributed computing, and enterprise-grade data platforms to support scalable analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintain scalable batch and real-time data pipelines using Databricks and Apache Flink.
• Build and optimize data processing workflows for high-volume structured and unstructured datasets.
• Develop streaming and event-driven architectures for real-time analytics and processing.
• Work with large-scale distributed systems and big data frameworks to ensure performance, scalability, and reliability.
• Design data ingestion, transformation, enrichment, and orchestration processes.
• Collaborate with cross-functional teams including data architects, business stakeholders, engineering teams, and analytics teams to deliver data solutions.
• Monitor, troubleshoot, and optimize performance of big data platforms and streaming applications.
• Implement best practices for data governance, data quality, security, and compliance.
• Participate in architecture discussions and contribute to technical design decisions.
• Mentor junior engineers and support technical leadership initiatives where required.

Required Skills & Qualifications

• 10+ years of experience in Data Engineering / Big Data technologies.
• Strong hands-on experience with Databricks and Apache Flink.
• Expertise in big data technologies such as Apache Spark, PySpark, Hadoop, Kafka, Hive, HDFS, or related frameworks.
• Experience building real-time streaming data pipelines and event-driven architectures.
• Strong understanding of distributed systems, ETL/ELT frameworks, and large-scale data processing.
• Proficiency in SQL and programming languages such as Python, Scala, or Java.
• Experience with data lakes, lakehouse architectures, and cloud-based data platforms.
• Strong problem-solving skills and ability to work in a fast-paced environment.

Preferred Skills

• Experience with cloud platforms such as Amazon Web Services, Microsoft Azure, or Google Cloud.
• Exposure to CI/CD pipelines, orchestration tools, and DevOps practices.
• Experience with performance tuning, monitoring, and optimization of streaming systems.
• Understanding of data governance and security frameworks.

Good to Have

• Exposure to data warehousing and analytics platforms.
• Certification in Databricks, cloud platforms, or big data technologies.
• Experience in enterprise-scale data modernization or transformation programs.

Soft Skills

• Strong communication and stakeholder management skills.
• Analytical mindset with strong troubleshooting capabilities.
• Ability to work independently and collaborate across teams