Senior Data Engineer
persistent
|
Experience: 7+ Years
|
Pune
|
Posted 14 hours ago
Job Description
- Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
- Investigate and resolve data quality issues, pipeline failures, and production incidents.
- Perform root cause analysis for data processing and framework-related failures.
- Monitor and optimize Spark workloads for performance, scalability, and reliability.
- Collaborate with cross-functional teams to understand business requirements and implement robust data solutions.
- Support production deployments and provide L2/L3 support for critical data platforms.
- Establish and enforce best practices for data engineering, code quality, monitoring, and operational excellence.
- Drive continuous improvements in data processing frameworks and platform stability.