We are looking for a passionate and experienced Senior Data Engineer to join our team in Hyderabad. In this role, you will design and build scalable, high-performance data platforms and pipelines that power data-driven decision-making across the organization. You will collaborate with cross-functional teams, mentor fellow engineers, and drive engineering best practices while leveraging modern cloud technologies, distributed data processing frameworks, and real-time streaming solutions. If you enjoy solving complex data challenges and shaping the future of enterprise data infrastructure, we'd love to hear from you.

Responsibilities:

• Drives best practices and sets standards for the team in areas like data modelling, pipeline architecture, and system design.
• Is a key influencer in the team's strategy and contributes significantly to the team planning.
• Shows good judgment making trade-offs between immediate and long-term business needs.
• Is a result-driven creative thinker who drives innovation and produces delightful experiences for our data consumers and internal customers.
• Is an advocate for data-driven decision-making, has an insatiable curiosity, and loves to invent and innovate to solve difficult problems.
• Has a strong point of view but remains open-minded.
• Takes ownership of their work and consistently delivers results in a fast-paced environment.
• You are a collaborative leader that makes other engineers and team members around you more productive, by sharing your knowledge and helping to tie-break key technical decisions.
• You play a leading role in designing and developing major data infrastructure components, involving yourself and other engineers.
• You are a key influencer in your team's strategy and contribute significantly to team planning.
• You show good judgement making trade-offs between immediate and long-term business needs.
• You troubleshoot production issues by reviewing logs, metrics, data pipelines, and system health to pinpoint specific problems and then resolve them. You identify root causes and identify learnings to improve both development processes and system design.
• Provide guidance on design, coding, and operational best practices for data engineering.
• You propose and create best practices proactively where none exist.
• You make high-impact decisions driving how and what data systems get built.
• You mentor junior engineers, overseeing their designs, code quality, and integration into a team.
• Your success is judged as much on your own productivity as on the positive impact you have on the engineers around you. 

Requirements:

• Proficient in Python and/or Scala/Java with 5-9 years of total experience in data engineering or related fields.
• Strong experience with distributed data processing frameworks such as Apache Spark, Flink, or similar technologies.
• Deep understanding of data pipeline orchestration tools like Apache Airflow, Dagster, or Prefect.
• Experience with cloud data platforms (AWS, GCP, or Azure) including services like S3, Redshift, BigQuery, Databricks, or Snowflake.
• Hands-on experience with real-time streaming technologies such as Apache Kafka, Kinesis.
• Experience with data lake/lakehouse architectures and formats like Parquet, Avro, Delta Lake, or Iceberg.
• Strong understanding of SQL and database technologies including PostgreSQL, NoSQL databases like DynamoDB or Cassandra.
• Experience with containerization (Docker) and orchestration (Kubernetes) in data engineering contexts.
• Familiarity with infrastructure-as-code tools like Terraform or CloudFormation.
• Ability to implement comprehensive monitoring, alerting, and observability using tools like Prometheus, Grafana, CloudWatch, or Datadog.
• Strong understanding of data modeling, ETL/ELT patterns, and data warehouse design principles.
• Experience with CI/CD pipelines and version control systems (Git). 

Quality Compliance:

Compliance to Quality and Information Security is critical in ensuring the integrity, confidentiality, availability of data and the consistent delivery of high-quality services are an important aspect of hiring for this position.