AWS Data Engineer

LSEG is hiring for the role of AWS Data Engineer with Python!

Responsibilities of the Candidate:

• Design, build, and maintain robust, scalable and efficient ETL pipelines using Python and Spark.
• Develop workflows leveraging AWS services such as EMR Serverless, Glue, Glue Data Catalog, Lambda and S3.
• Implement data quality frameworks and governance practices to ensure reliable data processing.
• Collaborate with cross-functional teams to gather requirements, provide technical insights and deliver high-quality solutions.
• Optimize existing workflows and drive migration to a modern data lakehouse architecture, integrating Apache Iceberg.
• Enforce coding standards, design patterns, and system architecture best practices.
• Monitor system performance and ensure data reliability through proactive optimizations.
• Contribute to technical discussions, mentor junior team members and foster a culture of learning and innovation.

Requirements:

• Expertise in Python and Spark, with proven experience in designing and implementing data workflows.
• Hands-on experience with AWS EMR Serverless, Glue, Glue Data Catalog, Lambda, S3, and EMR.
• Strong understanding of data quality frameworks, governance practices and scalable architectures.
• Practical knowledge of Apache Iceberg within data lakehouse solutions.
• Problem-solving skills and experience optimizing workflows for performance and cost-efficiency.
• Agile methodology experience, including sprint planning and retrospectives. Excellent communication skills for articulating technical solutions to diverse stakeholders.
• Familiarity with additional programming languages such as Java.
• Experience with serverless computing paradigms.
• Knowledge of visualization or reporting tools.