Data Engineer

Job Purpose

We are seeking a highly skilled Senior Data Engineer (Contractor) to design, build, and optimize scalable data platforms and pipelines. This role will play a critical part in developing modern data lakehouse architectures on AWS, enabling advanced analytics and data-driven decision-making across the organization

Job Description

• Design, develop, and maintain analytical data models and metrics using modern, scalable data platform technologies.

• Architect and implement data lakehouse solutions leveraging AWS services.

• Build, optimize, and maintain robust data pipelines using PySpark, Python, and AWS Glue.

• Ensure high performance, reliability, and scalability of data systems.

• Collaborate closely with business stakeholders to translate requirements into data solutions.

• Partner with architecture and operations teams to enforce best practices, standards, and governance.

• Monitor and troubleshoot data workflows, ensuring data quality and system efficiency.

• To adhere to the Information Security Management policies and procedures.

Job Specification / Skills and Competencies

• 5+ years of professional experience in software development, with a focus on analytical platform development.

• 3+ years of hands-on experience with data platforms and big data ecosystems.

• Strong expertise in big data technologies, including:

o AWS EMR

o Apache Spark

o PostgreSQL

o DynamoDB

o Apache Iceberg

• Proficiency in Python and Spark programming.

• Experience building highly scalable, low-latency data systems (e.g., DynamoDB, Aurora, PostgreSQL, Redshift).

• Proven experience designing and implementing data lakehouse architectures on AWS.

• Hands-on experience with AWS services such as Step Functions, Lambda, and EventBridge.

• Experience working in Agile development environments.

Preferred Qualifications

• Strong knowledge of additional AWS services such as:

o Kinesis

o Firehose

o SQS

o Bedrock

o SageMaker

• Experience with real-time data streaming and event-driven architectures.

• Familiarity with machine learning data pipelines is a plus.

Soft Skills & Competencies

• Strong ability to learn and adapt to new technologies quickly.

• Excellent problem-solving and analytical thinking skills.

• Ability to work independently and collaboratively in a team environment.

• Strong multitasking skills in a fast-paced, dynamic setting.

• Excellent verbal and written communication skills.