Data Engineer Lead - Java with Spark

Responsibilities:

• Lead and manage the development team, including assigning tasks and monitoring progress
• Part of Development team to analyse, design, and implement enterprise-wide Java based systems
• Hands on expertise in Microservices.
• Plan, analyse and define solutions through business, functional and technical expertise
• Work with product managers on issues or questions related to system requirements
• Respond to inquiries concerning application development and support
• Develop, program, and maintain applications using the Apache Spark open-source framework
• Work with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming

Requirements:

• At least a bachelor’s degree in computer engineering, or a relevant field
• Minimum 10 years of hands-on experience in Spark, Microservices and latest Java version
• Minimum 6 years of experience in cloud computing and architecture
• Experience with major cloud platforms e.g. AWS, Azure, GCP etc.
• Experience with containerization technologies e.g. Ec2, Docket, Kubernetes
• Experience using Spark framework and streaming data platforms
• Experience in hands-on programming skills in Java and Hadoop ecosystem
• Knowledge in relational databases i.e. SQL and ORM technologies e.g. JPA2, Hibernate
• Knowledge in database design and data normalisation
• Knowledge of DevOps methodologies and tools
• Knowledge of Oracle / PostgreSQL
• Knowledge of relational databases, SQL and ORM technologies i.e. JPA2, Hibernate and database design and data normalization

Preferred Requirements:

• Knowledge and hands-on experience in Hadoop concepts like Hadoop Common, HDFS, Hadoop Yarn, Hadoop MapReduce and data processing techniques
• Previous work experiences as a Java, Spring boot Technical Lead or similar roles
• Strong experience with Java and Spring boot frameworks like MVC, REST backend applications