Senior Data Engineer

Data Platform Engineer

Data Engineering Management Expert (TOC)

Responsibilities: <ul><li>Data engineers need to write code to develop, maintain, and optimize data pipelines, as well as to implement data storage solutions and data processing systems.&nbsp;</li><li>They typically use programming languages such as Python, Java, or Scala, along with big data technologies like Hadoop, Spark, and Kafka</li><li>Designing scalable data architecture that supports both transactional and analytical workloads. This involves understanding business requirements and translating them into technical solutions.</li><li>Developing robust data pipelines to ingest, cleanse, transform, and aggregate data from various sources (internal and external). This includes both batch and real-time/streaming data processing.</li><li>Integrating data from disparate sources and ensuring data consistency, integrity, and reliability across different systems.</li></ul> Requirements: <ul><li>5-8 years experience with GCP, Big Query, Data Fusion, Analytics Hub and DataFlow</li><li>Experience leading others</li><li>Experience managing projects</li><li>Excellent written and verbal communication skills</li><li>Time Management</li><li>Ability to work alone and with a team</li><li>Ability to mentor junior members</li><li>A Bachelor’s degree in MIS or mathematics, statistics, or computer science, international equivalent, or equivalent job experience</li></ul>

Responsibilities: <ul><li>Support daily business-as-usual (BAU) operations, audit and compliance</li><li>Communicate with vendor and BAU teams for data ingestion queries and issues</li><li>Manage data ingestion and DAG maintenance in Airflow</li><li>Develop and modify Python scripts for data ingestion</li><li>Write optimised SQL queries in Snowflake</li><li>Ensure documentation of applications</li></ul> Requirements: <ul><li>Bachelors degree in information technology/ computer science / software engineering</li><li>Knowledge and hands-on experience writing effective SQL queries and statements</li><li>Understanding of AWS services</li><li>At least 5 years of experience in a similar capacity</li><li>At least 3 years of proficiency in using Python to develop and modify scripts</li><li>At least 3 years of proficiency in managing data ingestion and DAG maintenance in Airflow</li></ul> Preferred Requirements: <ul><li>Knowledge in Hadoop Ecosystem like Spark or PySpark</li><li>Knowledge in AWS services like S3, Data Lake, Redshift, EMR, EC2, Lambda, Glue, Aurora, RDS, Airflow</li><li>1-2 years of proficiency in using Snowflake, including query optimisation to control Snowflake’s warehouse cost</li><li>Previous work experiences from investment banking/finance industry with demonstrated exposure to be ready to take on an individual contributor role</li></ul>

Responsibilities: <ul><li>Design, construct, and maintain highly scalable databases tailored to the needs of the Archie Initiative Data Explorer.</li><li>Implement robust data pipelines to facilitate efficient data ingestion, transformation, and dissemination.</li><li>Collaborate closely with a cross-disciplinary team including data scientists, analysts, and stakeholders to understand data needs and deliver on project goals.</li><li>Develop and optimize database structures, data-processing algorithms, and data output solutions to ensure data integrity and quality.&nbsp;</li><li>Support the team in conducting research on energy traceability and life cycle analysis, providing data-driven insights and recommendations.</li><li>Document processes, systems, and data flows to ensure reproducibility and facilitate knowledge sharing among the team.&nbsp;</li><li>Stay updated on the latest industry trends, technologies, and best practices in data engineering and relevant fields to continually improve our systems and processes.</li></ul> Requirements: <ul><li>Bachelors degree in Computer Science, Engineering, or a related field; Master’s degree preferred but not required.&nbsp;</li><li>5+ years of professional experience in data engineering, database administration, or a related role.</li><li>Proficiency in SQL, and experience with at least one scripting language such as Python or Ruby.</li><li>Experience with big data technologies like Hadoop, Spark, or similar frameworks is a plus.&nbsp;</li><li>Knowledge of energy traceability and life cycle analysis concepts is desirable.</li><li>Strong problem-solving skills, attention to detail, and ability to work both independently and in collaborative team environments.&nbsp;</li><li>Excellent communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.</li><li>Must possess good work habits, a strong work ethic, and be able to adhere to company work hours, policies, and standard business etiquette.</li></ul> Working Environment: <ul><li>Our high-performing employees are drawn by the challenging and rewarding professional, technical and industrial opportunities we offer, and are remunerated accordingly.</li><li>Our people work on truly world-scale projects, supported by investment in capital and technology that is second to none.&nbsp;</li><li>And because, as a global energy company, we are faced with addressing some of the world’s biggest technical, logistical and environmental challenges, we invest heavily in talent development.</li><li>We have a proud history of educating and training our workforce over many decades.&nbsp;</li><li>Employees at all levels are encouraged to improve their sector-specific knowledge and competencies through our workforce development programs – one of the largest in the world.</li></ul>

Skills

Industry

Location

Jobtitle

3 hadoop Jobs found

Senior Data Engineer

Data Platform Engineer

Data Engineering Management Expert (TOC)

WhiteCrow
WhiteCrow is on a mission to connect companies with the best talent across the globe.

EMPLOYER

TALENT

COMPANY

Contact Us

JOBS BY SKILLS

JOBS BY FUNCTION

JOBS BY INDUSTRY

JOBS BY LOCATIONS

MODEL JOBS

JOBS BY CITY

Skills

Industry

Location

Jobtitle

3 hadoop Jobs found

Senior Data Engineer

Data Platform Engineer

Data Engineering Management Expert (TOC)

WhiteCrow WhiteCrow is on a mission to connect companies with the best talent across the globe.

EMPLOYER

TALENT

COMPANY

Contact Us

JOBS BY SKILLS

JOBS BY FUNCTION

JOBS BY INDUSTRY

JOBS BY LOCATIONS

MODEL JOBS

JOBS BY CITY

WhiteCrow
WhiteCrow is on a mission to connect companies with the best talent across the globe.