Collaborates with: architects and other data engineers, data scientists, development and operations engineers, project managers, product / market managers, Product Owners.

MISSION  

The Data Engineer aims to define, develop, implement and maintain the appropriate tools and infrastructures for data analysis by Data Analysts and Data Scientists. He makes sure to create a solution allowing the processing of large volumes of data while guaranteeing their security. It represents the first link in the data processing chain.

The mission is varied:

• Provide expertise in extraction, transformation, loading of data, allowing the manipulation of data;

• Design platforms to process large volumes of data;

• Ensure that data pipelines are secure and clear to be analyzed and transformed;

• Contribution to the work of the data team more broadly.

ACTIVITES & PERFORMANCES

Main activities

As an indication, the activities, without being limiting, may include:

• Prepare the data for use by the Data Scientist or Data Analsyt;

• Design and organize pipeline automation;

• Provide support for the pipelines provided to the businesses;

• Train and support software engineers in their development of platform flows or integration;

• Monitor and participate in conferences to develop your level of knowledge on data management tools: data collection, storage, data distribution, processing and recovery platforms, in particular data cloud services, etc.

Performance indicators

• Relevance of architectures and data management solutions proposed

• Pipeline performance and maintainability;

• Clarity of documentation of pipelines, transformations, database models;

ProfIlE et compétences

PROFILE AND SKILLS

The Data Engineer generally comes from a Bac+5 training such as a master's degree in computer science, engineering school or alternative courses with a master's degree specializing in data science.

Required Skills :

1 - Technical skills:

• He is a specialist in structured languages ​​such as Javascript, Scala and Python.

• He/she also has skills in designing databases such as SQLServer, MariaDB, MongoDB, Couchbase which he/she creates using SQL and NoSQL;

• Strong expertise in data storage and ETL tools;

• He/she must master different technologies used in Big Data business such as Hadoop, Spark, Nifi;

• Proficiency in technical and fluent English.

2 - Personal skills:

• Analytical and synthetic mind;

• Strength of proposal;

•            Sense of organization ;

• Rigor and sense of quality;

• Excellent interpersonal skills and team spirit, especially in a multidisciplinary team.