Data Engineer II - Python/Spark/AWS
JP Morgan
You thrive on diversity and creativity, and we welcome individuals who share our vision of making a lasting impact. Your unique combination of design thinking and experience will help us achieve new heights.
Job responsibilities
Design and build scalable, high-performance, and reliable data pipelines.Gather, analyze, model, and transform datasets to extract valuable insights from a large and diverse pool of both structured and unstructured data.Organizes, updates, and maintains gathered data that will aid in making the data actionableProvide technical expertise in designing and implementing solutions related to data delivery.Ensure adherence to data governance principles, implement data quality checks, and maintain data lineage throughout the data lifecycle.Collaborate with cross-functional teams to gather business requirements and translate them into effective database designs and data flows.Prepare accurate documentation on database design, data flow architecture, and pipeline orchestration.Demonstrates basic knowledge of the data system components to determine controls needed to ensure secure data accessBe responsible for making custom configuration changes in one to two tools to generate a product at the business or customer request
Required qualifications, capabilities, and skills
Formal training or certification on software engineering concepts and 2+ years applied experienceBasic knowledge of the data lifecycle and data management functionsProficiency in SQL, ETL, data modeling, and Python.Hands-on experience with building data pipelines using Python and PySpark.Strong database skills with a thorough understanding of databases and data modelling concepts.Advanced at SQL (e.g., joins and aggregations)Working understanding of NoSQL databasesSignificant experience with statistical data analysis and ability to determine appropriate tools to perform analysisBasic knowledge of data system components to determine controls neededPreferred qualifications, capabilities, and skills Knowledge of Apache Iceberg.Knowledge of AWS and relevant services like S3, Glue.Knowledge of pipeline orchestrators like Airflow, Argo.Knowledge of version control systems like GitHub.Knowledge of metadata management, data lineage, and data glossaries.
Por favor confirme su dirección de correo electrónico: Send Email
Todos los trabajos de JP Morgan