DATA ENGINEER Exp Cons

Responsibilities:

• Requirements gathering, system analysis, design, development/configuration of new data sets/marts. 
• Build ETL pipelines in Spark / Python / Scala and standardize data fields across various data sources 
• Create data dictionaries, setup/monitor data validation alerts and execute periodic jobs  
• Define and build technical/data documentation and experience with code version control systems (e.g. git). Ensure data accuracy, integrity and consistency 

Requirements:

• Excellent interpersonal skills and the ability to successfully interact with stakeholders from senior management to junior staff 
• Proactively solves problems and seeks to own the resolution of issues throughout the lifecycle 
• Good to have: Tableau/PowerBI Other Visualization tools 
• Strong experience in creating large scale data engineering pipelines, data marts and warehouses  
• Experience with SQL for extracting, aggregating and processing big data Pipelines using Python  
• Good working knowledge of using and developing Data APIs 
• Experience with complex, high volume, multi-dimensional data based on unstructured, structured, and streaming datasets 

        Proficiency in: 

• Programming languages: Python, SQL 
• Apache Spark, Apache Airflow 
• Docker, Bash 
• Cloud: Either of below:
  •     Azure Data Factory 
  •    Google Cloud Dataflow / Data Fusion 
  •    AWS Data Pipeline / Glue 
  •    Stitch / Snowflake  

 Data Streaming: Kafka/Kinesis  

Education and Experience:

• Bachelor’s degree from Tier I/II colleges (Post Graduation – Good to have, not mandatory) 
• (5-7)years of experience in Data Engineering