Client :charter communication
Duration: Long Term
Consultant will be responsible for Analysis of Legacy System On-premise Hadoop data lake objects and structures.
Discussion and communication with client for understanding the technical design of data transformation in Talend.
Responsible for development, modification of orchestration using Azure Data Factory and Hive queries.
Review and optimize ADF data pipelines with focus on usability, performance, flexibility and Standardization.
Optimize and validate historical data loads using ADF orchestration and spark jobs.