Data Engineer (Java and Spark, Parquet, Avro, Hive, Iceberg)

Job Description

Detailed Job Description

\n

\n

\n 6+ years overall software development experience with at least 3+ years of experience in large scale data platforms

\n

.Excellent expertise in Spark and Java

\n

.Knowledge of Golang is highly desirable

\n

.Good understanding of containerization using Docker and Kubernetes

\n

.Understanding of version control like Git and CI/CD workflow

\n

.Understanding of Parquet/Avro/Hive/Iceberg

\n

.Well experienced in debugging and troubleshooting

\n

.Good Communication and excellent collaboration skills to interact with internal customer

\n

\n

\n

s

\n Deliverabl

\n

\n

es:
\n Help support Data Replication Service (Spark jobs) and develop features/bug fi

\n

xes.Help support SparkCp (a library) in debugging and fixing issues encountered by

\n

DRS.Help support the CDH Migration Tool in debugging and fixing issues encountered by custom

\n

ers.Implement monitoring/alerting improvements to the 3 tools mentioned previously for better supportabil

\n

ity.Perform onboarding tasks to help move customer d

\n

ata.Monitor and address user requests and issues as they ar

\n

ise.