Hadoop / Bigdata Developer
Job Description
Must Have Technical/Functional Skills
\n
\n
Primary Skill: Hadoop ecosystem (HDFS, Hive, Spark), PySpark, Python, Apache Kafka
\n
Experience: Minimum 10 years
\n
\n
Roles & Responsibilities
\n
\n
Architectural Leadership:
\n
· Define end-to-end architecture for data platforms, streaming systems, and web applications.
\n
· Ensure alignment with enterprise standards, security, and compliance requirements.
\n
· Evaluate emerging technologies and recommend adoption strategies.
\n
\n
Data Engineering:
\n
· Design and implement data ingestion, transformation, and processing pipelines using Hadoop, PySpark, and related tools.
\n
· Optimize ETL workflows for large-scale datasets and real-time streaming.
\n
· Integrate Apache Kafka for event-driven architectures and messaging.
\n
\n
Collaboration & Leadership:
\n
· Work closely with product owners, business analysts, and DevOps teams.
\n
· Mentor junior developers and data engineers.
\n
· Participate in agile ceremonies, code reviews, and design discussions.
\n
\n
Required Skills & Qualifications:
\n
\n
Technical Expertise:
\n
· Strong experience with Hadoop ecosystem (HDFS, Hive, Spark).
\n
· Proficiency in PySpark for distributed data processing.
\n
· Advanced programming skills in Python.
\n
· Hands-on experience with Apache Kafka for real-time streaming.
\n
\n
Architectural Skills:
\n
· Expertise in designing scalable, secure, and high-performance systems.
\n
· Familiarity with microservices, API design, and cloud-native architectures.
\n
\n
Additional Skills:
\n
· Knowledge of CI/CD pipelines, containerization (Docker/Kubernetes).
\n
· Exposure to cloud platforms (AWS, Azure, GCP). Soft Skills: · Strong analytical and problem-solving abilities.
\n
· Excellent communication and leadership skills.
\n
· Ability to thrive in a fast-paced, agile environment.
\n