Job Description
Job DescriptionJob Title: Cloud Engineer
Location: Boston, MA or Atlanta, GA - 2-3 days a week onsite
Length: 6 months
We are seeking an experienced Software Engineer/Cloud Developer to design, develop, and implement an end-to-end event-driven data processing pipeline on AWS. The ideal candidate will be responsible for building a scalable, resilient, and production-ready solution that can process incoming configuration-based messages, orchestrate data workflows, and integrate with downstream services deployed on AWS infrastructure.
Key Responsibilities:
• Design and develop an event-driven pipeline capable of receiving and parsing incoming messages containing configuration details.
• Implement message queuing mechanisms to reliably capture and process requests using AWS SQS or similar services.
• Develop logic to identify and retrieve the appropriate data files from Amazon S3 based on the information contained in the queued messages.
• Build robust data parsing and transformation components to process files according to business and configuration requirements.
• Integrate with containerized services running on Amazon EKS, ensuring the correct parameters and payloads are passed to downstream applications.
• Process and validate service responses and publish output messages to the appropriate destination queues for downstream consumption.
• Implement monitoring, logging, error handling, retry mechanisms, and operational alerting to ensure high availability and reliability.
• Create reusable Infrastructure-as-Code templates using Terraform for automated provisioning and deployment.
• Build and validate the solution in the Development environment and support infrastructure provisioning and deployment across UAT and Production environments.
• Collaborate with architects, DevOps engineers, and business stakeholders to ensure solution scalability, security, and maintainability.
• Participate in code reviews, testing, deployment planning, and production support activities.
Required Technical Skills:
• Strong programming experience in Python.
• Hands-on experience with AWS services, including:
- Amazon EKS
- AWS Lambda
- Amazon API Gateway
- Amazon SQS
- Amazon S3
- AWS Event-Driven Architectures and Event Triggers
• Experience with Docker containerization and container orchestration concepts.
• Strong knowledge of Terraform and Infrastructure as Code practices.
• Experience with Kubernetes and kubectl for cluster management and troubleshooting.
• Understanding of CI/CD pipelines, deployment automation, and cloud-native application development.
Preferred Skills:
• Experience working with Snowflake and cloud-based data platforms.
• Knowledge of data engineering, ETL processes, and distributed systems.
• Familiarity with monitoring and observability tools in AWS environments.
• Experience working in Agile development environments and cross-functional teams.
The successful candidate will play a key role in building and operationalizing a modern, event-driven AWS platform that supports scalable data processing and seamless integration across multiple environments.