Key Responsibilities:
. Design, develop, and maintain scalable and efficient data pipelines on AWS.
. Implement data ingestion, transformation, and processing using AWS services such as Glue, Lambda, Kinesis, S3, and Step Functions.
. Work with Redshift, Athena, RDS, DynamoDB, and Snowflake to optimize data storage and retrieval.
. Develop and maintain ETL workflows using AWS Glue, Apache Spark, or Python.
. Ensure data integrity, security, and compliance following best practices.
. Implement and manage CI/CD pipelines for data engineering workloads.
. Collaborate with Data Scientists, Analysts, and Business teams to understand data requirements.
. Optimize performance and cost of data pipelines and cloud storage solutions.
. Monitor and troubleshoot data workflows using CloudWatch, AWS X-Ray, and logging frameworks.
Experience Required:
. Experience in Terraform or CloudFormation for Infrastructure as Code (IaC).
. Working knowledge of Git, Docker, and Kubernetes for deployment and version control.
Strong problem-solving skills with a focus on automation and scalability.
Preferred Qualifications:
. AWS Certified Data Analytics – Specialty or Solutions Architect – Associate.
