Key Responsibilities:
• Cloud & Data Architecture: Design and implement cloud-based data models, ETL pipelines, and multi-cluster architectures (on-premises to cloud, cloud-to-cloud).
• Big Data Technologies: Work extensively with Apache Spark, Hive, HDFS, MapReduce, Apache Kafka, and AWS Kinesis to build scalable data solutions.
• DevOps & Automation: Implement and maintain CI/CD pipelines using Jenkins, Git, and other automation tools for data deployment and workflow orchestration.
• Programming & Development: Develop high-performance distributed computing solutions using Java, Python, and Scala.
• Data Management: Handle structured and unstructured data, ensuring data governance, data masking, and efficient data modeling.
• Database & Optimization: Optimize SQL-based databases, design high-throughput data ingestion pipelines, and ensure efficient query performance.
• Infrastructure & Tools: Manage and enhance Hadoop ecosystems, Hive data warehousing, and work with AWS, GCP, UNIX, Windows, and version control tools (Git, Subversion).
• Security & Compliance: Ensure data security best practices in cloud-based environments and comply with industry regulations for data privacy and protection.
