Key Responsibilities:
- Design and implement robust data ingestion pipelines to AWS cloud services such as Amazon S3, Redshift, and Amazon EC2.
- Develop and maintain AWS Glue ETL jobs for data extraction, transformation, and loading.
- Configure and manage AWS IAM roles and policies to ensure secure access control to AWS resources.
- Implement serverless data processing solutions using AWS Lambda functions.
- Integrate AWS IoT services for streaming data ingestion and processing.
- Manage metadata and data catalogs using AWS Glue Catalog for efficient data discovery and governance.
- Utilize Kinesis Analytics for real-time data processing and analysis.
- Integrate data ingestion pipelines with Enterprise Service Bus (ESB) for seamless data integration across systems.
- Optimize data ingestion processes for performance, scalability, and cost-effectiveness.
- Monitor, troubleshoot, and resolve issues related to data ingestion pipelines and AWS services.
Qualifications:
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- Proven experience in designing and implementing data ingestion solutions to AWS cloud services.
- Hands-on experience with AWS services such as AWS Glue, AWS Lambda, AWS IAM, AWS IoT, Kinesis Analytics, and Redshift.
- Familiarity with data modeling, ETL processes, and data warehousing concepts.
- Experience with enterprise integration patterns and technologies, including Enterprise Service Bus (ESB).
- Strong programming skills in languages such as Python, Java, or Scala.
- Excellent problem-solving skills and the ability to troubleshoot complex issues.
- Strong communication and collaboration skills, with the ability to work effectively in a team environment.
Preferred Qualifications:
- AWS certification(s) such as AWS Certified Solutions Architect or AWS Certified Data Analytics – Specialty.
- Experience with big data technologies such as Apache Spark, Apache Kafka, or Apache Hadoop.
- Knowledge of containerization and orchestration tools such as Docker and Kubernetes.