AWS Big Data Lead Software Engineer

Job description

Job responsibilities - Design & build new applications utilizing leading edge technologies and modernize existing applications - Implement batch & real-time software components consistent with architectural best-practices of reliability, security, operational efficiency, cost-effectiveness and performance - Ensure quality of deployed code via automated unit, integration & acceptance testing - Collaborate with multi-national agile development, support and business teams to meet sprint objectives - Participate in all agile meetings & rituals, including daily standups, sprint planning, backlog reviews, demos, and retrospectives - Provide level 2 support for production systems - Learn and applies system processes, methodologies, and skills for the development of secure, stable code and systems - Hands on applicaitn development leveraging distributed compute such as Apache flink or Spark on very large datasets - Design and development of applications that leverage the AWS infrastructure deploying software components on AWS using common compute and storage services such as EC2, EKS, Lambda, S3 - Lead and deliver projects from concept to production across PNI (Personalization and Insights) platform Required qualifications, capabilities, and skills - Formal training or certification on software engineering concepts and 5+ years applied experience - Hands-on practical experience in Frameworks, system design, application development, testing, and operational stability - Experience with Apache Spark, Apache Flink or similar large-scale data processing engines - Experience with Distributed Datastores (e.g. Cassandra, Red Shift) - Experience designing, developing and deploying software components on AWS using common compute and storage services such as EC2, EKS, Lambda, S3 - Experience with Big Data / Distributed / cloud technology (AWS Big data services like lambda, glue, glue emr, Performance tuning, Streaming, KAFKA, Entitlements etc., ) - Experience with Apache Spark, Ray or similar large-scale data processing engines - Proficiency in automation and continuous delivery methods - Proficient in all aspects of the Software Development Life Cycle - Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security - Demonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, BigData, artificial intelligence, machine learning, mobile, etc.) Preferred qualifications, capabilities, and skills - Experience building ETL/Feature processing pipelines - Experience using workflow orchestration tools—Airflow, Kubeflow etc. - Experience using Terraform to deploy infrastructure-as-code to public cloud - Experience with Linux scripting such as Bash, KSH, or Python - Certified AWS Cloud Practitioner, Developer or Solutions Architect strongly preferred