Experience: 8+ years of relevant experience in building cloud-native, hybrid, or multi-cloud solutions.
Expertise in production grade solutions using: AWS (Redshift, S3, Glue, Lambda, Airflow), Pyspark, Python, Data pipelines using Glue/Airflow/Sagemaker.
Hands-on experience in AWS Cloud with services such as AWS Glue, AWS Redshift, DynomoDB, AWS kinesis family, AWS RDS, AWS EC2, Lambda, S3 for ETL.
Proficient in writing Python/Pyspark and SQL scripts.
Experience in SQL and Query optimization. Proficient in SQL-based technologies (MySQL, Oracle DB,SQL Server etc.)
Exposure to ETL tools (DBT, Matillion) and Data warehouse.
Experience in designing relational, multidimensional , and Master data management systems.
Exposure in designing solution based on medallion architecture, Lake house
Experience in creation and maintenance of data dictionaries, metadata repositories, and data lineage documentation.
Strong skills to mentor and manage teams of junior and senior data engineers and leading end to end delivery of technical workloads.
Experience using Github/codecommit for developmental activities
Experience in unit testing python codes
Experience in building data pipelines using Glue/Airflow/Sagemaker pipelines would be preferable
Good to have skills:
Experience in leading enterprise grade ML model migration workloads from on-premise/GCP to AWS.
Prior experience of planning migration roadmaps for production workloads deployed on-prem or on IAAS provider infrastructure to AWS.
Ability to collaborate with cross-functional teams such as Developers, QA, Project Managers, and other stakeholders to understand their requirements and implement solutions.
Ability to clearly communicate technical roadmap, challenges and mitigation.
Established data governance policies, procedures, and standards.