AWS-BI Developer with strong experience across the entire Cloud Data stack. The ideal candidate will have extensive experience in data pipelines (ELT/ETL), data warehousing and dimensional modeling, and curation of data sets for Data Scientists and Business Intelligence users. Experience in building Data Lake. This candidate will also have excellent problem-solving ability dealing with large volumes of data and various data sources.
Primary Responsibilities:
Design, Build and Operate the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, cloud migration tools, and 'big data' technologies.
Optimize various RDBMS engines in the cloud and solve customers' security, performance, and operation problems.
Design, Build, and Operate large, complex data lakes that meet functional / non-functional business requirements.
Optimize various data types of ingestion, storage, processing, and retrieval from near real-time events, and IoT, to unstructured data as images, audio, video, and documents, and in between.
Work with customers' and internal stakeholders including the Executive, Product, Data, Software Development, and Design teams to assist with data-related technical issues and support their data infrastructure and business needs.
Qualifications:
3+ years of experience in a Data Engineer role in a cloud-native eco-system.
Bachelor (Graduate preferred) degree in Computer Science, Mathematics, Informatics, Information Systems, or another quantitative field.
3+ years working experience in AWS Glue ETL, Redshift, Step Function, Athena
Experience in implementing data pipelines for both streaming and batch integrations using tools/frameworks like Glue ETL, Lambda, Spark, Spark Experience in Lake Formation Orchestration, Maintain and enhance models for SDLC using CI/CD pipelines in AWS.
Experience in Relational and NoSQL databases, such as MySQL or Postgres and DynamoDB or Cassandra.
Functional and scripting languages: Python, Advanced SQL.
Experience building and optimizing 'big data' pipelines, architectures, and data sets.
We are an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other category protected by law.
