The Big Data Engineer will be responsible for interpreting the requirements of various Big Data Analytic Use Cases and Scenarios, and driving the design and implementation of specific data models to ultimately help drive better business decisions through insights from a combination of external and AT&T's data assets. This role is also accountable for developing the necessary enablers and data platform in the Big Data Lake Environment and has the responsibility of maintaining its integrity during the life cycle phases. Automate Machine Learning pipelines using disparate Enterprise data sources, Support and Mentor Data Science, Date Engineering resources, Become Data Source SME, Prototype using new technologies and platforms. Assess quality of data points from disparate datasets keeping in mind volume and time to market.

Key Roles and Responsibilities

This position will define data requirements, gather and mine large scale of structured and unstructured data, and validate data by running various data tools in the Big Data Environment. This function needs to support the standardization, customization and ad-hoc data analysis, and will develop the mechanisms to ingest, analyze, validate, normalize and clean data. Implements statistical data quality procedures on new data sources, and by applying rigorous iterative data analytics, supports Data Scientists in data sourcing and preparation to visualize data and synthesize insights of commercial value. Will work with Big Data Policy and Security teams and Legal to create data policy and develop interfaces and retention models which requires synthesizing or anonymizing data. Develop and maintain data engineering best practices and contribute to Insights on data analytics and visualization concepts, methods and techniques.

Qualifications

Bachelors of Science in Computer Science, Math or Scientific Computing preferred. Typically requires 5-8 years experience. Azure Data Engineer, Azure Data Scientist, Azure AI Engineer, Spark, DataBricks, Data Orchestration Frameworks required. AWS, Kafka, SAFEAgile preferred.

.

Principal Functional Skills / Competencies associated with this Title:

• Agile Development
• Application Programming Interface (API)
• Big Data Software Engineering
• Business Assessment
• Data Analysis - Software
• Data Mining and Data Science
• Data Movement Tools
• Data Privacy
• Emerging Technologies
• Informatics/Information Sciences
• Information Security Management
• Logical Data Modeling
• Modeling: Data, Process, Events, Objects
• Requirements Analysis
• Software Development Life Cycle
• Statistics and Actuarial Modeling
• Testing