Big Data Engineering for Analytics

Introduction

This course helps data engineers focus on essential design and architecture while building a data lake and relevant processing platform. Participants will learn various aspects of data engineering while building resilient distributed datasets. Participants will learn to apply key practices, identify multiple data sources appraised against their business value, design the right storage, and implement proper access model(s).

Course Objectives of Big Data Engineering for Analytics

• Understand the fundamental characteristics, storage, analysis techniques, and the relevant distributions.
• Gain expertise with the fault-tolerant computing framework.
• Construct configurable and executable tasks.
• Understand the nuances of writing functional programs.
• Understand various data processing, querying, and persistence available for usage in RDD’s context. 
• Perform tasks such as filtering, selection, and categorization.

Course Outlines of Big Data Engineering for Analytics

Day 1

Data Science, Data Engineering, Big Data, and Analytics Perspective

• Introduction to Data Science, Data Engineering, and Big Data.
• Data Scientist vs. Data Engineer.
• Different Roles in Data Engineering.
• Core Data Engineering Skills and Resources.
• Understand Big Data from an Analytics Perspective.

Day 2

Architectural Viewpoints and Hadoop Ecosystem

• Architectural Viewpoints in Big Data.
• Reference Architecture Conceptual View.
• Reference Architecture Logical View.
• Oracle Product Mapping View.
• The Hadoop Ecosystem for Big Data.

Day 3

File Storage and Databases for Big Data

• Distributed File Storage.
• NoSQL Databases for Big Data.
• Spark and Functional Programming for Big Data.

Day 4

Management of Big Data

• Spark and Resilient Distributed Data Sets.
• Spark QL for Big Data.
• Spark and Real-Time Stream Processing.
• Management of Big Data initiatives.

Day 5

Dealing with a case study

• Case study.
• Project Requirement Elaboration.
• Project and Assessment.
• Project Demonstration.
• Report Submission and Presentations.