Study Learn Grow
Real Time Streaming Using Apache Spark Streaming

Real Time Streaming Using Apache Spark Streaming


Spark is the technology that allows us to perform big data processing in the MapReduce paradigm very rapidly, due to performing the processing in memory without the need for extensive I/O operations.

Overview

Description
Analyze data in real-time using the Apache Spark Streaming API.

Spark is the technology that allows us to perform big data processing in the MapReduce paradigm very rapidly, due to performing the processing in memory without the need for extensive I/O operations.

Recently, the streaming approach to processing events in near real time became more widely adopted and more necessary. In this course, you will learn how to handle big amount of unbounded infinite streams of data. You will analyze data and draw conclusions from it. Furthermore, we will look at common problems when processing event streams: sorting, watermarks, deduplication, and keeping state (for example, user sessions). You will also implement streaming processing using Spark Streaming and analyze traffic on a web page in real time.

About the Author

Tomasz Lelek is a software engineer, programming mostly in Java and Scala. A fan of microservices architecture and functional programming, he has dedicated considerable time and effort to getting better every day and has recently delved into big data technologies such as Apache Spark and Hadoop. He is passionate about nearly everything associated with software development; his belief is that we should always try to consider different solutions and approaches before solving a problem. Recently he was a speaker at conferences in Poland, Confitura, and JDD (Java Developers Day), and also at Krakow Scala User Group. He has also conducted a live coding session at Geecon Conference.JDD.

Course Information

A basic understanding and functional knowledge of Apache Spark, stream processing, and big data are required

Implement stream processing using Apache Spark Streaming
Consume events from the source (for instance, Kafka), apply logic on it, and send it to a data sink
Understand how to deduplicate events when you have a system that ensures at-least-once deliver
Learn to tackle common stream processing problems
Create a job to analyze data in real time using the Apache Spark Streaming API
Master event time and processing time
Single event processing and the micro-batch approach to processing events
Learn to sort infinite event streams

The course is for software engineers interested in big data processing.

• Lifetime Access to Each Course
• Certificate on Completion of Course
• No Extra Charges Or Admin Fees
• Easy Access to Courses
• High Priority Support After Sales.
• Big Discounts on Individual Courses

Course Specifications

IT and Computing courses are available to study on our learning platform. 

See All Courses

Adult education is the non-credential activity of gaining skills and improved education. 

See All Courses

Online education is electronically supported learning that relies on the Internet for teacher/student interaction. 

See All Courses

A short course is a learning programme that gives you combined content or specific skills training in a short period of time. Short courses often lean towards the more practical side of things and have less theory than a university course – this gives you a more hands-on experience within your field of interest.

See All Courses

Course duration is 24 hours.

See All Courses

Study Learn Grow

Related Jobs