MODULE 1 – Introduction to Big Data and Apache Kafka
- Introduction to Big Data
- Big Data Analytics
- Need for Kafka
- What is Kafka?
- Kafka Features
- Kafka Concepts
- Kafka Architecture
- Kafka Components
- ZooKeeper
- Where is Kafka Used?
- Kafka Installation
li>
- Kafka Cluster
- Types of Kafka Clusters
MODULE 2 – Kafka Producer
- Configuring Single Node Multi Broker Cluster
- Constructing a Kafka Producer
- Sending a Message to Kafka
- Producing Keyed and Non-Keyed Messages
- Sending a Message Synchronously & Asynchronously
- Configuring Producers
- Serializers
- Serializing Using Apache Avro
- Partitions
MODULE 3 – Kafka Consumer
- Consumers and Consumer Groups
- Standalone Consumer
- Consumer Groups and Partition Rebalance
- Creating a Kafka Consumer
- Subscribing to Topics
- The Poll Loop
- Configuring Consumers
- Commits and Offsets
- Rebalance Listeners
- Consuming Records with Specific Offsets
- Deserializers
MODULE 4 -Kafka Internals
- Cluster Membership
- The Controller
- Replication
- Request Processing?
- Physical Storage
- Reliability
- Broker Configuration
- Using Producers in a Reliable System
- Using Consumers in a Reliable System
- Validating System Reliability
- Performance Tuning in Kafka
MODULE 5 - Kafka Cluster Architectures & Administering Kafka
- Use Cases - Cross-Cluster Mirroring
- Multi-Cluster Architectures
- Apache Kafka’s MirrorMaker
- Other Cross-Cluster Mirroring Solutions
- Topic Operations
- Consumer Groups
- Dynamic Configuration Changes
- Partition Management
- Consuming and Producing
- Unsafe Operations
MODULE 6 - Kafka Monitoring and Kafka Connect
- Considerations When Building Data Pipelines
- Metric Basics
- Kafka Broker Metrics
- Client Monitoring
- Lag Monitoring
- End-to-End Monitoring
- Kafka Connect
- When to Use Kafka Connect?
- Kafka Connect Properties
MODULE 7 - Kafka Stream Processing
- Stream Processing
- Stream-Processing Concepts
- Stream-Processing Design Patterns
- Kafka Streams by Example
- Kafka Streams: Architecture Overview
MODULE 8 - Integration of Kafka With Hadoop, Storm and Spark
- Apache Hadoop Basics
- Hadoop Configuration
- Kafka Integration with Hadoop
- Apache Storm Basics
- Configuration of Storm
- Integration of Kafka with Storm
- Apache Spark Basics
- Spark Configuration
-
- Kafka Integration with Spark
MODULE 9 - Integration of Kafka With Talend and Cassandra
- Flume Basics
- Integration of Kafka with Flume
- Cassandra Basics such as and KeySpace and Table Creation
- Integration of Kafka with Cassandra
- Talend Basics
- Integration of Kafka with Talend
MODULE 10 - Sample Project