Steve Hoffman - Apache Flume: Distributed Log Collection for Hadoop - Second Edition [2015, ePub, ENG]

1: Overview and Architecture
Flume 0.9
Flume 1.X (Flume-NG)
The problem with HDFS and streaming data/logs
Sources, channels, and sinks
Flume events
The Kite SDK
Summary

2: A Quick Start Guide to Flume
Downloading Flume
An overview of the Flume configuration file
Starting up with "Hello, World!"
Summary

3: Channels
The memory channel
The file channel
Spillable Memory Channel
Summary

4: Sinks and Sink Processors
HDFS sink
Compression codecs
Event Serializers
Sink groups
MorphlineSolrSink
ElasticSearchSink
Summary

5: Sources and Channel Selectors
The problem with using tail
The Exec source
Spooling Directory Source
Syslog sources
JMS source
Channel selectors
Summary

6: Interceptors, ETL, and Routing
Interceptors
Tiering flows
The embedded agent
Routing
Summary

7: Putting It All Together
Web logs to searchable UI
Archiving to HDFS
Summary

8: Monitoring Flume
Monitoring the agent process
Monitoring performance metrics
Summary

9: There Is No Spoon – the Realities of Real-time Distributed Data Collection
Transport time versus log time
Time zones are evil
Capacity planning
Considerations for multiple data centers
Compliance and data expiry
Summary