Review Board 2.0.8


FLUME-685: Flume requires restart after HBase goes down

Review Request #1847 - Created June 27, 2011 and updated

Information
Alex Baranau
old-flume
FLUME-685
Reviewers
flume
\* added retry after put fails with IOException.
\* did small refactoring along the way (related to the changes) extracted base class for hbase sinks with common functionality: open/close sink, writing with retries, etc.

  
Review request changed
Updated (Aug. 22, 2011, 4:35 p.m.)

Change Summary:

Updated diff based on https://review.cloudera.org/r/1900/. Fixed nit mentioned before

Diff:

Revision 2 (+14 -2)

Show changes

Posted (Aug. 24, 2011, 8:33 p.m.)



  
Was curious what happens when interrupted, saw that hbase client converts into IOException.  (HConnectionManager.processBatchOfPuts in HBase 0.90.4) 

This is actually troublesome because we'll never be able to cleanup / reconfigure this logical node if the put gets into an error state\! 

Unfortunately, this might require changing HBase.  

Also, using Flume's native recovery would suffer from the same problem.

Can you add this caveat to the docs?  (Also can you test to verify this?)
Posted (Aug. 30, 2011, 10:58 a.m.)



  
A somewhat related bit: The more you want to get away with exceptions, the more you move closer to Asynchronous HBase Client. It is used in Flume-OpenTSDB sink too. Its benefits are here: http://groups.google.com/group/opentsdb/browse\_thread/thread/c952c5b2a057ae3a?pli=1