HUE-9089 [hive] Upgrade Thrift protocol to V11 to support LLAP

Review Request #14531 — Created Nov. 22, 2019 and discarded

jgauthier
hue
master
HUE-9089
hue
jgauthier, johan, ranade, romain, weixia, yingc

commit 918a6f44ebf493e3ec01e88fa10dad017909e22f
Author: Jean-Francois Desjeans Gauthier <jf.desjeans.gauthier@gmail.com>
Date: Wed Nov 20 11:52:13 2019 -0800

HUE-9089 \[hive\] Upgrade Thrift protocol to V11 to support LLAP

As part of task support, LLAP requires HIVE_CLI_SERVICE_PROTOCOL_V8 and changes the behavior of hasResultSet in TOperationHandle as it can be updated in TGetOperationStatusResp. This is specific to LLAP as TEZ keeps the old behavior of setting hasResultSet in TExecuteStatementResp. Some changes are made in notebook.api.check_status to update hasResultSet.
In further commits, we might want to also use TProgressUpdateResp that came in as part of HIVE_CLI_SERVICE_PROTOCOL_V10 to track query progress as an alternative to looking at Yarn for job progress.

:100755 100755 e2c6ed40e7... ca6770ed84... M apps/beeswax/gen-py/TCLIService/TCLIService-remote
:100644 100644 d802603128... 9a58642367... M apps/beeswax/gen-py/TCLIService/TCLIService.py
:100644 100644 4bc79fa98d... a2daff3d25... M apps/beeswax/gen-py/TCLIService/constants.py
:100644 100644 7036f498fc... 849ac5df1e... M apps/beeswax/gen-py/TCLIService/ttypes.py
:100644 100644 28ea572096... 80548d9b2f... M apps/beeswax/src/beeswax/conf.py
:100644 100644 74d652089e... b5482559de... M apps/beeswax/thrift/TCLIService.thrift
:100644 100644 c1920021dd... 3042aaee06... M desktop/core/src/desktop/js/apps/notebook/notebook.js
:100644 100644 b7c25e90d9... 2764603da2... M desktop/core/src/desktop/js/apps/notebook/snippet.js
:100644 100644 2f9cd3e760... eca58b5747... M desktop/libs/notebook/src/notebook/api.py
:100644 100644 9a0ca9037b... 9a342c554e... M desktop/libs/notebook/src/notebook/connectors/hiveserver2.py

On Impala, TEZ & LLAP
create table, insert into table , select * from

On SQLAlchemy
select * from

Left assist
get databases, tables & samples

  • 2
  • 0
  • 3
  • 0
  • 5
Description From Last Updated
Also fix in notebook2/notebook.js updateHistoryCall() ? romain romain
Will have to check in notebook2 romain romain
jgauthier
jgauthier
jgauthier
romain
  1. Nice! Quick first review, sending as many review updates in the meantime.

    BTW: splittings in multiple commits Thrift update, Notebook id, Notebook status changes would make it bit simpler to review

  2. apps/beeswax/src/beeswax/conf.py (Diff revision 1)
     
     

    Update inis?

  3. Also fix in notebook2/notebook.js updateHistoryCall() ?

    1. I see loading query from history doesn't load results, let me fix this.

    2. Looking into this it looks like LLAP doesn't support fetching from the beginning.

  4. Will have to check in notebook2

    1. Code looks fine in notebook2, but need to test.

  5. I see, you fixed the update History above in notebook

  6. 
      
romain
  1. Quick review

  2. We probably should fix in the API to make it smoother?

  3. 
      
jgauthier
Review request changed

Status: Discarded

Loading...