[editor] Propagate hasResultSet changes.

Review Request #14542 — Created Nov. 22, 2019 and submitted

jgauthier
hue
master
HUE-9089
hue
jgauthier, johan, ranade, romain, weixia, yingc
commit e9bd35e5d6eedcd93e1f85c147d1c4cdb9878660
Author: Jean-Francois Desjeans Gauthier <jf.desjeans.gauthier@gmail.com>
Date:   Fri Nov 22 15:57:14 2019 -0800

    HUE-9089 [editor] Propagate hasResultSet changes.

:100644 100644 1f535adf9c... 07c0ef9e8e... M	desktop/core/src/desktop/js/api/apiHelper.js
:100644 100644 b7c25e90d9... 2cc301ecaa... M	desktop/core/src/desktop/js/apps/notebook/snippet.js
:100644 100644 20261c2305... ae67cdf731... M	desktop/libs/notebook/src/notebook/api.py
:100644 100644 9a0ca9037b... 9a342c554e... M	desktop/libs/notebook/src/notebook/connectors/hiveserver2.py


  • 0
  • 0
  • 1
  • 1
  • 2
Description From Last Updated
romain
  1. 
      
  2. Is there more background on when/what it changes? (could not see in the jira)

    1. When running agains't LLAP, I noticed that a query using runAsync=True https://github.com/cloudera/hue/blob/b7a9bd5834fe4f23504df47ad74e0dfba6698105/apps/beeswax/src/beeswax/server/hive_server2_lib.py#L953 returned no results. I then noticed that hasResultSet=False in TExecuteStatementResp contrary to TEZ which returned hasResultSet=True. When hasResultSet=False, we do not fetch results https://github.com/cloudera/hue/blob/b7a9bd5834fe4f23504df47ad74e0dfba6698105/apps/beeswax/src/beeswax/server/hive_server2_lib.py#L993.
      I asked Gopal Vijayaraghavan and he mentioned that as part of HIVE-12442 an incompatible change was made and that getOperationStatus needed to be called to obtain the latest value of hasResultSet. He said, this was due to HS2 multi-threaded compilation support.
      He also pointed me to this line, where the JDBC driver is doing the check: https://github.com/apache/hive/blob/master/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java#L348.
      I found some information here: https://community.cloudera.com/t5/Community-Articles/Hiveserver2-Hung-issues-in-Compilation-Phase/ta-p/248566. My understanding is that after TExecuteStatementResp, LLAP has not yet parsed the query and doesn't know the value of hasResultSet.

  3. 
      
johan
  1. 
      
  2. Move to backend?

    1. Can you explain what you mean?

    2. If no snippet is provided, the backend correctly handles the case, but currently the frontend is sending this information so it needs to be up to date. When the frontend stops sending the snippet then this code won't be needed.

    3. E.g. if we drop https://github.com/cloudera/hue/blob/b7a9bd5834fe4f23504df47ad74e0dfba6698105/desktop/core/src/desktop/js/api/apiHelper.js#L2436 then https://github.com/cloudera/hue/blob/b7a9bd5834fe4f23504df47ad74e0dfba6698105/desktop/libs/notebook/src/notebook/api.py#L289 will handle.

    4. That being said, the front end looks at has_result_set to determine if it needs to fetch results when the query is in state 'available'.

    5. Seeing as this is just for fetchSample, we can drop https://github.com/cloudera/hue/blob/b7a9bd5834fe4f23504df47ad74e0dfba6698105/desktop/core/src/desktop/js/api/apiHelper.js#L2436

    6. Actually for samples, we don't create a notebook (so no UUID). So we have to do this.

  3. 
      
jgauthier
jgauthier
  1. 
      
  2. After chat, Johan said it was ok.

  3. 
      
jgauthier
Review request changed

Status: Closed (submitted)

Loading...