HUE-8506 [editor] Add options to limit number of fetched results

Review Request #13274 - Created Aug. 17, 2018 and updated

Jean-Francois Desjeans Gauthier
hue
master
HUE-8506
hue
jgauthier

Add 4 new options:

Limit the number of rows that can be fetched by the user interactively. When set to -1 there are no limits.

query_fetch_limit=-1

Cancel query when user reaches last row. The last row is either the last row of the query results or when the limit is reached via query_fetch_limit.

cancel_query_on_end=false

Maximum number of rows fetched per page.

e.g. if query_fetch_page_size is set to 100, rows will be fetched in increment of 100 each time the user scrolls to the end of the rows or prefetched via query_prefetch_results.

query_fetch_page_size=100

Prefetch all the rows in increment of query_fetch_page_size. To be used in conjunction with query_fetch_limit.

query_prefetch_results=False

When all set, the query will be canceled right after the results are returned.

commit 072c6f55c554f3d56b23f37ab3a937f695a6cfb8
Author: jdesjean <jgauthier@cloudera.com>
Date: Thu Aug 16 10:16:43 2018 -0700

HUE-8506 \[editor\] Add options to limit number of fetched results

:100644 100644 bef4db4bb9... 8ff3565d7b... M desktop/conf.dist/hue.ini
:100644 100644 0174080321... c83a852314... M desktop/conf/pseudo-distributed.ini.tmpl
:100644 100644 dc34e0c127... cba0eb4546... M desktop/libs/notebook/src/notebook/api.py
:100644 100644 a62160a978... 3cad8f0c72... M desktop/libs/notebook/src/notebook/conf.py
:100644 100644 3d20f86acc... 97d0e79578... M desktop/libs/notebook/src/notebook/connectors/hiveserver2.py
:100644 100644 b709c35995... dc15bbe0fe... M desktop/libs/notebook/src/notebook/connectors/sqlalchemyapi.py
:100644 100644 815702d996... bb0a10c639... M desktop/libs/notebook/src/notebook/static/notebook/js/notebook.ko.js
:100644 100644 44baabca83... 56229a3458... M desktop/libs/notebook/src/notebook/templates/editor_components.mako

Manual on chrome

  • 0
  • 0
  • 1
  • 2
  • 3
Description From Last Updated
  1. Is the main goal to support SQL Alchemy/JDBC API which don't support fetching a certain number of records at the time? (trying to see if it worths complicating the codebase)

    Typically download row limit is 100k+ of records, so very low probably to fetch via the UI that many.

    1. Goal is to give users option to cancel query after fetching results in order to free up resources (e.g. impala). Currently user experience on download when the query has been canceled (via user, timeout, etc.) is not very good and should be revisited in next commit.

  2. Do we really need this option?

    1. I agree that the download limit is typically 100k+ records. In the latest revision, I changed the config to be a separate row limit from download.

  3. Should be false or users won't be able to download certain queries

  4. 
      
Review request changed
  1. 
      
  2. Can we revisit this?

  3. 
      
Loading...