HUE-9046 [jb] Hive query history

Review Request #14393 — Created Oct. 22, 2019 and submitted

jgauthier
hue
master
HUE-9046
hue
jgauthier, johan, ranade, romain, weixia, yingc
commit c5cbc6379f7cc6f368729bd9c581f5fd735e4b2e
Author: Jean-Francois Desjeans Gauthier <jf.desjeans.gauthier@gmail.com>
Date:   Tue Oct 22 09:35:21 2019 -0700

    HUE-9046 [jb] Hive query history
    Change-Id: I189a503ef40dfd5e669fc60cf1dedb92273870e4

:100644 100644 0e3fadc4ce... 9d044da506... M	apps/beeswax/src/beeswax/hive_site.py
:000000 100644 0000000000... 35b0c0e840... A	apps/beeswax/src/beeswax/management/commands/create_table_query_data.py
:000000 100644 0000000000... 2d855f2684... A	apps/beeswax/src/beeswax/templates/create_table_query_data.mako
:000000 100644 0000000000... 508e88fe3c... A	apps/beeswax/src/beeswax/templates/msck.mako
:000000 100644 0000000000... 4a0fe4eac0... A	apps/beeswax/src/beeswax/templates/select_table_query_data_from.mako
:000000 100644 0000000000... 05e4454647... A	apps/beeswax/src/beeswax/templates/select_table_query_data_latest.mako
:100644 100644 2710990ec4... 03cd3b4201... M	apps/jobbrowser/src/jobbrowser/apis/base_api.py
:100644 100644 084b18e4a3... d220d6e13f... M	apps/jobbrowser/src/jobbrowser/templates/job_browser.mako
:100644 100644 a1580355d6... 15dd900c0e... M	desktop/core/src/desktop/lib/python_util.py
:100644 100644 ec37e4de15... ea0ec42f9c... M	desktop/libs/indexer/src/indexer/templates/gen/create_database_statement.mako

Need to
1) Add unit tests
2) Add link from editor to job browser
3) Ability to close queries
4) Move to Queries tab
5) Blacklist

  • 1
  • 0
  • 5
  • 0
  • 6
Description From Last Updated
We should revisit caching at some point and remove most of them (caching without expiration is not scalable, and in ... romain romain
jgauthier
jgauthier
jgauthier
jgauthier
romain
  1. +1 to see if we can avoid forking with one more type of interface. Probably better to do it from the start.

  2. Could keep 2 tabs for now, but maybe everything else we refactor with one sub level?

    e.g.
    queries-impala?
    queries-hive?

    So that we combine and start converging SQL queries browser. Fine to have an impala-tree specific for now, etc.

  3. 
      
jgauthier
jgauthier
romain
  1. Probably better to keep it minimal for now and not do "3) Ability to close queries" and just have the link from the editor.

    To keep in mind that the "API calls" will hang Hue until Task Server is there (and so it high priority).

  2. apps/beeswax/src/beeswax/query_history.py (Diff revision 3)
     
     
     

    We should revisit caching at some point and remove most of them (caching without expiration is not scalable, and in k8s world not useful if not at a global level via another service). Same for the locks.

  3. nit: Could indend on multiple line cf. below

  4. nit: could add more new lines to indent and make it easier for humane to read, e.g.
    
      proposed_query = django_mako.render_to_string(
        "select_table_query_data_latest.mako", {
          'table': {
            'name': 'query_data',
            'request_user': request_user, 
            'query_id': query_id, ...
  5. apps/jobbrowser/src/jobbrowser/apis/base_api.py (Diff revision 3)
     
     
     
     
     

    And at some point we could do a sub SQL query api level:

    e.g.

    jobbrowser/apis/query_api/api.py
    jobbrowser/apis/query_api/beeswax_query_api.py
    jobbrowser/apis/query_api/impala_query_api.py
    jobbrowser/apis/query_api/..._query_api.py
    jobbrowser/apis/worklow_api.py
    ...

    That way the new engines (Phoenix, Druid...) could be added more easily

  6. 
      
jgauthier
jgauthier
jgauthier
jgauthier
jgauthier
Review request changed

Status: Closed (submitted)

Loading...