HUE-7941 [impala] Reset the status of the query link for new queries

Review Request #12509 - Created Feb. 9, 2018 and submitted

Jean-Francois Desjeans Gauthier
hue
master
HUE-7941
hue
jgauthier

commit 2d02dd8a4e8d247add483ba6ea20ef7b7284ff41
Author: jdesjean <jgauthier@cloudera.com>
Date: Fri Feb 9 15:46:32 2018 -0800

HUE-7941 \[impala\] Reset the status of the query link for new queries

:100644 100644 9d90b21912... 49f3a61174... M desktop/libs/notebook/src/notebook/api.py
:100644 100644 a8fcb54524... 715987a84a... M desktop/libs/notebook/src/notebook/static/notebook/js/notebook.ko.js
:100644 100644 4d5301a623... dc06ab9a03... M desktop/libs/notebook/src/notebook/templates/editor_components.mako



  • 0
  • 0
  • 2
  • 2
  • 4
Description From Last Updated
  1. The original fix was to just reset the progress on a new query, if we want to also report the progress, let's see how we could do it without changing too much.

  2. desktop/libs/notebook/src/notebook/api.py (Diff revision 1)
     
     
     
     

    Why isn't it in get_jobs() instead?

  3. Why would we need this know?

    1. We look at the job browser job's list and updates the progress of the jobs in the editor. If no jobs are listed in the job browser, we assume that the jobs in the editor are complete. With Impala, there are no jobs in the job browser, so we incorrectly set the progress to 100%. We should not update the progress of Impala jobs in the editor with jobs in the jobs browser.

  4. 
      
  1. Commit title is not that clear, reuse the one from the jira or improve it?

    Wouln't it be simply a conflict with below which was designed for Hive only (which can have many jobs)

      huePubSub.subscribe('jobbrowser.data', function (jobs) {
        if (jobs.length > 0) {
          jobs.forEach(function (job) {
            if ($("#" + job.shortId).length > 0) {
              var _job = ko.dataFor($("#" + job.shortId)[0]);
              if (!isNaN(parseInt(job.mapsPercentComplete))) {
                _job.percentJob(parseInt(job.mapsPercentComplete));
              }
            }
          });
        } else {
          if (viewModel.selectedNotebook()) {
            viewModel.selectedNotebook().snippets().forEach(function (snippet) {
              snippet.jobs().forEach(function (job) {
                job.percentJob(100);
              });
            });
          }
        }
      }, HUE_PUB_SUB_EDITOR_ID);
    

    In case of Impala, we could ignore any progress statuts for now, and integrate it via the same 'jobbrowser.data' subscribe

    1. You're correct, my previous fix needs to go in. I'm actually not sure how come it got removed.

    2. ping

  2. Why we would need it now? (Impala typically has only one query id)

    1. This needs to go in as well.

      There's 2 problem:
      1) Impala query status gets updated by jobs status (which is incorrect). This is resolved by your previous comment.
      2) There's currently nothing updating the impala job status. This is fixed via adding the status in get_logs & here. This code is required, not because there can be multiple jobs, but because once percentJob is created on the object it needs to be updated every time we call get_logs

  3. 
      
  1. Ship It!
  2. 
      
Review request changed

Status: Closed (submitted)

Loading...