HUE-7258 [jb] Fix fetching Spark job logs in Kerberized environment

Review Request #12446 - Created Feb. 1, 2018 and submitted

Ying Chen
hue
master
HUE-7258
hue
enricoberti, jgauthier, johan, krish, romain, weixia
commit 4d882d33d1bb86103939d64ded7e2f0274aa12be 
Author: Ying Chen <yingchen@cloudera.com>
Date:   Thu Feb 1 10:17:15 2018 -0800

    HUE-7258 [jb] Fix fetching Spark job logs in Kerberized environment

:100644 100644 aaa45baa69... b996057f2c... M    apps/jobbrowser/src/jobbrowser/apis/job_api.py
:100644 100644 ffd6ccf902... e715471cb2... M    apps/jobbrowser/src/jobbrowser/tests.py
:100644 100644 6c2d46c381... 83b35eb09b... M    apps/jobbrowser/src/jobbrowser/yarn_models.py
:100644 100644 aa2e01a993... 8f300cc394... M    desktop/core/src/desktop/lib/rest/resource.py
:100644 100644 272969e405... c67c320294... M    desktop/libs/hadoop/src/hadoop/yarn/spark_history_server_api.py

spark-conf/spark-history-server.conf:
* spark.history.ui.admin.acls=hue


  • 0
  • 0
  • 7
  • 0
  • 7
Description From Last Updated
  1. 
      
  2. self.url or self.trackingUrl ?

    or

    hostname of self.trackingUrl here
    and remaining replacing '' in the exectue()?

  3. Set to true and still error on the GET?

    1. pushing new changes to Kerberos-enabled cluster for test
  4. Then do we get the correct page content back?

    1. resp = urllib2.urlopen(self.trackingUrl, timeout=5.0)
      actual_url = resp.url
      
      It will get proxy URL to redirected URL which will be the base URL for executors, but HttpClient only retures the page content.
  5. 
      
  1. Nice!

    1. I think it's safe. Before previous commit, there is offset parameter in other api's, so default it as LOG_OFFSET_BYTES.
      Trying to keep it as the change as HUE-7142: self._get_api(appid).logs(appid, app_type, log_name, is_embeddable).
  2. apps/jobbrowser/src/jobbrowser/yarn_models.py (Diff revision 3)
     
     
     
     
     
     
     

    Maybe remove !

  3. No internationalization for logging info

  4. desktop/core/src/desktop/lib/rest/resource.py (Diff revision 3)
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     

    Move to util methode to avoid duplication?

  5. 
      
  1. Unit test still green?

    1. Fix minor in test. Passed Build #213 in master.
  2. 
      
  1. Ship It!
  2. 
      
Review request changed

Status: Closed (submitted)

Loading...