Fix for HUE-2092

Review Request #4476 - Created July 3, 2014 and discarded

Chris Conner
old-hue
master
HUE-2092
hue
abec, enricoberti, romain
So I wonder if there are two different issues here.  The issue in the thread:

9:44:32 AM	INFO	middleware	
Processing exception: [Errno 2] Copy src 'lib/atlasRemoveEntitlementJob-1.0.0-SNAPSHOT.jar' does not exist: Traceback (most recent call last):
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/apps/oozie/src/oozie/decorators.py", line 52, in decorate
    return view_func(request, *args, **kwargs)
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/apps/oozie/src/oozie/views/editor.py", line 335, in submit_workflow
    job_id = _submit_workflow(request.user, request.fs, request.jt, workflow, mapping)
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/apps/oozie/src/oozie/views/editor.py", line 356, in _submit_workflow
    job_id = submission.run()
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/desktop/libs/liboozie/src/liboozie/submittion.py", line 79, in run
    deployment_dir = self.deploy()
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/desktop/libs/liboozie/src/liboozie/submittion.py", line 144, in deploy
    self._do_as(self.user.username , self._copy_files, deployment_dir, oozie_xml)
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/desktop/libs/liboozie/src/liboozie/submittion.py", line 281, in _do_as
    return fn(*args, **kwargs)
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/desktop/libs/liboozie/src/liboozie/submittion.py", line 275, in _copy_files
    self.fs.copyfile(jar_file, jar_lib_path)
  File "/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line 480, in copyfile
    raise IOError(errno.ENOENT, _("Copy src '%s' does not exist") % src)
IOError: [Errno 2] Copy src 'lib/cluster-build-cdh5-1.0.0.jar'

happens when you create a workflow outside of the default location, IE:

/user/cconner/hue-2092

And then put a jar in the lib directory:

/user/cconner/hue-2092/lib/cluster-build-cdh5-1.0.0.jar

Then the Hue code removes it before trying to copy it. 
Tested to make sure it resolves the issue and still works with jars with a full path and workflows in the default location where Hue creates them.
  • 1
  • 0
  • 0
  • 0
  • 1
Description From Last Updated
If we have /user/cconner/hue-2092/lib/cluster-build-cdh5-1.0.0.jar we are not going to copy it in the workspace of the workflow it seems? If ... Romain Rigaux
  1. Thanks!
    
    I am not 100% convinced, cf. the comment? I am going to create a test case for _copy_files that way we can test all the combos easily. I don't trust this function anymore.
  2. If we have /user/cconner/hue-2092/lib/cluster-build-cdh5-1.0.0.jar
    
    we are not going to copy it in the workspace of the workflow it seems?
    
    If like this it will go in the if:
    /user/cconner/hue-2092/AAAAAAA/cluster-build-cdh5-1.0.0.jar
  3. 
      
Review request changed

Status: Discarded

Change Summary:

Old, not sure if this was finished or not.

Loading...