HUE-9150 [importer] Avoid creating scratchdir outside of encryption zone

Review Request #14771 — Created Feb. 21, 2020 and updated

yingc
hue
master
HUE-9150
johan, quadoss, ranade, romain, weixia
commit 3fef410d8e4190498b01c3e304d36fe27a32033e (HEAD -> master)
Author: Ying Chen <yingchen@cloudera.com>
Date:   Wed Feb 12 19:34:17 2020 -0800

    HUE-9150 [importer] Avoid creating scratchdir outside of encryption zone

:100644 100644 999bd9e0cc... d687495462... M    desktop/libs/hadoop/src/hadoop/fs/webhdfs_types.py
:100644 100644 18f9dc5928... 34e6cb8652... M    desktop/libs/indexer/src/indexer/indexers/sql.py
:100644 100644 336e8eff3d... 2622ea8c17... M    desktop/libs/indexer/src/indexer/indexers/sql_tests.py


  • 0
  • 0
  • 5
  • 2
  • 7
Description From Last Updated
yingc
romain
  1. Nice!

    Just a few nits

  2. Variable needed?
    
    + new line like before
  3. "# Only for HDFS, import data and non-external table" --> "# For import data or non-external table creation, validate that hive has the perms to move the files
    # Note: the user still needs to upload the data in the same encryption zone as the warehouse."
  4. return self.fs.do_as_user('hive', self.fs.check_access, path, 'rwx')

    ?

    1. It return None if permission is ok, but throws exception when no access. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Check_access
  5. test_create_table_from_a_file_to_csv_with_enc --> test_create_table_from_a_file_to_csv_with_security_and_kms_encryption

  6. check_access=Mock(check_access=Mock(side_effect=WebHdfsException("check access denied!"))),

    ?

  7. Add another call/test with check_perm_for_hive returning True?

    Could also add another call with stats.encBit == False

  8. 
      
yingc
romain
  1. 
      
  2. Split in a new function with '_kms_encryption' removed?

    @patch('uuid.uuid4', mock_uuid)
    def test_create_table_from_a_file_to_csv_with_security_and(self):

  3. 
      
yingc
Review request changed

Change Summary:

  1. split new unit test function
  2. reuse get_source and get_destination
  3. fix chown to the right user bases on source_type

Diff:

Revision 4 (+119 -12)

Show changes

Loading...