HUE-6915 [importer] Extending support of DBMS ingestion made easy to Teradata

Review Request #11361 - Created Aug. 15, 2017 and updated

Prachi Poddar
hue
HUE-6915
69517f9...
romain

Please ignore the changes in following files:
apps/security/src/security/api/hive.py
apps/security/src/security/api/sentry.py
desktop/libs/indexer/src/indexer/urls.py


This also fixes the following JIRA:
HUE-6914 [importer] Extending support of "DBMS ingestion made easy" to PostGreSQL

commit 884c1231509d90bb4d53276692f3d191dc963d23
Author: Prachi Poddar <prachi.poddar@cloudera.com>
Date: Mon Aug 14 09:56:27 2017 -0700

\[importer\] Extending support of "DBMS ingestion made easy" to Teradata

:100644 100644 0a48a6d086... 6a4b16d5cc... M desktop/libs/indexer/src/indexer/api3.py
:100644 100644 d32f7ca6a0... d254fb197b... M desktop/libs/indexer/src/indexer/indexers/rdbms.py
:100644 100644 4278b6c6e0... c3a9cc367b... M desktop/libs/indexer/src/indexer/templates/importer.mako
:100644 100644 a774b86504... 816d01b413... M desktop/libs/notebook/src/notebook/connectors/base.py
:100644 100644 5c63072b08... d11f33a95a... M desktop/libs/notebook/src/notebook/connectors/jdbc.py

The change in desktop/libs/notebook/src/notebook/connectors/jdbc.py file breaks the auto populate of databases/tables in the left pane in Teradata/PostgreSQL editor. Need to fix that. I'll create seperate JIRA once this gets pushed in.

Well tested with JDBC (both custom and config) and non-JDBC (both custom and config) for Teradata as well as PostgreSQL.

  • 9
  • 0
  • 6
  • 0
  • 15
Description From Last Updated
Could we avoid changing the API? (why do we need this) Romain Rigaux
We need this as the order of the column names in the list which we get from "columns" is different ... Prachi Poddar
Same Romain Rigaux
It is used just once in get_columns() method. Prachi Poddar
Note: long term we might split it in separate classes but good for now Romain Rigaux
Okay Prachi Poddar
Not needed anymore? Romain Rigaux
Nope Prachi Poddar
Could you rever this, it is break some refreshes https://github.com/cloudera/hue/commit/f30e9f75011f777458873fbc4c807bd2d100a4cb Romain Rigaux
  1. Nice!

  2. desktop/libs/indexer/src/indexer/api3.py (Diff revision 1)
     
     
     
     

    Could we avoid changing the API? (why do we need this)

  3. '/',1 --> '/', 1

  4. Could we just define it once?

    e.g. 3 lines above:
    driver_name = source['rdbmsHostname'].split(':', 2)[1]

  5. desktop/libs/indexer/src/indexer/templates/importer.mako (Diff revision 1)
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     

    Could we have only one

    createWizard.source.rdbmsType() == 'jdbc'

    and

    createWizard.source.rdbmsType() != 'jdbc'

    ?

  6. Note: long term we might split it in separate classes but good for now

  7. Could we move

    driver_name=None

    here?

    (so that it is cleaner)

  8. Not needed anymore?

  9. 
      
Review request changed

Description:

  +

Please ignore the changes in following files:

  + apps/security/src/security/api/hive.py
  + apps/security/src/security/api/sentry.py
  + desktop/libs/indexer/src/indexer/urls.py

  +
  +
  +
   

This also fixes the following JIRA:

    HUE-6914 [importer] Extending support of "DBMS ingestion made easy" to PostGreSQL

   
   

commit 884c1231509d90bb4d53276692f3d191dc963d23

    Author: Prachi Poddar <prachi.poddar@cloudera.com>
    Date: Mon Aug 14 09:56:27 2017 -0700

   
   
\[importer\] Extending support of "DBMS ingestion made easy" to Teradata
   
   

:100644 100644 0a48a6d086... 6a4b16d5cc... M desktop/libs/indexer/src/indexer/api3.py

    :100644 100644 d32f7ca6a0... d254fb197b... M desktop/libs/indexer/src/indexer/indexers/rdbms.py
    :100644 100644 4278b6c6e0... c3a9cc367b... M desktop/libs/indexer/src/indexer/templates/importer.mako
    :100644 100644 a774b86504... 816d01b413... M desktop/libs/notebook/src/notebook/connectors/base.py
    :100644 100644 5c63072b08... d11f33a95a... M desktop/libs/notebook/src/notebook/connectors/jdbc.py

   
   

The change in desktop/libs/notebook/src/notebook/connectors/jdbc.py file breaks the auto populate of databases/tables in the left pane in Teradata/PostgreSQL editor. Need to fix that. I'll create seperate JIRA once this gets pushed in.

Commit:

+69517f9aaeb8b34c280c682063f1da39a49d3332

Diff:

Revision 2 (+315 -73)

Show changes

  1. 
      
  2. desktop/libs/indexer/src/indexer/api3.py (Diff revision 1)
     
     
     
     

    We need this as the order of the column names in the list which we get from "columns" is different from the order in which "sample" data is.

  3. It is used just once in get_columns() method.

  4. 
      
  1. Rebase?

  2. Could you rever this, it is break some refreshes

    https://github.com/cloudera/hue/commit/f30e9f75011f777458873fbc4c807bd2d100a4cb

  3. 
      
Loading...