HUE-8775 [hive] Add back interface to talk to Hive Metastore

Review Request #13768 - Created March 29, 2019 and submitted

Romain Rigaux
hue
master
HUE-8775
hue
jgauthier, johan, ranade, romain, weixia, yingc
commit 2243bd48c9ff55575dcc3d7b9c08fb0200496320
Author: Romain Rigaux <romain@cloudera.com>
Date:   Fri Mar 29 09:48:35 2019 -0700

    HUE-8775 [notebook] Link HMS connector to autocomplete view

:100644 100644 471bde8... b5cb125... M	desktop/libs/notebook/src/notebook/connectors/hive_metastore.py

commit f6bcaa1cef1f9fc9e1b711867e54573cc3015846
Author: Romain Rigaux <romain@cloudera.com>
Date:   Fri Mar 29 09:33:48 2019 -0700

    HUE-8775 [hive] Regenerate Thrift with older Hive 2 than upstream master

:100755 100755 b6c3069... 27aadd5... M	apps/beeswax/gen-py/hive_metastore/ThriftHiveMetastore-remote
:100644 100644 3f3d6ed... bf28c7a... M	apps/beeswax/gen-py/hive_metastore/ThriftHiveMetastore.py
:100644 100644 743b8d4... 7c232a7... M	apps/beeswax/gen-py/hive_metastore/constants.py
:100644 100644 535940f... 1cf881a... M	apps/beeswax/gen-py/hive_metastore/ttypes.py
:100644 100644 afeafe7... aea1617... M	apps/beeswax/src/beeswax/server/hive_metastore_server.py
:100644 100644 5430d03... 22dee4e... M	apps/beeswax/thrift/hive_metastore.thrift
:100644 100644 f32edb6... 471bde8... M	desktop/libs/notebook/src/notebook/connectors/hive_metastore.py

commit 11ed829dc44831052c3e7f4e42f2fc6e0510c2cd
Author: Romain Rigaux <romain@cloudera.com>
Date:   Fri Mar 29 09:20:53 2019 -0700

    HUE-8775 [notebook] Add HMS connector implementation

:100644 100644 5dc5dcf... f13e53d... M	apps/beeswax/src/beeswax/server/dbms.py
:000000 100644 0000000... f32edb6... A	desktop/libs/notebook/src/notebook/connectors/hive_metastore.py

commit 7cef934c519bd0283fa698f5faa80cf0f457c9f9
Author: Romain Rigaux <romain@cloudera.com>
Date:   Fri Mar 29 08:39:52 2019 -0700

    HUE-8775 [hive] Add back interface to talk to Hive Metastore

:100755 100755 7a10a98... e2c6ed4... M	apps/beeswax/gen-py/TCLIService/TCLIService-remote
:100644 100644 3bf8cee... e828b25... M	apps/beeswax/gen-py/TCLIService/TCLIService.py
:100644 100644 432e579... 6fbee27... M	apps/beeswax/gen-py/TCLIService/constants.py
:100644 100644 ef3f0a9... 0143570... M	apps/beeswax/gen-py/TCLIService/ttypes.py
:000000 100755 0000000... 244cc3d... A	apps/beeswax/gen-py/fb303/FacebookService-remote
:000000 100644 0000000... ecd6ea6... A	apps/beeswax/gen-py/fb303/FacebookService.py
:000000 100644 0000000... 14c89e0... A	apps/beeswax/gen-py/fb303/__init__.py
:000000 100644 0000000... 81960a7... A	apps/beeswax/gen-py/fb303/constants.py
:000000 100644 0000000... 97c2fec... A	apps/beeswax/gen-py/fb303/ttypes.py
:000000 100755 0000000... b6c3069... A	apps/beeswax/gen-py/hive_metastore/ThriftHiveMetastore-remote
:000000 100644 0000000... 3f3d6ed... A	apps/beeswax/gen-py/hive_metastore/ThriftHiveMetastore.py
:000000 100644 0000000... e9ee039... A	apps/beeswax/gen-py/hive_metastore/__init__.py
:000000 100644 0000000... 743b8d4... A	apps/beeswax/gen-py/hive_metastore/constants.py
:000000 100644 0000000... 535940f... A	apps/beeswax/gen-py/hive_metastore/ttypes.py
:100755 100755 e588d9d... 9418c80... M	apps/beeswax/regenerate_thrift.sh
:100644 100644 3db78c0... 77b1c9e... M	apps/beeswax/src/beeswax/conf.py
:000000 100644 0000000... afeafe7... A	apps/beeswax/src/beeswax/server/hive_metastore_server.py
:000000 100644 0000000... 89bd6eb... A	apps/beeswax/thrift/fb303.thrift
:000000 100644 0000000... 5430d03... A	apps/beeswax/thrift/hive_metastore.thrift
:100644 100644 fccc99f... de088a6... M	desktop/conf.dist/hue.ini
:100644 100644 42278ef... b58141e... M	desktop/conf/pseudo-distributed.ini.tmpl

Regen also takes care of Python 3 support

./build/env/bin/hue shell

In [1]: from notebook.connectors.hive_metastore import HiveMetastoreApi
...: from django.contrib.auth.models import User
...: a = User.objects.get(username='romain')
...: HiveMetastoreApi(a).autocomplete({})
...:
[29/Mar/2019 09:47:51 +0000] thrift_util DEBUG Thrift call: <class 'hive_metastore.ThriftHiveMetastore.Client'>.get_all_databases(args=(), kwargs={})
[29/Mar/2019 09:47:51 +0000] thrift_util DEBUG Thrift call: <class 'hive_metastore.ThriftHiveMetastore.Client'>.get_all_databases(args=(), kwargs={}) returned in 80ms: ['default']
Out[1]: {'databases': ['default']}

  • 1
  • 0
  • 0
  • 0
  • 1
Description From Last Updated
Can we move this in a different category? With the intention of potentially disabling beeswax? Jean-Francois Desjeans Gauthier
  1. 
      
  2. Can we move this in a different category?
    With the intention of potentially disabling beeswax?

    1. (e.g. having beeswax disabled, but hms enabled)

    2. Good point, there is the issue of impala only too. Let me come back with a new design (i.e. add empy 'hive' apps, etc and ideally move beeswax to a lib but while avoiding too much change). Actually, apps and libs are all 'libs', with core Hue being the overall frameworks for those connectors

  3. 
      
Review request changed

Status: Closed (submitted)

Loading...