We should update the UI at some point to not show it or do all the logic in the backgrond but already a nice improvement with this.
How about moving all the new logic to desktop.models in a utility?
that way we keep it clean here, and it will be easier to unit test and evolve it with the connector revamps. And later we could check for SparkSql etc in the utility function without leaking.
Move diretly to _get_beeswax_tables() ?
(more isolated this way and same we can clean-up more easily with the connector refactoring)
If more than 10 tables, how about just failing with a message saying to do the full invalidate as 'too many tables (N) to invalidate...'?
HUE-8882 [impala] Fix invalidate delta when hive is missing.
Review Request #14033 - Created June 20, 2019 and submitted
|Jean-Francois Desjeans Gauthier|
commit c55cc84b407b9585cc7a23e09e858a7838e367aa Author: Jean-Francois Desjeans Gauthier <firstname.lastname@example.org> Date: Thu Jun 20 14:25:08 2019 -0700 HUE-8882 [impala] Fix invalidate delta when hive is missing. :100644 100644 be90bc711e... 9a6a8b321a... M apps/impala/src/impala/dbms.py
Let me clean-up the API so that we don't need to do this
How about failing and saying: please do a full refresh instead?
(better than doing something else than ask in the UI, which might also fail if not full INVALIDATE perms anyway too)
(and at some point we update the UI to adequately look)
How about renaming
has_hive_metastore --> get_hive_metastore_interepreters():
then just pick yp the first from there.
This might fail as not builtin in py2 AFAIK
This should break if not mocked?
Also testing only full db invalidate
Combine in one line with
[ .... if interpreter == 'hive' or interpreter == 'hms', interpreters)]
in the first list comprehension?
Then returning the list
Maybe a few ideas to simplify a bit. Also using assert_called_once() in the Mock() might be great here (as we could prove there is no more 10s of call)?
%s --> %d
How about just:
interpreters = Cluster(self.client.user).get_app_config().get_hive_metastore_interpreters()]
and we manage the case interpreter == 'hive' directy inside beeswax_query_server_config()?
if name == 'hive':
name == 'beeswax'
And we can see later for the ordering of the hive connectors.
assert_equal the hql query or make sure it was called once?
return [interpreter['type'] for interpreter in get_ordered_interpreters(self.user) if interpreter == 'hive' or interpreter == 'hms']