Fish Trophy

Romain Rigaux got a fish trophy!

HUE-3797 [scheduler] Refactor the scheduling API to be more abstracted

Review Request #14041 - Created June 24, 2019 and submitted

Romain Rigaux
hue
master
HUE-3797
hue
jgauthier, johan, ranade, romain, weixia, yingc
commit 8787dd6d0caad60fa34ff207a96ebee95010d70a
Author: Romain <romain.rigaux@gmail.com>
Date:   Mon Jun 24 10:28:54 2019 -0700

    HUE-3797 [oozie] Update to use common scheduler submission template

:100644 100644 b131e0305e d03013e50d M	apps/oozie/src/oozie/views/dashboard.py
:100644 100644 77175a4dce b80ff6748d M	apps/oozie/src/oozie/views/editor2.py

commit aaa03af862946409ade5137799001c1d535fec89
Author: Romain <romain.rigaux@gmail.com>
Date:   Mon Jun 24 09:33:59 2019 -0700

    HUE-3797 [scheduler] Moving beat schedules to a submodule config

:100644 100644 cfd5b1c823 2ac0641545 M	desktop/conf.dist/hue.ini
:100644 100644 75add7bd84 aafc7baa19 M	desktop/core/src/desktop/celery.py
:100644 100644 270b54c6ab dee86dfdc9 M	desktop/core/src/desktop/conf.py

commit 6ed7a4ad9022fabc6656883e2d943290905625fb
Author: Romain <romain.rigaux@gmail.com>
Date:   Wed May 15 14:49:57 2019 -0700

    HUE-3797 [scheduler] Do not error if oozie is not present

:100644 100644 c3d17086c8 bafda9329e M	desktop/core/src/desktop/lib/scheduler/api.py
:100644 100644 f3b7287e1d d2b247d14f M	desktop/libs/notebook/src/notebook/connectors/oozie_batch.py

commit c6b4288cb244f069864233162a5bd3588af9ffa3
Author: Romain <romain.rigaux@gmail.com>
Date:   Tue Apr 30 17:38:12 2019 -0700

    HUE-3797 [scheduler] Refactored and moved Oozie submit coordinator to API

:100644 100644 d7fe928dd6 3e4ac7246b M	apps/oozie/src/oozie/forms.py
:100644 100644 135c95eb1e 77175a4dce M	apps/oozie/src/oozie/views/editor2.py
:100644 100644 83d2b7d768 c3d17086c8 M	desktop/core/src/desktop/lib/scheduler/api.py
:100644 100644 30a7e7dec6 1af6e13ad9 R080	desktop/core/src/desktop/lib/scheduler/lib/base.py	desktop/core/src/desktop/lib/scheduler/lib/api.py
:100644 100644 f84279f467 e471e72a18 M	desktop/core/src/desktop/lib/scheduler/lib/beat.py
:100644 100644 548803520c 522888ecd3 M	desktop/core/src/desktop/lib/scheduler/lib/oozie.py
:100644 100644 eebb99dc57 90bb4db86c R087	apps/oozie/src/oozie/templates/editor2/submit_job_popup.mako	desktop/core/src/desktop/templates/scheduler/submit_job_popup.mako
:100644 100644 7e8dbe1ab0 2e1f47e203 M	desktop/core/src/desktop/urls.py

commit 7aa0df52ccb136a6bf6fafba24edb450475d075b
Author: Romain <romain.rigaux@gmail.com>
Date:   Tue Apr 30 13:00:32 2019 -0700

    HUE-3797 [scheduler] Get schedule document via generic interface

:100644 100644 210175e1ef a828d32ed8 M	desktop/core/src/desktop/js/apps/notebook/notebook.js
:100644 100644 3dd0d606c4 4b5d2e9c77 M	desktop/core/src/desktop/js/apps/notebook2/notebook.js
:100644 100644 e1677e6f12 83d2b7d768 M	desktop/core/src/desktop/lib/scheduler/api.py
:100644 100644 e08b102f09 97db898973 M	desktop/core/src/desktop/lib/scheduler/urls.py
:100644 100644 d8fb97afd1 7e8dbe1ab0 M	desktop/core/src/desktop/urls.py

commit 3ff0918ce453f2a162e4a195086019ba498df3af
Author: Romain <romain.rigaux@gmail.com>
Date:   Mon Apr 29 14:22:08 2019 -0700

    HUE-3797 [scheduler] Skeleton of a schedulable SQL task

:100644 100644 3551760dbf b272346cfc M	apps/useradmin/src/useradmin/management/commands/useradmin_sync_with_unix.py
:100644 100644 bbe0b3b0cd 75add7bd84 M	desktop/core/src/desktop/celery.py
:100644 100644 f690cb07f1 30a7e7dec6 M	desktop/core/src/desktop/lib/scheduler/lib/base.py
:100644 100644 d707cba4ae e343d200b1 M	desktop/core/src/desktop/lib/scheduler/tasks.py
:100644 100644 258e69adfe 930a73143e M	desktop/libs/notebook/src/notebook/connectors/base.py
:100644 100644 35b29fcfbc cee0ddaeb9 M	desktop/libs/notebook/src/notebook/models.py
:100644 100644 17465f68ed 7f281bc609 M	desktop/libs/notebook/src/notebook/tasks.py
:100644 100644 59e2a9005d daa0ffb9a1 M	desktop/libs/notebook/src/notebook/views.py

commit d4b19aee98f1c636f1b2fa9024fc6c9f8e7f33ee
Author: Romain <romain.rigaux@gmail.com>
Date:   Mon Apr 29 11:09:37 2019 -0700

    HUE-3797 [task] Add a generic execute Django command task

:100644 100644 7db5067387 bbe0b3b0cd M	desktop/core/src/desktop/celery.py
:100644 100644 f3269de7fb f84279f467 M	desktop/core/src/desktop/lib/scheduler/lib/beat.py
:000000 100644 0000000000 d707cba4ae A	desktop/core/src/desktop/lib/scheduler/tasks.py
:100644 100644 f7e496a520 612c80526a M	desktop/core/src/desktop/management/commands/desktop_document_cleanup.py
:100644 100644 1e21b6b221 258e69adfe M	desktop/libs/notebook/src/notebook/connectors/base.py
:100644 100644 b45739a3d1 17465f68ed M	desktop/libs/notebook/src/notebook/tasks.py

commit a8deaff7342fa68d2aca9927c97a567ab49bf01f
Author: Romain <romain.rigaux@gmail.com>
Date:   Sat Apr 27 12:56:21 2019 -0700

    HUE-3797 [scheduler] Refactor the scheduling API to be more abstracted

:100644 100644 7d353ace2b 7db5067387 M	desktop/core/src/desktop/celery.py
:000000 100644 0000000000 1854b36356 A	desktop/core/src/desktop/lib/scheduler/__init__.py
:000000 100644 0000000000 e1677e6f12 A	desktop/core/src/desktop/lib/scheduler/api.py
:000000 100644 0000000000 1854b36356 A	desktop/core/src/desktop/lib/scheduler/lib/__init__.py
:000000 100644 0000000000 f690cb07f1 A	desktop/core/src/desktop/lib/scheduler/lib/base.py
:000000 100644 0000000000 f3269de7fb A	desktop/core/src/desktop/lib/scheduler/lib/beat.py
:000000 100644 0000000000 548803520c A	desktop/core/src/desktop/lib/scheduler/lib/oozie.py
:000000 100644 0000000000 d053ec41e3 A	desktop/core/src/desktop/lib/scheduler/models.py
:000000 100644 0000000000 e08b102f09 A	desktop/core/src/desktop/lib/scheduler/urls.py
:100644 100644 3bea416564 1e21b6b221 M	desktop/libs/notebook/src/notebook/connectors/base.py
:100644 100644 0a4fb12a33 b45739a3d1 M	desktop/libs/notebook/src/notebook/tasks.py

manual + test updated/added
Not all the API is fully abstracted yet, it will be gradual as it is large change. V1 is to get the proper right panel in the editor.

This change introduces:
- beginning of generification of the scheduler API (by supporting Oozie, Beat...)
- move submission API only to the common API (edit workflow etc... do not change now)

  • 1
  • 0
  • 0
  • 0
  • 1
Description From Last Updated
Nit: I assume this is moved from somewhere but to keep in mind we should stop using mako python logic... Johan Åhlén
  1. Ship It!
  2. Nit: I assume this is moved from somewhere but to keep in mind we should stop using mako python logic...

  3. 
      
  1. Ship It!
  2. 
      
Review request changed

Status: Closed (submitted)

Loading...