Solr Smart Indexer pre rebase

Review Request #7854 - Created June 24, 2016 and submitted

Aaron Peddle
hue
solr-smart-indexer
HUE-4244
hue
romain
commit a7b6d985c212047d01bdb94781a07e7f71842677
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Fri Jun 24 18:53:28 2016 -0700

    further pre rebase cleanup

:100644 100644 34a4d67... 415eed8... M	desktop/libs/indexer/src/indexer/api3.py
:100644 100644 4b64bdf... b3e430e... M	desktop/libs/indexer/src/indexer/smart_indexer.py

commit 4357617fec2c3641bef5b8b2a37f4923c710423a
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Fri Jun 24 18:36:00 2016 -0700

    pre rebase cleanup

:100644 000000 b2d7d76... 0000000... D	desktop/libs/indexer/src/data/oozie_workspace/grok-patterns.txt
:100644 100644 b8cf9a6... 984060c... M	desktop/libs/indexer/src/data/oozie_workspace/morphline_template.conf
:100644 100644 924eaf5... 745b037... M	desktop/libs/indexer/src/data/oozie_workspace/split_operation.conf
:100644 100644 cb69f4d... 34a4d67... M	desktop/libs/indexer/src/indexer/api3.py
:100644 100644 bec3b8a... 007728f... M	desktop/libs/indexer/src/indexer/conf.py
:100644 100644 56baa0b... 4b64bdf... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 e64511e... 0424882... M	desktop/libs/indexer/src/indexer/templates/indexer.mako
:100644 100644 e6dd913... 7d70f01... M	desktop/libs/indexer/src/indexer/views.py

commit 15e01c1d29e12c6cd6a742b2dc42e60250c92f7c
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Fri Jun 24 17:29:11 2016 -0700

    end to end grok

:000000 100644 0000000... b2d7d76... A	desktop/libs/indexer/src/data/oozie_workspace/grok-patterns.txt
:000000 100644 0000000... 86c9e8c... A	desktop/libs/indexer/src/data/oozie_workspace/grok_operation.conf
:100644 100644 e938a97... b8cf9a6... M	desktop/libs/indexer/src/data/oozie_workspace/morphline_template.conf
:100644 100644 b55ee38... 5d241a1... M	desktop/libs/indexer/src/data/oozie_workspace/workflow.xml
:100644 100644 fd82577... 56baa0b... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 8527876... e64511e... M	desktop/libs/indexer/src/indexer/templates/indexer.mako

commit 3151f90c7cdc23ad10ae0db4c41724ef45c963bb
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Thu Jun 23 12:44:30 2016 -0700

    ui grok option, collection name update and check on keypress, positive feedback on collection name available"

:100644 100644 898a211... 924eaf5... M	desktop/libs/indexer/src/data/oozie_workspace/split_operation.conf
:100644 100644 ed502af... fd82577... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 6d64719... 8527876... M	desktop/libs/indexer/src/indexer/templates/indexer.mako

commit 33f6e4c293c66cbd0528ba01838f298dcaffa963
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Wed Jun 22 18:03:05 2016 -0700

    indexer treats empty string data as a valid string

:100644 100644 e36e01b... b2087b8... M	desktop/libs/indexer/src/data/oozie_workspace/log4j.properties
:100644 100644 ebcdcb5... e938a97... M	desktop/libs/indexer/src/data/oozie_workspace/morphline_template.conf
:100644 100644 e9d3005... b55ee38... M	desktop/libs/indexer/src/data/oozie_workspace/workflow.xml
:100644 100644 0bace9d... ed502af... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 2e7200b... 6d64719... M	desktop/libs/indexer/src/indexer/templates/indexer.mako

commit f89ac626f8818e5222c91c976f67645707867b8d
Author: krish <krish@cloudera.com>
Date:   Mon Jun 20 12:14:22 2016 -0700

    HUE-4141 [oozie] Graph breaks for external workflows when there is more than one kill node

:100644 100644 6cd3954... 7495378... M	apps/oozie/src/oozie/models2.py

commit bb44cf81ff8779d96a89c839ba767e9bb611f9be
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Wed Jun 22 13:37:36 2016 -0700

    parameterized

:100644 100644 de15357... 6cd3954... M	apps/oozie/src/oozie/models2.py
:000000 100644 0000000... e36e01b... A	desktop/libs/indexer/src/data/oozie_workspace/log4j.properties
:100644 100644 a44a73f... ebcdcb5... M	desktop/libs/indexer/src/data/oozie_workspace/morphline_template.conf
:000000 100644 0000000... 898a211... A	desktop/libs/indexer/src/data/oozie_workspace/split_operation.conf
:100644 100644 67cc7ac... e9d3005... M	desktop/libs/indexer/src/data/oozie_workspace/workflow.xml
:100644 100644 c9394ab... cb69f4d... M	desktop/libs/indexer/src/indexer/api3.py
:100644 100644 6e7b14b... 0bace9d... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 fd183df... 2e7200b... M	desktop/libs/indexer/src/indexer/templates/indexer.mako
:100644 100644 6b65e7a... 4c7d232... M	desktop/libs/indexer/src/indexer/tests_indexer.py

commit 9fa3205224b2df740def9db65f318421231f65c1
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Mon Jun 20 13:02:51 2016 -0700

    indexer auto updates fields and preview

:100644 100644 0176247... c9394ab... M	desktop/libs/indexer/src/indexer/api3.py
:100644 100644 77103f6... 6e7b14b... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 c0c109f... fd183df... M	desktop/libs/indexer/src/indexer/templates/indexer.mako
:100644 100644 44c9752... 6b65e7a... M	desktop/libs/indexer/src/indexer/tests_indexer.py
:100644 100644 c24f613... b685ee1... M	desktop/libs/indexer/src/indexer/urls.py

commit aa336a8b9f6e96d4107f8024334bc791fe8415e9
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Fri Jun 17 16:27:49 2016 -0700

    basic ui end to end smart indexing

:100644 100644 2c0181c... 67cc7ac... M	desktop/libs/indexer/src/data/oozie_workspace/workflow.xml
:000000 100644 0000000... 0176247... A	desktop/libs/indexer/src/indexer/api3.py
:100644 100644 73ada26... 77103f6... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:000000 100644 0000000... c0c109f... A	desktop/libs/indexer/src/indexer/templates/indexer.mako
:100644 100644 358271b... 44c9752... M	desktop/libs/indexer/src/indexer/tests_indexer.py
:100644 100644 0ab50dd... c24f613... M	desktop/libs/indexer/src/indexer/urls.py
:100644 100644 c10bb54... e6dd913... M	desktop/libs/indexer/src/indexer/views.py

commit 82ca9da79eb0a4e766c9bb2857e1a43ca6c56591
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Thu Jun 16 14:26:04 2016 -0700

    code review smart indexer cleanup

:100644 100644 65b3ec0... 2c0181c... M	desktop/libs/indexer/src/data/oozie_workspace/workflow.xml
:100644 100644 079343e... 52d1497... M	desktop/libs/indexer/src/data/solrconfigs/solrcloud/conf/solrconfig.xml
:100644 100644 a08f96f... bec3b8a... M	desktop/libs/indexer/src/indexer/conf.py
:100644 100644 fb1cf02... 73ada26... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 480ac6c... 358271b... M	desktop/libs/indexer/src/indexer/tests_indexer.py
:100644 100644 06f413b... ddf7219... M	desktop/libs/libsolr/src/libsolr/api.py

commit 4a5bb397d2002cfe6aa0edefa2c9be234e3c5df3
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Wed Jun 15 14:59:40 2016 -0700

    smart indexer cleanup and refactor

:100644 100644 5250e41... a44a73f... M	desktop/libs/indexer/src/data/oozie_workspace/morphline_template.conf
:100644 100644 f0aa71e... fb1cf02... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 5ef8ed4... 480ac6c... M	desktop/libs/indexer/src/indexer/tests_indexer.py

commit e4e49e8eea45f3c51e1fa34822caa4df651b3a2a
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Wed Jun 15 12:43:27 2016 -0700

    smart_indexer refactor

:000000 100644 0000000... 5250e41... A	desktop/libs/indexer/src/data/oozie_workspace/morphline_template.conf
:000000 100644 0000000... 65b3ec0... A	desktop/libs/indexer/src/data/oozie_workspace/workflow.xml
:100644 100644 be4e3b8... a08f96f... M	desktop/libs/indexer/src/indexer/conf.py
:100644 100644 a56093a... f0aa71e... M	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 000000 b776392... 0000000... D	desktop/libs/indexer/src/indexer/templates/morphline_template.conf
:100644 000000 ffbb616... 0000000... D	desktop/libs/indexer/src/indexer/templates/schema.mako
:100644 100644 71e0daa... 5ef8ed4... M	desktop/libs/indexer/src/indexer/tests_indexer.py

commit b4c9ada2c0194d8beb798cd26b25a8fb364f160e
Author: Aaron Peddle <aaron.peddle@cloudera.com>
Date:   Wed Jun 15 13:22:55 2016 -0700

    solr smart indexer end to end

:100644 100644 4572726... 079343e... M	desktop/libs/indexer/src/data/solrconfigs/solrcloud/conf/solrconfig.xml
:100644 000000 114eb60... 0000000... D	desktop/libs/indexer/src/indexer/indexer.py
:100644 000000 b6739d3... 0000000... D	desktop/libs/indexer/src/indexer/simple.csv
:000000 100644 0000000... a56093a... A	desktop/libs/indexer/src/indexer/smart_indexer.py
:100644 100644 e46490d... b776392... M	desktop/libs/indexer/src/indexer/templates/morphline_template.conf
:000000 100644 0000000... 71e0daa... A	desktop/libs/indexer/src/indexer/tests_indexer.py
:100644 100644 ddf7219... 06f413b... M	desktop/libs/libsolr/src/libsolr/api.py


  • 13
  • 0
  • 9
  • 0
  • 22
Description From Last Updated
aggregate the lines together? (no new lines everywhere) Romain Rigaux
same Romain Rigaux
same Romain Rigaux
test --> request.user.username Romain Rigaux
remove? Romain Rigaux
delete comments? Romain Rigaux
if counts: Romain Rigaux
add a log.warn() message? Romain Rigaux
remove? Romain Rigaux
same? Romain Rigaux
from nose.plugins.skip import SkipTest from hadoop.pseudo_hdfs4 import is_live_cluster def setup(): if is_live_cluster(): raise SkipTest() To make the tests skipped on ... Romain Rigaux
re-index message v1+ Romain Rigaux
from indexer.smart_indexer import ... Romain Rigaux
  1. Nice!

  2. apps/oozie/src/oozie/models2.py (Diff revision 2)
     
     

    from django.contrib.auth.models import User

    above

    and if type(user) is User: ?

  3. desktop/libs/indexer/src/indexer/api3.py (Diff revision 2)
     
     
     
     
     
     
     
     

    aggregate the lines together? (no new lines everywhere)

  4. desktop/libs/indexer/src/indexer/api3.py (Diff revision 2)
     
     
     
     
     
     
  5. desktop/libs/indexer/src/indexer/api3.py (Diff revision 2)
     
     
     
     
     
     
     
     
     
     
  6. test --> request.user.username

  7. desktop/libs/indexer/src/indexer/smart_indexer.py (Diff revision 2)
     
     
     
     
     
     

    delete comments?

  8. add a log.warn() message?

  9. from nose.plugins.skip import SkipTest

    from hadoop.pseudo_hdfs4 import is_live_cluster

    def setup():
    if is_live_cluster():
    raise SkipTest()

    To make the tests skipped on the current test machine (would brake the current build as there is no Solr server yet in the infra). Then, need to

    export LIVE_CLUSTER=true

    in your shell

  10. 
      
Review request changed

Status: Closed (submitted)

Loading...