-
-
desktop/core/src/desktop/migrations/0027_truncate_documents.py (Diff revision 1) In a comment, could you add reference to the Django bug and when we hit it?
(and add it to the jira description too)
HUE-7402 [core] Truncate SEARCH Column in Document2 for Oracle database
Review Request #11692 — Created Oct. 2, 2017 and submitted
Information | |
---|---|
yingc | |
hue | |
master | |
HUE-7402 | |
Reviewers | |
hue | |
enricoberti, jgauthier, johan, krish, romain, weixia |
commit 41746e058ab63eb10dbf64260be601dd301f91e2 Author: Ying Chen <yingchen@cloudera.com> Date: Mon Oct 2 14:25:29 2017 -0700 HUE-7402 [core] Truncate SEARCH Column in Document2 for Oracle database Searching saved documents in Oracle returns following error: DatabaseError: ORA-06502: PL/SQL: numeric or value error: character string buffer too small This is caused by DBMS_LOB.SUBSTR(%s, 4000) in Django framework django/db/backends/oracle/base.py Django has a ticket for this issue but unfixed: https://code.djangoproject.com/ticket/11580. Buffer size 4000 limit the length of field equals or less than 2000 characters. :000000 100644 0000000000... 593a415514... A desktop/core/src/desktop/migrations/0027_truncate_documents.py :100644 100644 aef7652dfe... 99e586238c... M desktop/core/src/desktop/models.py :100644 100644 dac1a9aa8c... 2dafdd9e2a... M desktop/core/src/desktop/settings.py
Description: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Diff: |
Revision 2 (+148 -1) |
-
-
desktop/libs/notebook/src/notebook/api.py (Diff revision 2) Shouldn't we do it for any DB to not slow down the search anyway?
Then maybe better in the Document2.save method?
-
-
desktop/libs/notebook/src/notebook/api.py (Diff revision 3) Didn't we said we would move this to document2 save method?
Description: |
|
---|
-
Just renaming the constant and adding a comment and looks good!
-
desktop/core/src/desktop/migrations/0027_truncate_documents.py (Diff revision 4) We only truncate for Oracle, but this is good enough (other fields will be truncated over time via the save method).
-
desktop/core/src/desktop/settings.py (Diff revision 4) Also add: for performance reasons and to avoid searching in huge fields, we also trunctae to a max length
-
desktop/core/src/desktop/settings.py (Diff revision 4) ORACLE_DOCUMENT2_SEARCH_LENGTH --> DOCUMENT2_SEARCH_MAX_LENGTH
?
Description: |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Diff: |
Revision 5 (+148) |
-
-
desktop/core/src/desktop/settings.py (Diff revision 5) Add a new line to split the sections? ############################################################ # Searching saved documents in Oracle returns following error: # DatabaseError: ORA-06502: PL/SQL: numeric or value error: character string buffer too small # This is caused by DBMS_LOB.SUBSTR(%s, 4000) in Django framework django/db/backends/oracle/base.py # Django has a ticket for this issue but unfixed: https://code.djangoproject.com/ticket/11580. # Buffer size 4000 limit the length of field equals or less than 2000 characters. # # For performance reasons and to avoid searching in huge fields, we also trunctae to a max length
-
Description: |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Diff: |
Revision 6 (+149) |