HUE-8738 [sqlalchemy] Adding a pre-ping option when picking up connection

Review Request #15479 — Created Sept. 30, 2020 and submitted

romain
hue
master
HUE-8738
hue
Amlesh1902, ayush.goyal, johan, ranade, romain, Sreenath, yingc
commit 31308acf103ac4b69602f034f1b04bea9b27d378
Author: Romain <romain.rigaux@gmail.com>
Date:   Mon Aug 24 10:47:35 2020 +0200

    HUE-8738 [sqlalchemy] Adding a pre-ping option when picking up connection
    
    https://docs.sqlalchemy.org/en/13/core/pooling.html#disconnect-handling-pessimistic
    
    To avoid:
    
    Traceback (most recent call last):
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/decorators.py", line 119, in wrapper
        return f(*args, **kwargs)
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/decorators.py", line 235, in wrapper
        raise e
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/decorators.py", line 178, in wrapper
        return f(*args, **kwargs)
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/api.py", line 231, in execute
        response = _execute_notebook(request, notebook, snippet)
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/api.py", line 154, in _execute_notebook
        response['handle'] = interpreter.execute(notebook, snippet)
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 106, in decorator
        raise e
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 100, in decorator
        return func(*args, **kwargs)
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/decorators.py", line 235, in wrapper
        raise e
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/decorators.py", line 178, in wrapper
        return f(*args, **kwargs)
      File "/home/romain/projects/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 224, in execute
        connection = engine.connect()
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2206, in connect
        return self._connection_cls(self, **kwargs)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 103, in __init__
        else engine.raw_connection()
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2305, in raw_connection
        return self._wrap_pool_connect(
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2278, in _wrap_pool_connect
        Connection._handle_dbapi_exception_noconnection(
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1544, in _handle_dbapi_exception_noconnection
        util.raise_from_cause(sqlalchemy_exception, exc_info)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
        reraise(type(exception), exception, tb=exc_tb, cause=cause)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
        raise value.with_traceback(tb)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2275, in _wrap_pool_connect
        return fn()
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection
        return _ConnectionFairy._checkout(self)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 760, in _checkout
        fairy = _ConnectionRecord.checkout(pool)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
        rec = pool._do_get()
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 139, in _do_get
        self._dec_overflow()
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
        compat.reraise(exc_type, exc_value, exc_tb)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
        raise value
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 136, in _do_get
        return self._create_connection()
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
        return _ConnectionRecord(self)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
        self.__connect(first_connect_check=True)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 639, in __connect
        connection = pool._invoke_creator(self)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
        return dialect.connect(*cargs, **cparams)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 453, in connect
        return self.dbapi.connect(*cargs, **cparams)
      File "/home/romain/projects/hue/build/env/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
        conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
    sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
            This probably means the server terminated abnormally
            before or while processing the request.

:100644 100644 7576e9e962 a2a02760df M	desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py

manual

johan
  1. Ship It!
  2. 
      
romain
Review request changed

Status: Closed (submitted)

Loading...