Skip to content

GitLab

Closed
Created by Ricardo Grim Cabrita@grimMaintainer

Unhandled database connection timeout causes application exit

Not a critical issue, since the cluster restarts the application, but something to keep in mind.

Database server closes connection due to inactivity.

$ kc logs -f remoteqc-db-6f7d78cff-sz84p

2021-09-08 13:52:38 139 [Warning] Aborted connection 139 to db: 'nodemysql' user: 'remoteqc' host: '10.244.3.235' (Got timeout reading communication packets)
2021-09-08 14:02:42 140 [Warning] Aborted connection 140 to db: 'nodemysql' user: 'remoteqc' host: '10.244.3.235' (Got timeout reading communication packets)
2021-09-08 14:12:46 141 [Warning] Aborted connection 141 to db: 'nodemysql' user: 'remoteqc' host: '10.244.3.235' (Got timeout reading communication packets)
2021-09-08 14:22:50 142 [Warning] Aborted connection 142 to db: 'nodemysql' user: 'remoteqc' host: '10.244.3.235' (Got timeout reading communication packets)
2021-09-08 14:32:55 143 [Warning] Aborted connection 143 to db: 'nodemysql' user: 'remoteqc' host: '10.244.3.235' (Got timeout reading communication packets)
2021-09-08 14:42:59 144 [Warning] Aborted connection 144 to db: 'nodemysql' user: 'remoteqc' host: '10.244.3.235' (Got timeout reading communication packets)

Application doesn't know how to handle it so it exits.

$ kc logs -f remoteqc-567cd68bd9-kffcn

> zdmp@1.0.0 start
> node app.js

Server running at port: 4000.
Database connected.
node:events:371
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:220:20)
Emitted 'error' event on Connection instance at:
    at Connection._handleProtocolError (/app/node_modules/mysql/lib/Connection.js:423:8)
    at Protocol.emit (node:events:394:28)
    at Protocol._delegateError (/app/node_modules/mysql/lib/protocol/Protocol.js:398:10)
    at Protocol.handleNetworkError (/app/node_modules/mysql/lib/protocol/Protocol.js:371:10)
    at Connection._handleNetworkError (/app/node_modules/mysql/lib/Connection.js:418:18)
    at Socket.emit (node:events:394:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -104,
  code: 'ECONNRESET',
  syscall: 'read',
  fatal: true
}

Consequently we are getting around 136 restarts a day.

NAME                             READY   STATUS    RESTARTS   AGE
remoteqc-567cd68bd9-kffcn        1/1     Running   136        24h
remoteqc-db-6f7d78cff-sz84p      1/1     Running   0          25h
remoteqc-http-857b4fdd5f-f48fq   1/1     Running   0          16h
Edited by Ricardo Grim Cabrita