Peer resets connection to Docker Swarm database

To address the issue, on a Linux machine, execute the provided command. Additionally, the properties for the Tomcat connection pool have been specified. The solution to the problem can be found in a blog post titled “Dealing with NodeNotAvailable Exceptions in Elasticsearch.” Alternatively, to prevent idle connections from closing, another solution is available. It should be noted that if the previous solution was effective for you, the issue you are experiencing is not related to the Docker connection.


Solution 1:

Encountered an error while deploying a Docker Swarm stack of Spring Boot app and PostgreSQL, which persisted for a week. Finally, after much struggle, it was discovered that the issue was caused by the firewall dropping connections between containers due to prolonged inactivity. The solution was to execute a command on the Linux machine.

sudo sysctl -w 
net.ipv4.tcp_keepalive_time=600 
net.ipv4.tcp_keepalive_intvl=60 
net.ipv4.tcp_keepalive_probes=3

Additionally, the subsequent properties related to the connection pool of Tomcat have been incorporated.

tomcat:
  max-active: 10
  initial-size: 5
  max-idle: 8
  min-idle: 5
  test-on-borrow: true
  test-while-idle: true
  test-on-return: false
  test-on-connect: true
  validation-query: SELECT 1
  validation-interval: 30000
  max-wait: 30000
  min-evictable-idle-time-millis: 60000
  time-between-eviction-runs-millis: 5000
  remove-abandoned: true
  remove-abandoned-timeout: 60

A blogpost titled “Dealing with NodeNotAvailable Exceptions in Elasticsearch” provided the solution.


Solution 2:


To avoid closing idle connections, an alternative method can be used. The issue is connected to the default swarm service discovery, which terminates idle connections after 15 minutes. By explicitly specifying the endpoint mode, such as shown in

dnsrr

example, the issue can be resolved.

version: '3.3'
services:
  foo-service:
    image: example/foo-service:latest
    hostname: foo-service
    networks:
      - foo_network
    deploy:
      endpoint_mode: dnsrr
      # ...
networks:
  foo_network:
    external: true
    driver: overlay

Frequently Asked Questions