Websockets with Docker setup based on espocrm installer (letsencrypt) not working

[BigAl]

Hey,

i used the espocrm installer to setup a new instance with letsencrypt certificate as docker-compose deployment.
After logging in as admin, I see an error message in console of browser:

autobahn.js:341 WebSocket connection to 'wss://DOMAIN:8080/?authToken=79ca12c4a7aaf29d67a92a699be68bae&userId=1' failed:

after digging a bit deeper, I found following error message in nginx error.log

2022/11/02 14:17:31 [error] 10#10: *7 connect() failed (111: Connection refused) while connecting to upstream, client: xx.xx.xx.xx, server: espo.finstreet.it, request: "GET /wss?authToken=7b4f686cf23dea1da9ab8e6f5c189ecd&userId=1 HTTP/1.1", upstream: "http://172.20.0.6:8080/wss?authToken=7b4f686cf33dea1da9ab8e6f5c189ecd&userId=1", host: "DOMAIN"

Since everything was setup by installer I am quite confused, that it is not working and I dont have an idea what causes this behaviour.

Thanks for every hint in advance.

[lazovic]

Hi BigAl,

Could you show the part of your docker-compose.yml file that describes the details of the websocket container?

[BigAl]

Hey lazovic,

of course, its pretty that what the installer has generated after calling:

sudo bash install.sh --ssl --letsencrypt --domain=DOMAIN --email=MAIL

# MODE: letsencrypt

version: '3'

services:

  espocrm-mysql:
    image: mysql:8
    container_name: espocrm-mysql
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: XXX
      MYSQL_DATABASE: XXX
      MYSQL_USER: XXX
      MYSQL_PASSWORD: XXX
    volumes:
      - ./data/mysql/data:/var/lib/mysql
    networks:
      - internal

  espocrm-nginx:
    image: nginx
    container_name: espocrm-nginx
    restart: always
    depends_on:
      - espocrm
    volumes:
      - ./data/nginx/conf.d/:/etc/nginx/conf.d/
      - ./data/espocrm:/var/www/html
      - ./data/nginx/espocrm.conf:/etc/nginx/espocrm.conf
      - ./data/nginx/logs:/var/log/nginx
      - ./data/nginx/ssl:/etc/nginx/ssl
      - ./data/nginx/certbot:/var/www/certbot
    ports:
      - "80:80"
      - "443:443"
    command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
    networks:
      - external

  espocrm:
    image: espocrm/espocrm:fpm
    container_name: espocrm
    environment:
      ESPOCRM_DATABASE_HOST: espocrm-mysql
      ESPOCRM_DATABASE_USER: XXX
      ESPOCRM_DATABASE_PASSWORD: XXX
      ESPOCRM_ADMIN_USERNAME: XXX
      ESPOCRM_ADMIN_PASSWORD: XXX
      ESPOCRM_SITE_URL: "https://DOMAIN"
      ESPOCRM_CONFIG_WEB_SOCKET_ZERO_M_Q_SUBMISSION_DSN: "tcp://espocrm-websocket:7777"
    restart: always
    depends_on:
      - espocrm-mysql
    volumes:
     - ./data/espocrm:/var/www/html
    networks:
      - external
      - internal

  espocrm-daemon:
    image: espocrm/espocrm:fpm
    container_name: espocrm-daemon
    volumes:
      - ./data/espocrm:/var/www/html
    restart: always
    depends_on:
      - espocrm
    entrypoint: docker-daemon.sh
    networks:
      - external
      - internal

  espocrm-websocket:
    container_name: espocrm-websocket
    image: espocrm/espocrm:fpm
    environment:
      ESPOCRM_CONFIG_WEB_SOCKET_ZERO_M_Q_SUBSCRIBER_DSN: "tcp://*:7777"
      ESPOCRM_CONFIG_USE_WEB_SOCKET: "true"      
    volumes:
     - ./data/espocrm:/var/www/html
    restart: always
    depends_on:
      - espocrm
    entrypoint: docker-websocket.sh
    ports:
      - 8080:8080
    networks:
      - external
      - internal

  espocrm-certbot:
    image: certbot/certbot
    container_name: espocrm-certbot
    restart: always
    volumes:
      - ./data/nginx/ssl:/etc/letsencrypt
      - ./data/nginx/certbot:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
    networks:
      - external

networks:
  external:
    external: true
  internal:
    external: false​

[lazovic]

BigAl,

sudo bash install.sh --ssl --letsencrypt --domain=DOMAIN --email=MAIL

Did you replace the word DOMAIN in this command with your actual domain, where your instance should be located?
Similarly, you need to enter your valid email address instead of EMAIL.

[BigAl]

Hey lazovic

of course, I replaced those values with placeholders just like the secrets in the docker compose.
The Letsencrypt is working as expected with that domain, the given domain is set in the docker compose with same values.

Added:
And the whole app is working, except the websockets.

[lazovic]

BigAl,

Perhaps port 8080 (espocrm-websocket) on your server is occupied by some other process? Try to change it to 8081 or 8082, for example.
Before any changes remember to stop all containers first by running command sudo docker-compose down -v from the directory where the docker-compose.yml file is located.

[BigAl]

lazovic

The whole system is only servince EspoCRM via Docker:

0dff610aa080   espocrm/espocrm:fpm   "docker-websocket.sh"    20 hours ago   Up 20 hours   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 9000/tcp                        espocrm-websocket
​

Host port 8080 is directly bound to the websocket container. I used the docker-compose file liste above.
netstat is showing that the host is LISTENING on port 8080.

tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      off (0.00/0/0)

NMAP is showing, that the ports are open and reachable

Starting Nmap 7.93 ( https://nmap.org ) at 2022-11-04 09:08 CET
Nmap scan report for DOMAIN (IP)
Host is up (0.024s latency).
rDNS record for XXX

PORT     STATE SERVICE
80/tcp   open  http
443/tcp  open  https
8080/tcp open  http-proxy

Nmap done: 1 IP address (1 host up) scanned in 0.09 seconds​

So I really dont see that this port es occupied by any other service than the websocket container itself.

After digging a bit deeper in the websocket container, I found in the logs following message:

[2022-11-03 12:32:52] ERROR: Uncaught Exception PDOException: "SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution" at /var/www/html/application/Espo/ORM/PDO/DefaultPDOProvider.php line 97 {"exception":"[object] (PDOException(code: 2002): SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution at /var/www/html/application/Espo/ORM/PDO/DefaultPDOProvider.php:97)\n[previous exception] [object] (PDOException(code: 0): PDO::__construct(): php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution at /var/www/html/application/Espo/ORM/PDO/DefaultPDOProvider.php:97)"} []

If I had to guess, I would assume that the websocket container is having problems reaching the database via the specified hostname.
I have checked in the websocket container whether the name as specified in the docker-compose file (espocrm-mysql) is resolvable and reachable, this is the case.

But I also don't understand here why the websockets need their own port at all.

[BigAl]

Anyone? Doesn't anyone really have the same problem?

[lazovic]

Have you tried restarting all docker containers?

[BigAl]

Yes, several times.

[BigAl]

So I tried now a full new install with the install script, on a new server. Same thing. Websockets are not working.

Is there anyone out there running a docker based setup with working websockets? Can you please share your docker-compose configuration, please?

[victor]

BigAl,

Could not reproduce your error, there must be something wrong with your virtual machine.

- I created a virtual machine (clean without LAMP and other settings).
- Launched two commands from https://docs.espocrm.com/administrat...on-by-script/:

​wget https://github.com/espocrm/espocrm-installer/releases/latest/download/install.sh
sudo bash install.sh --adminUsername=victor --adminPassword=1 --yes​

- EspoCRM was installed with admin victor and password 1. I went to Administration -> Settings and checked that there was a check mark in the Use WebSocket field.
- I created a User and logged in under him in the anonymous tab of the browser.
- I opened two parallel windows in Contact and wrote to the Stream of this Contact from both Users. Posts in the Stream appeared without refreshing the page either in the main browser or in the anonymous tab. This indicates the successful operation of the Websocket.