Docker upgrades in production environments

[bandtank]

What is the process for updating EspoCRM and extensions in a Docker container that has multiple environments and instances? Upgrading the development environment is easy, but what about production? Once the files have been upgraded in development, the upgrade scripts won’t run again. The same problem applies to extensions.

I don’t see a way to fix this problem without breaking a lot of “rules” regarding version controlled container images. Is there a process for upgrading, testing, building, and releasing updates from a development environment into a production environment? Specifically, how do database upgrades work when a new container version connects to an existing database?

Storing configuration data in docker is also a problem because sensitive information, such as passwords for APIs, must now be added through Docker compose.

Overall, the idea of using Docker for Espo sounds great, but I don’t understand how it works in practice because the upgrade process and config-internal parameters present a lot of problems. Any insight would be greatly appreciated.

[tarasm]

It seems like you are not using the official docker image, https://hub.docker.com/r/espocrm/espocrm.
In this official image, you do not need to worry about upgrading or scripts run. To upgrade the container you just need to run the command

docker compose pull && docker compose up -d​

In this case, the container will be upgraded with all needed scripts will be run.

Also, you can define your config parameters in the docker-compose.yml:

ESPOCRM_CONFIG_WEB_SOCKET_URL: "wss://my-espocrm.com:8080"​

But, please note to hide the option, you have to add it to the "systemItems" list in the "data/config-internal.php".

If we talk about your case, it is not trivial.

[bandtank]

It is not possible to use an immutable container with a production instance of EspoCRM if you use custom code. I need to develop in a development environment, add the code to a running EspoCRM instance, and commit everything to version control. Using the official docker image does not allow any of this to happen and it makes EspoCRM useless to me. I have 4,000+ hours of development into custom files, only some of which are packaged into extensions. Putting every custom file in an extension is limiting and not a good use of time.

I am not looking to change EspoCRM to fit my use case. The question is about how to manage the upgrade process when EspoCRM is treated like a vendor-provided library in my application. It touches the database once, which means there is a chicken and egg problem in terms of how to upgrade the database at the same time as multiple front-end instances of the webserver. There is also the problem of layout and label customization, which means changes to production instances cause files to be modified. The layout and label information needs to be in the database, which is something my custom code manages. In other words, the json files are generated by a job, which reads the information from a database table.

A bigger issue with the official docker container for EspoCRM is the lack of multiple instantiation of the front-end container. I also do not want a container running my production database because it is a terrible idea. The official docker container is nice to have as a reference, but I have more traffic than one front-end container can manage, so I instantiate 5 to 10 containers to handle web traffic and four containers for daemons. Some of my jobs process hundreds of GBs of data every hour. The scale of my application is simply far beyond what the official container can handle.

[tarasm]

The best way to apply your customization is creating extensions. There is a template for building extensions, https://github.com/espocrm/ext-template. You can simply create an instance or pack files.
EspoCRM is an open-source application. So, if you have any ideas, please implement them and push to the official repository.

[rinorway]

From the documentation https://docs.espocrm.com/administrat...se-environment
it is not clear if upgrading docker would also upgrade the Espocrm version.



Upgrading Docker:

docker compose pull && docker compose up -d

all fine, but I still had to upgrade Espo itself


Upgrading Espo:

docker exec -it espocrm bash

then

php command.php upgrade
Current version is 7.4.6.
EspoCRM will be upgraded to version 7.5.6 now. Enter [Y] to continue.

tarasm bandtank Can you confirm ? Or was I just too early? If not, we should add it to the documentation.

[lazovic]

Hi rinorway,

Please tell me when did you try to update EspoCRM in this way? Now the EspoCRM repository on Dockerhub contains the latest version 8.0.0 (updated 4 days ago): https://hub.docker.com/r/espocrm/espocrm/tags

[rinorway]

Yesterday. I found it strange indeed. Maybe some caching issue.

But maybe it is good to mention it in the documentation that upgrading docker WILL upgrade Espo as well.

Actually it would be nice to have some feedback about versions after upgrading when using

command.sh upgrade

[lazovic]

The corresponding section in the documentation is called “Upgrading EspoCRM in Docker Compose environment”: https://docs.espocrm.com/administrat...se-environment

The upgrade by “command.sh upgrade“ command also worked correctly for me, but it's worth noting that I initially installed EspoCRM by script: https://docs.espocrm.com/administrat...tion-by-script

[rinorway]

Related : For those searching how to update the cert when using docker :

https://docs.espocrm.com/administration/installation-by-script/#1-lets-encrypt-free-certificate

Certificate generation¶

Generate a new Let's Encrypt certificate. It can be used to create a certificate for the first time or after a domain change.

sudo /var/www/espocrm/command.sh cert-generate

Certificate renewal¶

Renew an existing Let's Encrypt certificate. It can be used in crontab to automatically renewal.

sudo /var/www/espocrm/command.sh cert-renew

[bandtank]

Do any of you run Espo in a production container through AWS ECR? I keep versions of the production container in ECR because I want to test each version before I release it to production. I don't know how to upgrade Espo versions in an offline container, however, because the beforeInstall and afterInstall scripts need to run on the production database. Therefore, I am forced to upgrade Espo in the production container, which is not how it should be done.

It seems like the only way to make the upgrade work is to do the following:

1. Reduce the production container count to 1
2. Put the production container in maintenance mode
3. Upgrade the production container by using the command line (php command.php upgrade)
4. Build an offline container using the same steps as above to push to ECR. Tag the new version
5. Replace the production container with the new version in ECR

Is there a better way? I am using AWS Fargate to host development and production containers. In the containers, many extensions are installed, which is part of the complication with the overall process. When containers die or go down, they need to be brought back with the correct extensions and settings, which is why I use ECR to create versions of production containers. Every version has extensions installed and configured already.