Tweet
Hello EspoCRM Team,
I am working with a large dataset import in EspoCRM. For each record, a Workflow HTTP Request is triggered to an external API endpoint.
The problem is that the endpoint server cannot handle too many API calls at once. When EspoCRM sends requests for all records simultaneously, the endpoint starts failing or slowing down. Because of this, I would like to limit the number of requests sent from EspoCRM, for example:
Is there any built-in configuration in EspoCRM (Workflow, BPM, Scheduled Jobs, or Queue system) that allows:
Additional Issue
I am also facing another problem during this process.
The endpoint server actually returns the response successfully, but after a certain number of records EspoCRM stops saving the data. When this happens, I have to restart my Linode server, and after restarting it starts working again.
This behavior suggests that something may be getting blocked or overloaded on the server side (possibly cURL, PHP workers, or queue processing).
Could you please advise:
I am working with a large dataset import in EspoCRM. For each record, a Workflow HTTP Request is triggered to an external API endpoint.
The problem is that the endpoint server cannot handle too many API calls at once. When EspoCRM sends requests for all records simultaneously, the endpoint starts failing or slowing down. Because of this, I would like to limit the number of requests sent from EspoCRM, for example:
- Only 100 requests per minute, or
- A delay of 3–4 seconds between each HTTP request, or
- Processing requests in batches instead of sending them all at once.
Is there any built-in configuration in EspoCRM (Workflow, BPM, Scheduled Jobs, or Queue system) that allows:
- Limiting the number of HTTP requests per minute
- Adding a delay between HTTP requests
- Processing records in controlled batches
Additional Issue
I am also facing another problem during this process.
The endpoint server actually returns the response successfully, but after a certain number of records EspoCRM stops saving the data. When this happens, I have to restart my Linode server, and after restarting it starts working again.
This behavior suggests that something may be getting blocked or overloaded on the server side (possibly cURL, PHP workers, or queue processing).
Could you please advise:
- What might cause EspoCRM to stop saving responses after many HTTP requests?
- Are there any server configurations or Espo settings that should be adjusted to prevent this issue?