Tutorial - Calling External API To Populate Entities

[blueprint]

I thought I'd share this seeing as I went through the pain of getting this to work.

In the following example, I integrated the GitLab issue tracking API with EspoCRM so that our EspoCRM users could see all issues associated with various software repositories.

This example uses the CURL library for the HTTP requests.

Create Issue Entity
Using the Administration interface, I created a new EspoCRM Entity called Issue. This entity is ultimately populated with data returned from the GitLab API.

Implementing the External API
The EspoCRM and API go-between is done via the Issue service in the custom/Espo/Custom/Services/Issue.php file. By default, the service will call the Repositories/Issue.php functions (and base functions) to pull data from the underlying database. We want to override this behaviour and ensure that the API is called for the various CRUD methods.

The functions we need to implement are as follows (function bodies removed for brevity):

public function create($data)
{
    // Responsible for creating an entity
}

public function read($id)
​​​​​​​{
    // Responsible for reading a single entity
}

public function update($id, $data)
​​​​​​​{
    // Responsible for updating an entity
}

public function delete($id)
​​​​​​​{
    // Responsible for deleting an entity
}

public function find($params)
{
    // Responsible for searching for entities 
}

Implementation So Far
The following code is my implementation of the external GitLab API so far. Nothing is persisted in the EspoCRM database - all entities are created/read directly from the external API.

Obviously, this should be tweaked to suit the API you want to integrate with but fundamentally, most of the code would be the same (creating and populating entities, returning collections, etc).

<?php

namespace Espo\Custom\Services;

use \Espo\ORM\Entity;
use \Espo\ORM\EntityManager;
use \Espo\ORM\EntityCollection;

class Issue extends \Espo\Core\Templates\Services\Base
{[INDENT]const API_URL = 'http://gitlab/api/v4/';
const PRIVATE_TOKEN = 'INSERT_TOKEN_HERE';

private function callApi($method, $url, $data = false)
{[/INDENT][INDENT=2]$curl = curl_init();

switch ($method)
{[/INDENT][INDENT=3]case "POST":[/INDENT][INDENT=4]curl_setopt($curl, CURLOPT_POST, 1);

if ($data)[/INDENT][INDENT=5]curl_setopt($curl, CURLOPT_POSTFIELDS, $data);[/INDENT][INDENT=4]break;[/INDENT][INDENT=3]case "PUT":[/INDENT][INDENT=4]curl_setopt($curl, CURLOPT_PUT, 1);
break;[/INDENT][INDENT=3]default:[/INDENT][INDENT=4]if ($data)[/INDENT][INDENT=5]$url = sprintf("%s?%s", $url, http_build_query($data));[/INDENT][INDENT=2]}

curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

$result = curl_exec($curl);

curl_close($curl);

return $result;[/INDENT][INDENT]}

private function buildUrl($action, $params = [])
{[/INDENT][INDENT=2]$params['private_token'] = self::PRIVATE_TOKEN;

$url = self::API_URL . $action . '?';

$count = count($params);
$i = 0;

foreach ($params as $key => $value) {[/INDENT][INDENT=3]if (!empty($value)) {[/INDENT][INDENT=4]$url .= $key . '=' . $value;[/INDENT][INDENT=3]}
$i++;
[/INDENT][INDENT=2] [/INDENT][INDENT=3]if ($i < ($count)) {[/INDENT][INDENT=4]$url .= '&';[/INDENT][INDENT=3]}[/INDENT][INDENT=2]}

return $url;[/INDENT][INDENT]}

private function populateEntityFromJsonResult(Entity $entity, $result)
{[/INDENT][INDENT=2]// Populate the entity with JSON data[/INDENT][INDENT=2]$entity->id = $result->iid;
$entity->set('name', $result->title);
$entity->set('description', $result->description);
$entity->set('status', $result->state);
$entity->set('project', $result->project_id);
$entity->set('priority', $result->weight);[/INDENT][INDENT]}

public function create($data)
{[/INDENT][INDENT=2]// Get the project Id
$project = $data->project;

$postData['project_id'] = $data->project;
$postData['title'] = $data->name;
$postData['description'] = $data->description;
$postData['state'] = 'opened';
$postData['weight'] = $data->priority;

$url = $this->buildUrl('projects/' . $project . '/issues', []);

$json = $this->callApi('POST', $url, $postData);

// Decode the returned GitLab JSON Issue data
$result = json_decode($json);

// Create a new Issue entity
$entity = $this->getEntityManager()->getEntityFactory()->create($this->entityType);

if ($entity) {[/INDENT][INDENT=3]// Populate the Issue Entity with the GitLab Issue data[/INDENT][INDENT=3]$this->populateEntityFromJsonResult($entity, $result);
return $entity;[/INDENT][INDENT=2]}

return null;[/INDENT][INDENT]}

public function read($id)
{[/INDENT][INDENT=2]$url = $this->buildUrl('issues', [ 'iids[]' => $id ]);

$json = $this->callApi('GET', $url);

$result = json_decode($json);

$entity = $this->getEntityManager()->getEntityFactory()->create($this->entityType);

if ($entity) {[/INDENT][INDENT=3]$this->populateEntityFromJsonResult($entity, $result[0]);
return $entity;[/INDENT][INDENT=2]}

return null;[/INDENT][INDENT]}

public function update($id, $data)
{[/INDENT][INDENT=2]// Not yet implemented[/INDENT][INDENT=2]$GLOBALS['log']->warn('update', [$id, $data]);[/INDENT][INDENT]}

public function delete($id)
{[/INDENT][INDENT=2]// Not yet implemented[/INDENT][INDENT=2]$GLOBALS['log']->warn('delete', [$id]);[/INDENT][INDENT]}

public function find($params)
{[/INDENT][INDENT=2]// This needs a bit of work but its a started for ten[/INDENT][INDENT=2] [/INDENT][INDENT=2]$selectParams = $this->getSelectParams($params);

// Get the whereClause
$whereClause = $selectParams['whereClause'];

$urlParams = [];

if (empty($whereClause)) {[/INDENT][INDENT=3]$urlParams['scope'] = 'all';[/INDENT][INDENT=2]}
else {[/INDENT][INDENT=3]// Not yet implemented[/INDENT][INDENT=2]}

// Build the URL and append the URL parameters to the URL string
$url = $this->buildUrl('issues', $urlParams);

$json = $this->callApi('GET', $url);

$result = json_decode($json);

$count = count($result);

// Create a new EntityCollection to hold our entity list
$items = new EntityCollection([], $this->entityType, null);

// Iterate through the items returned by the GitLab Issue API
for ($i = 0; $i < $count; $i++) {[/INDENT][INDENT=3]// Create an entity
$entity = $this->getEntityManager()->getEntityFactory()->create($this->entityType);

if ($entity) {[/INDENT][INDENT=4]$this->populateEntityFromJsonResult($entity, $result[$i]);
$items->append($entity);[/INDENT][INDENT=3]}[/INDENT][INDENT=2]}

// We must return both a 'total' and a 'collection' object to allow the list view to populate
return [[/INDENT][INDENT=3]'total' => $count,
'collection' => $items,[/INDENT][INDENT=2]];[/INDENT][INDENT]}[/INDENT]
 }

[esforim]

Thank you!

(Hopefully I don't backlog this too long)

PS: is it me or the URL is a bit wrong?

You wrote it as gitlab/api/v4/
But I have the feeling it should be gitlab.com/api/v4/

[blueprint]

Ah no - sorry. So the URL is correct because we locally host GitLab - its not hosted on GitLab.com. The URL is just to give an idea of the URL for the external API. Maybe I should change this...

[esforim]

Thanks for clarification. Make sense. Thank you.

[tothewine]

Nothing is persisted in the EspoCRM database - all entities are created/read directly from the external API.

I was thinking of doing this since a long time... so do we just need to override the Service class for an Entity to do CRUD on an external database / table ?

[telecastg]

Thanks for posting this blueprint !

[blueprint]

No worries - may as well share our experiences

[Tziere]

Hi blueprint, thanks for sharing your work! I was wondering if you've developed this any further in the meantime?