Tweet
Environment
When importing a CSV (or otherwise saving) an entity with an array attribute whose values contain multibyte UTF-8 characters (Cyrillic, em-dash —, etc.) and whose byte length exceeds the array_value.value max length (default 100), the import fails with:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near ","
LINE 1: ...entity_type") VALUES ('69f867d4...', false, , 'painPoints', ...
^
…followed by the cascading:
SQLSTATE[25P02]: In failed sql transaction: current transaction is aborted, commands ignored until end of transaction block
The same input file fails in different rows on each retry — the failure looks unstable, but it is fully deterministic once you look at the data byte-by-byte. Root cause
application/Espo/Repositories/ArrayValue.php, inside storeEntityAttribute():
phpif (strlen($value) > $itemMaxLength) {
$value = substr($value, 0, $itemMaxLength);
}
strlen() and substr() are byte-level. For a UTF-8 string consisting mostly of 2-byte characters (Cyrillic) plus an occasional 3-byte one (—), cutting at byte 100 frequently lands in the middle of a multibyte sequence, producing an invalid UTF-8 string.
- EspoCRM 9.x
- PostgreSQL (client_encoding = UTF-8)
- PHP 8.3
When importing a CSV (or otherwise saving) an entity with an array attribute whose values contain multibyte UTF-8 characters (Cyrillic, em-dash —, etc.) and whose byte length exceeds the array_value.value max length (default 100), the import fails with:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near ","
LINE 1: ...entity_type") VALUES ('69f867d4...', false, , 'painPoints', ...
^
…followed by the cascading:
SQLSTATE[25P02]: In failed sql transaction: current transaction is aborted, commands ignored until end of transaction block
The same input file fails in different rows on each retry — the failure looks unstable, but it is fully deterministic once you look at the data byte-by-byte. Root cause
application/Espo/Repositories/ArrayValue.php, inside storeEntityAttribute():
phpif (strlen($value) > $itemMaxLength) {
$value = substr($value, 0, $itemMaxLength);
}
strlen() and substr() are byte-level. For a UTF-8 string consisting mostly of 2-byte characters (Cyrillic) plus an occasional 3-byte one (—), cutting at byte 100 frequently lands in the middle of a multibyte sequence, producing an invalid UTF-8 string.
Comment