Validation issue in URL field

[shalmaxb]

I have the following issue espoCRM 7.2.4, custom entity URL field, validation problem

I store in an URL field the adress of a OneDrive folder, which looks like this: https://onedrive.live.com/?id=A04084...408433333B8B5B (id is fake).

I am able to do the following in full edit view:

I create a new record including the mentioned Url, on save everything ok

I edit an existing record ( even in in another field), on save validation error

Inline editing:

I am able to edit another field, no validation error.

[yuri]

Could not reproduce. Not enough details provided. Log error. URL provided seems not actual as it contains three dots.

[shalmaxb]

The url does not work, because it contained an id to my data. The original url does not have dots, only digits.

Nothing in either espoCRM log, Apache Log and nothing in console.

Tried again and the provided url does not save. When I truncate all digits from the link I can save.

[yuri]

Not clear what exactly error. We have front-end and back-end validations. Screenshot or exact error text is needed.

[yuri]

Did you try reproducing with another field, maybe on another instance. Maybe some customization causing this issues. I go offline, it's my weekend I almost never have rest days.

[shalmaxb]

Hi Yuri,
I do not have any pressure and sure you deserve rest days.

I tried to reproduce in another instance, same error. It certainly has to do with the form of the URL, because, when I put in a more "normal" URL, it does not happen.

In the attachment a screenshot of the field (I could not capture the hint). The hint said "Backup OneDrive ist ungültig" (Backup OneDrive is not valid).

[eymen-elkum]

Can you reproduce on a new copy of espocrm? if yes just tell the steps, otherwise would be related to a change made to your crm directly or using an extension.
Best regards

[eymen-elkum]

Can you reproduce on a new copy of espocrm? if yes just tell the steps, otherwise would be related to a change made to your crm directly or using an extension.
Best regards

[eymen-elkum]

Can you reproduce on a new copy of espocrm? if yes just tell the steps, otherwise would be related to a change made to your crm directly or using an extension.
Best regards

[shalmaxb]

Installed a clean espoCRM instance and tested. The error happend there as well. When looking at the URL, that links to a OneDrive folder, I saw, that every OneDrive link seems to contain an exclamation char (!), that is the culprit. As soon as I delete that char, it works.
I conclude from this, that the validation algorithm sees an exclamation as non valid character.

I then substituted the ! with %21 and this way it worked, in the link appears the ! but it is encoded by %21. At least a workaround.

My question now is: If Yuri not could reproduce it, could it be because in my case I use in MySQL collation utf8mb4_general_ci. Is that correct?

[shalmaxb]

And this for the records:

RFC 1738 Uniform Resource Locators (URL) December 1994


Reserved:

Many URL schemes reserve certain characters for a special meaning:
their appearance in the scheme-specific part of the URL has a
designated semantics. If the character corresponding to an octet is
reserved in a scheme, the octet must be encoded. The characters ";",
"/", "?", ":", "@", "=" and "&" are the characters which may be
reserved for special meaning within a scheme. No other characters may
be reserved within a scheme.

Usually a URL has the same interpretation when an octet is
represented by a character and when it encoded. However, this is not
true for reserved characters: encoding a character reserved for a
particular scheme may change the semantics of a URL.

Thus, only alphanumerics, the special characters "$-_.+!*'(),", and
reserved characters used for their reserved purposes may be used
unencoded within a URL.

On the other hand, characters that are not required to be encoded
(including alphanumerics) may be encoded within the scheme-specific
part of a URL, as long as they are not being used for a reserved
purpose.

Source: https://www.rfc-editor.org/rfc/rfc1738

[yuri]

I made fixes to URL regex: https://github.com/espocrm/espocrm/c...5f95b8c2613d6b

[shalmaxb]

yuri, thank you, works.

[espcrm]

I recommend trying to reproduce on the demo, that way it is 'reproducible'. And eliminate the issue of "is it me or EspoCRM"

However people don't like that idea of using the demo for this purpose apparently.