Problem with email attachments after moving to the new server

[Jakub Grufik]

I took random corrupted PDF from new CRM, took its ID and found it in the Upload of the original CRM where it is working without any problem.. Moved it manually to the new CRM/Upload but it is still being displayed incorrectly

[Kharg]

Jakub Grufik can you try to upload and replace it by using your hosting service upload panel? Or another client such as CyberDuck? (Try both ways)

When you try to do the same with the corrupted pdf does it appear corrupted in windows as well?

[Jakub Grufik]

I just deleted corrupted PDF from the upload folder and then I put there a fresh PDF from the original CRM where I am 100% sure it is working properly (tested by adding .pdf and checked) . And it is corrupted again immediately. In the screens you can see original vs corrupted

[Kharg]

If you try to upload the same pdf using espo attachment field is the pdf uploaded correctly?

[Kharg]

Jakub Grufik can you also make sure to install all the required php extensions? From the screenshot you posted it doesn’t look like you have them all.

https://docs.espocrm.com/administration/server-configuration/#php-requirements

[Jakub Grufik]

Kharg I tried to upload PDF manually via documents (Create document) and it is working properly. Also system is saying that my server settings and its extensions are OK

[Jakub Grufik]

server settings are exactly the same as in the original CRM

[Kharg]

Please, still check from your hosting panel if all extensions are installed.

Also did you try to upload the attachment from your hosting panel?

[Jakub Grufik]

Hello Kharg I checked modules in the server console and you can see that we have all of the required php modules installed on our server(screen). I have really no idea what could be causing the problem

[item]

Hi,
i have this in the past, but a little forget, here what I have do in the past :

delete cache folder
disable cache in admin setting (enable cache and direct file is corrupted with extra space on beginning of downloaded file)
maybe I have copy from zip version of espocrm the vendor folder and past .. I have rename old vendor..and copy new.
clear cache on browser.

but if not forget, I have in one file some extra empty space on the beginning of file, this extra space is I think corrupt header so file become corrupted

actually, I have issue when I create docx from espocrm, if a field contains “&” and I use this field in docx, the file is corrupted. (For information, this is not valuable for your issue)

last think to do : you can copy all file from your version of espocrm to all folder and let custom and data folder of course.

To try.
juste one question, all file in upload folder is not corrupted if you download and open with FileZilla ?

[Jakub Grufik]

Hello sir, thank you for your time.

When I download some PDF from upload folder via FileZilla from original CRM (file name: 6412c50d4ee225226 for example) and add there.pdf to the name of the file, then it can be opened and it is working perfectly fine. When I upload this working file to the "broken CRM ftp" and immediately download it back without even opening CRM app then add .pdf to the name and open it. Its already broken even I did not open CRM app..

[item]

Haaa ok,
This is not a espocrm issue then !!!

Hum hum..never listen this kind of issue. Will look Google

[item]

Have you read and try something so :

https://www.jscape.com/blog/ftp-binary-and-ascii-transfer-types-and-the-case-of-corrupt-files

[Jakub Grufik]

Here on the video you can see that I have completely working file from Upload.
1. Added .pdf displayed without any problem.
2. Uploaded it to Upload of broken CRM.
3. Downloaded back immediately without opening CRM
4. Added .pdf to try to check it
5. its broken already..


I just dont get it

https://vimeo.com/819030075/5556c15ff8

[Jakub Grufik]

item omg man that could be it.. when I try to list all files in upload folder, server pops message that " Server does not support symbols, which are not in ASCII" and then proceeds to list all files.. maybe it is connected with this issue?

[Kharg]

Please try another FTP client too!

[Kharg]

Also, try to upload a working attachment from the old crm as a zip to the new one and unzip it to see if it works.

[Jakub Grufik]

OMG man!!! item when I set transfer method to "Binary" and then reuploaded attachment to Upload of broken CRM, it is now working as expected!!!!!!!!!!! This could be it I think...

Oh my god you are such a genius man! I am going to try to reupload all the attachment via method "binary" and hopefully it will be resolved. If so, I owe you whatever you ask for

[item]

happy this resolve your issue.

[Jakub Grufik]

it really did resolve the issue! thanks man! I appreciate your help!

[Jakub Grufik]

So guys, it seems like the issue has been resolved! rabii item yuri , Kharg

The problem was connected to the Transfer method (FileZila default method for FTP was ASCII and for images etc. it needs to be Binary method). When I switched the transfer method to Binary and reuploaded corrupted attachments, it is now working perfectly fine! I just need to reupload 100GB of attachments and we are done here!

https://www.jscape.com/blog/ftp-binary-and-ascii-transfer-types-and-the-case-of-corrupt-files

Thanks a lot for your time and for your help! Special and HUGE thanks goes to item who is clever enough to be able to understand my "not so good" description of the problem and was able to detect that it is not connected with Espo and that it should be caused somewhere during the transfer of the files!

Thank you a thousand times guys! I really appreciate the willingness and the effort you are putting into helping other users of the system.

I hope that sometime I will be able to pay back your time and helpfulness. I owe all of you, specially item !!!

[rabii]

great that you sorted it out. thanks for sharing and well done item

[item]

Hi Jakub, just for knowledge : what is operating system of your new server ? Can you give some info about ?

i have create from scratch many server, in Linux, RedHat, AlmaLinux, RockyLinux, Centos, Debian in various version, never had a issue like you.
i have never touch setting of FileZilla.

now, i create a VPS for espocrm in +-10min, with +-5 command line, and 0 issue.

ha too important for me, i never name a folder with space or special char when working with something interact with server (like FileZilla)

[Jakub Grufik]

Hey man, I will check the server tomorrow.

Btw it probably did not happen to you because you are transferring espo using the correct way (backup with the script and then restoring from backup). I did not have enough space on the server to be able to make backup in the same directory as my application. That is why I had to improvise and just copy-paste paste whole FTP from 1 server to another manually via FileZilla. So basically i downloaded whole directory with the app from FTP to my PC and then uploaded it to the fresh FTP on the server I was migrating to. That is whre the issue happened.

FileZilla has default transfer method "Automatically detect" and probably the fact that the format attachments are stored in the folder is without .pdf, .jpg etc. FileZila decided to choose ASCII method incorrectly and transfered files were corrutped during the transfer.

I bet you never tried to paste 100GB of attachments manually without proper backup-restore script via FileZilla.

Cheers

[Jakub Grufik]

+ our app is running on the shared webhosting by websupport.sk so I do not have full control of the server.

[item]

You bet ?

No 100GB .. but 60GB i think.
my big challenge was migrate from sugarCRM to espoCRM .. and there i have make a script, who read sugarCRM.. and then create record in espoCRM. Many need mapping field and so.
so i have just do the script in php, some entryPoint for download document... i used only fileZilla for upload my script

This give a new idea for extension to do : you just need dump Sql fromSql... and import in toCRM
fromCRM :
new entryPoint for download all files without Auth

toCRM
a job who just fileGetContent(fromCRM) and filePutContent(toCrm)

more easy