Issue with dependency in production - Testing works

[ThomasH]

Hey Community,

I experience an issue with a dependency I use for converting images.

I automatically want to convert attachments in HEIC format, as we can’t work with that format when processing the tickets.

After researching for options, I decided to go for a combination of HEIC Convert (for the conversion) and Unirest (for uploading the converted heics in a ticket comment).

I was able to successfully convert and upload the converted pics in test environment.

But I experience issues when using heic-convert in production:

async function convertFile(fileUrl, name) {
    try {
        const heicFile = await getFile(fileUrl);
        console.log(heicFile);
        console.log(convert);
        const jpgFile = await convert({
            buffer: heicFile, // the HEIC file buffer
            format: 'JPEG',      // output format
            quality: 1           // the jpeg compression quality, between 0 and 1
        });
        console.log(jpgFile);
        return {
            "name": name.replace(/\.[^.]+$/, '_converted.jpg'),
            "file": jpgFile
        }
    } catch (error) {
        // Log the error and end the function execution
        console.error(error);
        return null;
    }
}

Logs from test environment:

Ticket 1910084, sample1.heic (type image/jpeg) - convert to JPG
<Buffer 00 00 00 18 66 74 79 70 6d 69 66 31 00 00 00 00 6d 69 66 31 68 65 69 63 00 00 01 fe 6d 65 74 61 00 00 00 00 00 00 00 21 68 64 6c 72 00 00 00 00 00 00 … 293558 more bytes>
[AsyncFunction: one] { all: [AsyncFunction: all] }
<Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 84 00 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 … 2194927 more bytes>

Logs from production:

Time	Id	Message
Jan 4, 2024, 4:10:44 PM	65e58	Ticket 2053287, sample1.heic (type application/octet-stream) - convert to JPG
Jan 4, 2024, 4:10:44 PM	65e58	<Buffer 00 00 00 18 66 74 79 70 6d 69 66 31 00 00 00 00 6d 69 66 31 68 65 69 63 00 00 01 fe 6d 65 74 61 00 00 00 00 00 00 00 21 68 64 6c 72 00 00 00 00 00 00 … 293558 more bytes>
Jan 4, 2024, 4:10:44 PM	65e58	[AsyncFunction: one] { all: [AsyncFunction: all] }

In difference to test, production stops exactly when I do the conversion.
Why? An exception would be catched and I would see the error log, wouldn’t it? Does the “await convert…” never return?
Or what is happening here?

Hope for some help!

All the best,
Tom

[ThomasH]

Update:
Conversion is working now more or less. “Accidently” found the solution.
I now put the conversion quality down to 0.5 instead of 1.

Now the conversion takes around 8-10 seconds for a 300KB HEIC file (which takes 0.5 seconds in test environment).

I didn’t have any production example with bigger files (or with more than one file).
So I monitor the app closely now and try to figure out, which quality works best.

But the question arising out of this new finding is:
Why is there such a low performance on prod?
I understand that we are not the only one running serverless apps and need to share resources, but I did not expect such a low performance.

Has anyone had similar issues?

Tom

[ThomasH]

Update of the Update:
As kind of expected, bigger files do not work. So with a 300KB HEIC file it works (conversion time between 5-10 seconds), but usual filesizes we get are around 2-4 MB.

I reduced quality to 0.15, which is still okay, but does not help - App still stuck when converting.

I also tried to run conversion asynchronous in the hope, that this will allow for more resources → Did also not resolve the problem.

I also already asked Freddy Co-Pilot, which suggests to move the conversion to front-end instead of serverless.
Is also not an option, as in the future, it is planned to automatically convert + upload these pictures without any agent involved.

Is there a way to increase computing power for some apps?

[ThomasH]

Solution:
What I tried is not possible with serverless apps.
Serverless apps do have a timeout of 20 seconds and

• Converting several files (up to 15-25 MB in total)
• Uploading them via API call
  is not possible with given serverless resources

So I now do the conversion in a third party app and serverless app does only trigger that external service.