AWS S3 filesystem CDN

Yafar_iSi · August 13, 2020, 10:24am

I followed the S3 documentation (https://docs.shopware.com/en/shopware-platform-dev-en/how-to/use-s3-datastorage) but is not very complete
I manage to store the resources in the AWS bucket but it has some performance issues
It does not compress the files

This is without the CDN

And this is with the CDN active

This is my code in config/packages/shopware.yml:

shopware:
    filesystem:
      public:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      theme:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      asset:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      sitemap:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private

Do I miss something?

It pushes my site performance (lighthouse) like 20points down

musikdiscount24 · October 19, 2020, 2:43pm

Hello,

have you found a solution for your problem?

Would be glad to know.

Greetings

Yafar_iSi · October 23, 2020, 8:06am

Yes, I found the reason and the solution

Feel free to find me in Slack „shopwarecommunity.slack.com“ @ TaiKamilla

The problem was that my SW config file config/packages/shopware.yml for the CDN and the CDN was incomplete.

The problem is that I was using the ‚url: „https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%“‘

and that address is direct to the bucket and does not accept headers, you need to do it to the cloudfront service instead

SOLUTION

For this, you need an AWS bucket and an AWS cloudfront to the bucket, make sure you can access the files from the bucket from the cloudfront provided URL

Then

Go to the .env file on your SW installation and add this

AWS_DOMAIN=„CLOUDFRONT_PROVIDED_URL.com“
AWS_ACCESS_KEY_ID=HERE_YOUR_AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=HERE_YOUR_AWS_SECRET_ACCESS_KEY
AWS_BUCKET=THE_NAME_OF_YOUR_BUCKET

2 Then go to config/packages/shopware.yml and replace the content with this (feel free to modify „region“, „endpoint“ and „endpoint“ or anything else)

shopware:
    filesystem:
      public:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      private:
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "private" # On private adapters need this to be private
      theme:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      asset:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      sitemap:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private

musikdiscount24 · October 26, 2020, 1:12pm

Hello,

ah, i see. Thank you!

But as far as i can see there are no variables for „AWS_SECRET_ACCESS_KEY“ and „AWS_ACCESS_KEY_ID“ in the shopware.yml file. Or is it not necessary to set these at options or wherever?

Is it not possible like this?!

shopware:
    filesystem:
      public:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
              credentials:
                key: "%env(AWS_ACCESS_KEY_ID)%"
                secret: "%env(AWS_SECRET_ACCESS_KEY)%"

Thank you for your answers!!!

Yafar_iSi · October 26, 2020, 1:49pm

Hey, did you try my solution anyway?

“AWS_SECRET_ACCESS_KEY” and “AWS_ACCESS_KEY_ID” it does not show in shopware.yml
but is an SW naming for the bucket credentials in /vendor/aws/aws-sdk-php/src/Credentials/CredentialProvider.php

musikdiscount24 · October 26, 2020, 2:54pm

Hello,

yes, i tried it and it works. I was just wondering about the fact, that i don’t need the Access-Key.

Thank you! You helped me a lot!

Feel free to find me in Slack “shopwarecommunity.slack.com” @ TaiKamilla

Btw. i am not invited

Best regards!

shyim · October 26, 2020, 2:57pm

@musikdiscount24‍ You can join with https://slack.shopware.com

Yafar_iSi · October 26, 2020, 2:58pm

You don’t need to be invited
Just go here https://slack.shopware.com
and create an account

Also would be great if you can mark the answer as solved

Yafar_iSi · October 28, 2020, 9:06am

Hi @musikdiscount24‍

I have a question since you use an AWS CDN

I am experiencing slow down on theme and plugin installation

without the CDN the theme compilation takes around 30-40 seconds

but with the CDN is like 180 seconds (3min)

are you experiencing a slow down in this?

klug · August 11, 2021, 9:10am

Hi Yafar_iSi,

i have tried your solution but it doesn’t work.

In the .env file:
Should the parameter be inside ‚xxx‘ or without ‚‘?

like:
AWS_DOMAIN=‚XXX‘
AWS_ACCESS_KEY_ID=‚XXX‘
AWS_SECRET_ACCESS_KEY=‚xxx‘
AWS_BUCKET=‚xxx‘

are these parameters correct?

public:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private

Best regards!