AWS S3 filesystem CDN

[Yafar_iSi]

I followed the S3 documentation (https://docs.shopware.com/en/shopware-platform-dev-en/how-to/use-s3-datastorage) but is not very complete
I manage to store the resources in the AWS bucket but it has some performance issues
It does not compress the files

This is without the CDN

And this is with the CDN active

 

This is my code in config/packages/shopware.yml:

shopware:
    filesystem:
      public:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      theme:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      asset:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      sitemap:
        url: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private

 

Do I miss something?

It pushes my site performance (lighthouse) like 20points down

[musikdiscount24]

Hello,

have you found a solution for your problem?

Would be glad to know.

Greetings

[Yafar_iSi]

Yes, I found the reason and the solution

Feel free to find me in Slack „shopwarecommunity.slack.com“ @ TaiKamilla

The problem was that my SW config file config/packages/shopware.yml for the CDN and the CDN was incomplete.

The problem is that I was using the ‚url: „https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%“‘

and that address is direct to the bucket and does not accept headers, you need to do it to the cloudfront service instead 

SOLUTION  

For this, you need an AWS bucket and an AWS cloudfront to the bucket, make sure you can access the files from the bucket from the cloudfront provided URL

Then

1. Go to the .env file on your SW installation and add this

   AWS_DOMAIN=„CLOUDFRONT_PROVIDED_URL.com“
   AWS_ACCESS_KEY_ID=HERE_YOUR_AWS_ACCESS_KEY_ID
   AWS_SECRET_ACCESS_KEY=HERE_YOUR_AWS_SECRET_ACCESS_KEY
   AWS_BUCKET=THE_NAME_OF_YOUR_BUCKET

2 Then go to config/packages/shopware.yml and replace the content with this (feel free to modify „region“,  „endpoint“ and „endpoint“ or anything else)

shopware:
    filesystem:
      public:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      private:
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "private" # On private adapters need this to be private
      theme:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      asset:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
      sitemap:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private

 

[musikdiscount24]

Hello,

ah, i see. Thank you!

But as far as i can see there are no variables for „AWS_SECRET_ACCESS_KEY“ and „AWS_ACCESS_KEY_ID“ in the shopware.yml file. Or is it not necessary to set these at options or wherever?

Is it not possible like this?!

shopware:
    filesystem:
      public:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private
              credentials:
                key: "%env(AWS_ACCESS_KEY_ID)%"
                secret: "%env(AWS_SECRET_ACCESS_KEY)%"

Thank you for your answers!!!

[Yafar_iSi]

Hey, did you try my solution anyway?

“AWS_SECRET_ACCESS_KEY” and “AWS_ACCESS_KEY_ID” it does not show in shopware.yml
but is an SW naming for the bucket credentials in /vendor/aws/aws-sdk-php/src/Credentials/CredentialProvider.php

[musikdiscount24]

Hello,

yes, i tried it and it works. I was just wondering about the fact, that i don’t need the Access-Key.

Thank you! You helped me a lot!

Feel free to find me in Slack „shopwarecommunity.slack.com“ @ TaiKamilla

Btw. i am not invited

Best regards! 

[shyim]

@musikdiscount24‍ You can join with https://slack.shopware.com

[Yafar_iSi]

You don’t need to be invited
Just go here https://slack.shopware.com
and create an account

Also would be great if you can mark the answer as solved 

[Yafar_iSi]

Hi @musikdiscount24‍

I have a question since you use an AWS CDN

I am experiencing slow down on theme and plugin installation

without the CDN the theme compilation takes around 30-40 seconds

but with the CDN is like 180 seconds (3min)

are you experiencing a slow down in this?

[klug]

Hi Yafar_iSi,

i have tried your solution but it doesn’t work.

In the .env file:
Should the parameter be inside ‚xxx‘ or without ‚‘?

like:
AWS_DOMAIN=‚XXX‘
AWS_ACCESS_KEY_ID=‚XXX‘
AWS_SECRET_ACCESS_KEY=‚xxx‘
AWS_BUCKET=‚xxx‘

are these parameters correct?

public:
        url: "https://%env(AWS_DOMAIN)%"
        type: "amazon-s3"
        config:
            bucket: "%env(AWS_BUCKET)%"
            region: "eu-central-1"
            endpoint: "https://s3.eu-central-1.amazonaws.com/%env(AWS_BUCKET)%"
            root: "/"
            options:
              visibility: "public" # On private adapters need this to be private

Best regards!

[toco3]

Thank you for all your hints.
I’d had also some Issues connecting a Shopware 6.5.6.1 to AWS S3 / CloudFront

I need the following adaptions to config/packages/z-shopware.yml in comparison to the one of @Yafar_iSi :

endpoint: "https://s3.eu-central-1.amazonaws.com"

if I add the bucket as proposed the bucket name is twice in the url and does not work

           root: ""

If I add „/“ for root I get a directory with the name „/“ in the root of my s3 bucket with all stuff inside…

Beside configuring the CNAMES for S3 and CloudFront I had some CORS issues.

For CloudFront I use this settings for the behaivor:

and added a custom Header to the S3 origin:

[
    {
        "AllowedHeaders": [
            "Authorization"
        ],
        "AllowedMethods": [
            "GET",
            "HEAD"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]

For the S3 Bucket I have this access policies:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTorrent",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectTorrent",
                "s3:GetObjectVersion"
            ],
            "Resource": "arn:aws:s3:::static.shirt-helden.shop/*"
        }
    ]
}

and this CORS settings:

[
    {
        "AllowedHeaders": [
            "Authorization"
        ],
        "AllowedMethods": [
            "GET",
            "HEAD"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]

With all my tests, sometimes the directories and their content like bundles, theme, thumbnail got created - some times not.

These commands helped to get everything up and running:
./bin/console cache:clear
./bin/console theme:compile
./bin/console media:generate-thumbnails --strict -v
./bin/console system:update:finish

Perhaps this can help someone having the same issues as me - took me about an day to figure this all out

[toco3]

one additional information:

for everyone who can open the backend only in Firefox:

make sure that you disabled the Admin Worker in config/packages/z-shopware.yaml:
shopware:
admin_worker:
enable_admin_worker: false

(Admin worker should be disabled on production server anyway)

[AlexGalax]

I have the same issue – some thumbnails just do not get generated.

When I run media:generate-thumbnails --strict -v, I get the following output for a lot of files:

  Cannot process file 2505600920-06 (id: f57c3d5ae20d4b208a111cc90d71d7fe) due error: The file for media object with id f57c3d5ae20d4b208a111cc90d71d7fe is not supported for creating thumbnails.

It sounds like there is something wrong with the original image, which I doubt, because its just too many and they are good enough for Shopware 5. Now I have to figure out, whats wrong with these images or why sw6 does not like them.

Did you found out already, @toco3 ?