Can I use elasticsearch/opensearch in community edition?

For now I’m planning to use Shopware for free, with community edition for a startup project.

In the dev documentation I read about using Elasticsearch/Opensearch and started setting it up according to the docs. However, at the same time I see in the docs that:

To configure which fields and elements are searchable when using Elasticsearch, you must install the extension Advanced Search.

And seeing that:
The Advanced Search is available to you as an extension of the Shopware Evolve plan.

I’m not sure therefore, what is included in the Community/free plan, is opensearch included at all, and only configuring special/custom fields is part of the Evolve plan, or the whole Elasticsearch belongs to paid plans only?

Started thinking also as although I followed instructions to setup ES, I’m getting:

No alive nodes found in your cluster

For Shopware 6.4.20.x currently only elasticsearch 7 seems to be working properly.
For 6.5.0 neither elasticsearch nor opensearch ran successfully in my setup

Make sure to setup your .env properly:

SHOPWARE_ES_HOSTS=elasticsearch:9200
OPENSEARCH_URL=elasticsearch:9200
SHOPWARE_ES_ENABLED=1
SHOPWARE_ES_INDEXING_ENABLED=1
SHOPWARE_ES_INDEX_PREFIX=apcsw6
SHOPWARE_ES_THROW_EXCEPTION=1 # this prevents querying the DB as fallback
ADMIN_OPENSEARCH_URL=elasticsearch:9200
SHOPWARE_ADMIN_ES_ENABLED=1
SHOPWARE_ADMIN_ES_REFRESH_INDICES=1
SHOPWARE_ADMIN_ES_INDEX_PREFIX=sw6_admin
bin/console es:admin:index
bin/console es:index

For using OpenSearch in SW 6.5.x you need the following additional line in your .env (or .env.local):

ES_MULTILINGUAL_INDEX=1

This just fixed several things for me:

  • indexes for product_manufacturer and category were missing, now they are created
  • in admin > settings > seach, I now get additional features for configuring Advanced Search
  • in frontend, the search is now working. Previously I got an elasticsearch.CRITICAL index_not_found_exception: no such index [myprefix_product]
1 „Gefällt mir“

Getting back to this very late; thank you both for the valuable input.
I managed to finally set the environment up with AWS / Amazon OpenSearch service (if someone needs info about the AWS settings and configuring security, I’m happy to help).

I have two more additions:
After running es:index and es:admin:index I was still getting errors about indexes not found; running bin/console es:create:alias solved it finally.

Another interesting thing is that when I finally managed to set up the OpenSearch connection, I was getting a MySQL error when trying to run the indexes. It turned out that two tables: admin_elasticsearch_index_task and elasticsearch_index_task were not created with my latest installation, based on Shopware 6.5.8.7. However, I found those tables in a database instance from an earlier version and copied those over.
Not sure why those tables are missing or if an additional component or migration should be run.

  • in admin > settings > seach, I now get additional features for configuring Advanced Search

Did you install this in community edition? Did you need to install something separately to see Advanced Search configuration? For me it’s still not visible in the admin.

Yes, in the community edition, but with an (older) Diamond license.

I uninstalled the old SwagEnterpriseSearchPlatform plugin, which was required in v6.4.x. For v6.5.x I installed the SwagCommercial plugin, which is required for activating any feature for which you need a license. Elastic/OpenSearch is available from the Evolve license upwards, and for some of the older licenses.

Thanks for the clarification.
So at the current point it seems that I can have OpenSearch running in the community edition, but don’t have a tool to configure it :slight_smile: (unless I figure out how to do it manually).
But maybe the default configuration would do it, I’m still just experimenting/setting up my first webshop.

Hey @akos, did you figure out how to configure it manually // was the default config sufficient?

I’m looking into the same topic currently

Hi Nico, actually, yes. Here are my notes for everyone interested, how to set it up in AWS environment, using AWS OpenSearch:

  • Use Amazon OpenSearch service: https://eu-north-1.console.aws.amazon.com/aos/
  • Create a domain
    • Eligible for free tier: t3.small.search
  • Make sure to create in the same VPC as EC2
  • Under security configuration:
    • Enable fine-grain control and create an admin username and password (to be used in the connection string, find below as [os_username]:[os_password])
  • Create a VPC endpoint:
    • Same VPC as EC2
    • Same Subnet as EC2
    • The opensearch security group:
      • Allow inbound on 22 and 443 from the EC2’s security group
      • Apply this security group to the VPC and the OpenSearch domain

Shopware configuration (add to .env.local):

OPENSEARCH_URL=https://[os_username]:[os_password]@[vpc-....es.amazonaws.com]:443
SHOPWARE_ES_ENABLED=1
SHOPWARE_ES_INDEXING_ENABLED=1
SHOPWARE_ES_INDEX_PREFIX=sw6
SHOPWARE_ES_THROW_EXCEPTION=1 # this prevents querying the DB as fallback
ADMIN_OPENSEARCH_URL=https://[os_username]:[os_password]@[vpc-....es.amazonaws.com]:443
SHOPWARE_ADMIN_ES_ENABLED=1
SHOPWARE_ADMIN_ES_REFRESH_INDICES=1
SHOPWARE_ADMIN_ES_INDEX_PREFIX=sw6_admin
SHOPWARE_ES_MULTILINGUAL_INDEX=1

Running initial indexing (in Shopware root folder):

bin/console es:admin:index
bin/console es:index
bin/console es:create:alias

Setting up a tunnel to the dashboard

Set up a tunnel in SSH connection as:
ssh -i .\.ssh\[server-private-key].pem ubuntu@[IP] -N -L 9100:[vpc-....es.amazonaws.com]:443

Access the dashboard from the local machine through: https://localhost:9100/_dashboards/

Important part: Database

Two tables were missing in the latest install: admin_elasticsearch_index_task and elasticsearch_index_task were not created with my latest installation, based on Shopware 6.5.8.7. However, I found those tables in a database instance from an earlier version and copied those over.

--
-- Set character set the client will use to send SQL statements to the server
--
SET NAMES 'utf8';

--
-- Create table `elasticsearch_index_task`
--
CREATE TABLE elasticsearch_index_task (
  id binary(16) NOT NULL,
  `index` varchar(500) NOT NULL,
  alias varchar(500) NOT NULL,
  entity varchar(500) NOT NULL,
  doc_count int NOT NULL,
  PRIMARY KEY (id)
)
ENGINE = INNODB,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci,
ROW_FORMAT = DYNAMIC;

--
-- Create table `admin_elasticsearch_index_task`
--
CREATE TABLE admin_elasticsearch_index_task (
  id binary(16) NOT NULL,
  `index` varchar(500) NOT NULL,
  alias varchar(500) NOT NULL,
  entity varchar(500) NOT NULL,
  doc_count int NOT NULL,
  PRIMARY KEY (id)
)
ENGINE = INNODB,
AVG_ROW_LENGTH = 1024,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci,
ROW_FORMAT = DYNAMIC;
2 „Gefällt mir“

Awesome, thank you so much @akos ! :fist_right:t2: :fist_left:t2:

1 „Gefällt mir“