Release notes for ece-tools

The ece-tools package is compatible with Magento Commerce version 2.1.4 and later to provide a rich set of features you can use to manage your Magento Commerce Cloud project. It contains scripts and Magento Commerce Cloud commands designed to help manage your code and automatically build and deploy your projects.

You can list the available ece-tools commands using:

php ./vendor/bin/ece-tools list

The following updates describe the latest improvements to the ece-tools package, which uses the following version sequence: 200<major>.<minor>.<patch>. See Upgrades and patches for information about updating to the latest release of the ece-tools package.

The release notes include:

  • New features
  • Fixes and improvements

v2002.0.15

  • Docker updates

    • Now the Docker generator uses the services specified in the .magento.app.yaml and .magento/services.yaml configuration files when building your Docker environment. You can choose a different service version using build parameters.

    • Added PHP 7.2 image—Added support for PHP 7.2 in Cloud Docker; updated the Launch Docker configuration to include the docker:build --php option to specify the version of PHP compatible with your Magento Commerce version.

    • Added a Cron container based on the PHP-CLI image.

    • Added the following services to the Docker build:

      • RabbitMQ 3.5 and 3.7

      • ElasticSearch 1.7, 2.4, and 5.2

      • Redis 3.2 and 4.0

    • Now you have the DB dump capability when using the Cloud Docker database container. Also, you can share files between a host machine and a container using the docker/mnt directory.

  • Configure with PHP constants—Added support for PHP constants in the .magento.env.yaml configuration file.

  • New Environment variable—By default, only the Production environment has Google Analytics enabled. You can enable Google Analytics on the Staging and Integration environments using the ENABLE_GOOGLE_ANALYTICS environment variable.

  • Fixed an issue that removed customized cron configurations from the env.php file after a redeployment. Now, custom cron configurations safely remain in the env.php file.

  • Fixed inconsistencies in the messages and log levels for build, deploy, and post-deploy phases. Increased beginning and ending log message levels from info to notice for all phases and sub-phases. Added beginning and ending log messages, where appropriate.

  • Fixed an issue involving cron processes that prevented the start of the post-deploy phase, when configured. Now, if you have the post-deploy hook enabled, the cron processes are enabled again at the beginning of the post-deploy phase.

  • Resolved an issue that prevented a successful installation of Magento when specifying a custom database configuration. Previously, the installation process used the database configuration from the MAGENTO_CLOUD_RELATIONSHIP variable even if you designated customized connection information in the DATABASE_CONFIGURATION environment variable.

  • Corrected the config:dump command so that it includes each website locale in the system section of the config.php file.

  • Fixed an issue that resulted in warm-up errors during the post-deploy phase by correcting the source base URL reference.

  • Fixed an issue that generated files improperly during the setup:di:compile process, which affected the Amazon Pay module.

v2002.0.14

  • Verify Ideal State—The ideal-state wizard now verifies the current configuration during each deployment and provides clear instructions for updating the configuration to achieve a faster, zero-downtime deployment.

  • PCI Compliance—Updated the messaging protocols for Magento Commerce Cloud to require Transport Layer Security (TLS) version 1.2 when connecting with third-party messaging services. If you are using a message service that does not support TLS version 1.2, you must upgrade your service. Otherwise, the following error message displays when your Magento Commerce application tries to connect to the message server to send an email: Unable to connect via TLS.

  • Deployment improvement—Added validation to warn customers if a Staging or Production environment has dev, debug, or debug_logging options enabled to prevent performance issues caused by excessive logging activity.

  • Deployment fixes

    • Now maintenance mode is enabled at the start of the deploy phase and disabled at the end. If the deployment fails, the site remains in maintenance mode until the deployment issues are resolved. Previously, the site returned to production mode even if the deployment failed.

    • Reworked the deploy phase validation checks to downgrade the error level for the following deployment issues from CRITICAL to WARNING so that the deployment completes. Previously, these issues caused the deployment to fail.

      • ADMIN_EMAIL is not set on upgrade.

      • ADMIN_EMAIL or ADMIN_USERNAME is associated with another account.

      • Environment configuration contains incorrect values for deploy or cloud variables.

      • The Elasticsearch version on the cloud infrastructure is incompatible with the version of the elasticsearch/elasticsearch module supported by Magento Commerce Cloud. See the Elasticsearch troubleshooting article in the Magento Support Knowledgebase.

    • Fixed an issue with the shared configuration settings in the app/etc/config.php file that caused recursion detected errors during deployment.

  • Cron-related fixes

    • Fixed a cron scheduling issue that prevented jobs from running if you specify a cron frequency other than the default (1 minute).

    • Fixed an issue in the deploy phase that allowed cron jobs to continue running during deployment, which can cause database locks and other critical issues. Now, all cron jobs are stopped before the deploy phase begins and restarted after the deployment completes.

    • Fixed the cron job workflow in versions 2.2.x to unlock frozen cron jobs so that they can be stopped before beginning deployment. Previously, a frozen cron job caused the deployment to stall.

  • Changed the format of the config.php file generated by the vendor/bin/ece-tools config:dump command to use short array syntax and 4-space indentation to comply with Magento coding standards.

  • Fixed a deployment error that occurs when the .magento.env.yaml contains {{base_url}} and {{unsecure_base_url}} placeholders for web configurations instead of the default URL configuration for a Magento Commerce Cloud project./

v2002.0.13

  • Enable zero-downtime deployment—Now Magento Commerce Cloud queues requests with required database changes during deployment and applies the changes as soon as the deployment completes. Requests can be held for up to 5 minutes to ensure that no sessions are lost. See Static content deployment options to reduce deployment downtime on Cloud.

  • Docker Compose for Cloud—Made the following improvements to the Docker setup and configuration process:

    • Added a command—docker:config:convert to convert PHP configuration files to Docker ENV format to simplify environment configuration. Now, you copy the PHP configuration files to the Docker directory and convert them to Docker ENV files. See Launch Docker.

    • The Magento Commerce Cloud installation process now supports deploying to both read-only and read-write file systems to more closely emulate the Cloud file system. See Configure Docker.

    • Redis service support—Added a Redis image, which is deployed to a Docker container and configured automatically to work with your Docker installation.

    • Varnish service support— Added a Varnish image, which is deployed automatically to a Docker container. After deployment, you can manually configure Varnish following Magento best practices. See Configure and use Varnish.

    • Secure site access—Added SSL support to access your Magento Commerce store and Admin panel.

  • Improved Magento Commerce Cloud extension support—Downgraded the minimum version requirement for the guzzlehttp/guzzle package in the Magento Commerce Cloud composer.json file to version 6.2 so that the ece-tools package is compatible with more extensions.

  • Apply custom changes to your Magento Commerce application during the build phase—We split the build phase into two separate processes so that you can use hooks to apply custom changes to the generated static content before packaging the application for deployment. The build:generate process generates code, applies patches, and generates static content. The build:transfer process transfers the generated code and static content to the final destination. See Application hooks.

  • Environment configuration checks—Improved validation of the environment configuration to warn customers about version incompatibilities and configuration errors before building and deploying Magento Commerce Cloud.

    • Added version-specific validation to identify unsupported or deprecated environment variables and values.

    • Added an Elasticsearch compatibility check to warn users about Elasticsearch configuration issues. Now, the deployment fails if the version of Elasticsearch service on the server is incompatible with Magento Commerce. Previously, the deployment succeeded even if the Elasticsearch version was incompatible, which caused product catalog issues after site deployment.

      You can resolve the incompatibility by submitting a Support ticket to upgrade Elasticsearch to a compatible version, or change the Magento Commerce configuration to specify a compatible version of the Elasticsearch PHP client.

      • For Magento Commerce version 2.1.x to 2.2.2, upgrade Elasticsearch to version 2.4.

      • For Magento Commerce version 2.2.3 and later, upgrade Elasticsearch to version 5.2.

      • If you have Elasticsearch 1.x or 2.x and do not want to upgrade, update the Magento Commerce Elasticsearch PHP client version requirement in composer.json to "elasticsearch/elasticsearch": "~2.0".

    • Improved validation of environment variables to identify configuration settings that can cause conflicts during the build, deploy, and post-deploy phases. For example, a warning message displays during the install and upgrade process if the global setting for static content deployment conflicts with settings on the build or deploy phase.

  • Environment variable updates—Changed the following environment variables:

    • SKIP_HTML_MINIFICATION global variable—Changed the default value to true to enable on-demand HTML content minification, which minimizes downtime when deploying to Staging and Production environments. This configuration is required for zero-downtime deployments.

    • CLEAN_STATIC_FILES deploy variable—Added the capability to manage the clean static files processing for static content generated during the build phase based on the CLEAN_STATIC_FILES environment variable setting. Previously, static content files generated during the build phase were always cleaned.

  • Logging—Made the following changes to improve log messages and reduce log size:

    • Deployment failure log entries now include the command output from the operations that cause the failures even if your environment configuration does not specify debug level logging. See MIN_LOGGING_LEVEL.

    • Added logging for deployment failures that occur when generated factories required by some extensions cannot be generated correctly because the file system is in a read-only state.

    • Reduced the deploy log size and fixed formatting issues caused by setup commands that use the interactive progress bar.

    • Eliminated unnecessary verbosity and updated the priority levels for some log statements.

  • Cron-specific fixes

    • Changed the default cron job configuration settings for history lifetime from 3d (4320 min) to 1h (60 min) to prevent performance issues and deployment failures that can occur when the cron queue fills too quickly.

    • Improved the cron job management process during the deploy phase to prevent database locks and other critical issues. Now, all cron jobs stop during the deploy phase and restart after deployment completes.

    • Fixed an issue with the locking mechanism for scheduling consumers launched by cron jobs in Magento Commerce versions 2.2.0 and later to prevent cron jobs from launching duplicate consumers.

  • Fixed an issue with the static content compression process (gzip) that caused not overwritten and no such file or directory errors when referencing the compressed file during the deployment process.

  • Fixed an issue that prevented the php ./vendor/bin/ece-tools config:dump command from removing redundant sections from the config.php file during the dump process if the store locale is not specified. Now you can easily move your configuration files between environments. After you update to ece-tools v2002.0.13, regenerate older config.php files with the improved config:dump command. See Configuration management for store settings.

  • Fixed an issue that caused an error during the deploy phase if the route configuration in the .magento/routes.yaml file redirects from an apex domain to a www domain.

  • Fixed an issue with the _merge option for the SEARCH_CONFIGURATION variable that caused incorrect merge results if you do not include the engine parameter in the updated .magento.env.yaml configuration file. Now, the merge operation correctly overwrites only the values you specify in the updated .magento.env.yaml without requiring you to set the engine parameter.

  • Fixed a Redis configuration issue that incorrectly enabled session locking for Magento Commerce Cloud versions 2.2.1 and later, which can cause slow performance and timeouts. Now, session locking is disabled by default. The issue was caused by a change in the default behavior of the disable_locking parameter introduced in v1.3.4 of the Redis session handler package. See colinmollenhour/php-redis-session-abstract package.

v2002.0.12

  • Docker Compose for Cloud—Added a command—docker:build—to generate a Docker Compose configuration from the Cloud ece-tools repository.

  • Change Locales—Now you can change store locale without the exporting and importing configuration process. While Magento is in Production and the SCD_ON_DEMAND is enabled, the Magento store and admin locale options are available.

  • Site map and Robots—Created a workflowto add a robots.txt file and generate a sitemap.xml file for a single domain configuration without requiring a change to the infrastructure.

  • Wizards—Added two wizards to help you with Cloud configuration:

    • ideal-state—configure the ideal state for minimal deployment downtime
    • master-slave—configure load balancing for database and Redis
  • Module Refresh—Added a Cloud command—module:refresh—to enable modules that were disabled or not explicitly enabled, similar to the way that it is done automatically during a build. See Build and deploy on local in Build phase.

  • Added the ability to choose to merge or overwrite configuration for services using the _merge option in CACHE, SESSION, QUEUE, and SEARCH configurations.

  • Environment Configuration sample file—We added a .magento.env.yaml sample file to the ece-tools package that includes a detailed description and possible values for each environment variable.

    • We also added a deep validation for the .magento.env.yaml configuration that prevents failures in the deployment process caused by unexpected values. When a failure occurs, you now receive a detailed error message that begins with: Environment configuration is not valid. Please correct .magento.env.yaml file with next suggestions:

  • Added the following Environment variables:

    • Now you can define multiple locales for each theme using the new SCD_MATRIX environment variable, which reduces the amount of theme files to deploy.

    • Added the DATABASE_CONFIGURATION environment variable to customize your database connections for deployment.

    • The new MIN_LOGGING_LEVEL variable overrides the minimum logging level for all output streams without making changes to the code.

  • Fixed an issue that caused downtime between the deploy and post-deploy phase. Now, the post_deploy phase begins immediately after the deploy phase ends.

  • Fixed an issue that did not clean the successful cron jobs, those with status = success, from the schedule.

  • Fixed an issue with the post_deploy hook that cleared the cache in the deploy phase instead of the post-deploy phase of the project.

  • Fixed an issue when using SCD with multiple locales, which generated the same js-translation.json file for each locale.

  • Optimized the db:dump command in the ece-tools package to avoid locking tables and increase speed.

v2002.0.11

The ece-tools version 2002.0.11 is required for 2.2.4 compatibility.

New features

  • Configuring read-only connections to non-master nodes—This release adds the ability to configure a read-only connection to a non-master node to receive read-only traffic (for Redis and for MariaDB).

  • Configuration Wizard—Added a wizard to help verify your configuration for static content deployment. See Smart wizards.

  • Symfony Console support—Added support for Symfony Console 4 with Magento 2.3.

  • Cron scheduling optimizations—Improved the queue management and enhanced logging to help with debugging cron-related issues.

  • Deployment validation fails if an ADMIN_EMAIL or ADMIN_USERNAME value is the same as an existing Magento administrator account.

  • Removed SOLR support for 2.2.x versions. 2.1.x versions retain the ability to enable SOLR.

  • The first installation of the Staging & Production environments of a PRO project now includes different index prefixes for ElasticSearch to prevent possible conflicts while identifying records belonging to each environment.

  • Fixed an issue that interrupted the build phase for legacy architecture during static content deployment.

  • Cron-specific Improvements—Re-worked the cron implementation:

    • Fixed an issue that caused the cron queue to fill quickly. Now it clears the outdated cron jobs in a more reliable way.

    • Re-organized the sequence of cron jobs so that all jobs in separate threads launch prior to the general group.

    • Improved logging to better assist in debugging cron issues.

    • NOTE—This release addresses many cron-related issues. If you currently use some cron-related patches in m2-hotfixes, remove them.

  • SCD-specific improvements

    • You can use the VERBOSE_COMMANDS and the SCD_COMPRESSION_LEVEL environment variables during both build and deploy phases.

    • Fixed an issue that caused deployment to fail with a random error when encountering an unexpected value for the SCD_COMPRESSION_LEVEL environment variable. Improved the configuration validation to provide meaningful notifications. See SCD_COMPRESSION_LEVEL for acceptable values.

    • Fixed the behavior of the SCD_COMPRESSION_LEVEL environment variable configuration flow so the overrides work as expected.

    • Fixed an issue that prevented the configuration of the SCD_THREADS environment variable in the .magento.env.yaml file deploy stage.

v2002.0.10

  • Static Content Deployment (SCD)—There is a new, alternative deployment process to generate static content when requested (on-demand). This decreases downtime and improves cache handling by generating the most critical assets.

    • New environment variable—Added the SCD_ON_DEMAND global environment variable to generate static content when requested.

    • Post-deploy hook—Added a post_deploy hook for the .magento.app.yaml file that clears the cache and pre-loads (warms) the cache after the container begins accepting connections. It is available only for Pro projects that contain Staging and Production environments in the Project Web UI and for Starter projects. Although not required, this works in tandem with the SCD_ON_DEMAND environment variable.

  • Optimization—Optimized moving or copying files during deployment to improve deployment speed and decrease loads on the file system.

  • Deployment Logging—Added the ability to enable Syslog and Graylog Extended Log Format (GELF) handlers for outputting logs during the deployment process. See Logging handlers.

  • Added the following Environment variables:

    • CRYPT_KEY—Provide a cryptographic key to another environment when moving a database.

    • SKIP_HTML_MINIFICATIONGlobal environment variable that skips copying the static view files in the var/view_preprocessed directory and generates minified HTML when requested.

    • SCD_ON_DEMANDGlobal environment variable to generate static content when requested.

    • WARM_UP_PAGES—You can list the pages to use to pre-load the cache. Available in the new Post-deploy variables.
  • Fixed an issue that involved a locally applied patch breaking the deployment on an instance. Now, ece-tools can detect that a patch has been applied.

  • Fixed a conflict between JavaScript bundling and GZIP functionality. Now these features work correctly together.

  • Fixed an issue that caused ece-tools CLI commands to fail when using earlier PHP 7.0.x versions.

  • Fixed an issue that prevented static content deployment with the compact strategy in multiple threads.

  • Fixed a Redis session locking issue that caused an Admin login delay. Also, the fix is available for 2.1.x.

v2002.0.9

You must upgrade the Magento Commerce Cloud metapackage to get this and all future updates.

  • ece-tools—The ece-tools package now supports Magento 2.1.x.

  • Redis configuration—You can now configure Redis page and default cache and Redis session storage using an environment variable.

  • Search, AMQP, and Redis service improvements—We unified the service configuration flow so that it now behaves the same way for all services. Manually editing the env.php file to configure services is no longer supported. You must use environment variables or the .magento.env.yaml file instead.

  • Environment variables

    • The use of env:STATIC_CONTENT_THREADS was deprecated and will be removed in a future release. Use the STATIC_CONTENT_THREADS environment variable instead.

    • The STATIC_CONTENT_EXCLUDE_THEMES environment variable was deprecated. You must use the SCD_EXCLUDE_THEMES environment variable instead.

  • Logging—We simplified logging around built-in patching operations.

  • We removed developer mode support and the APPLICATION_MODE environment variable because they were causing unexpected behavior.

  • We fixed an issue that was causing static content deployment failures related to Redis. Now, multi-threaded static content deployment runs as designed.

  • We fixed an issue that was preventing users from saving modifications to configuration fields in the Admin, which are marked as sensitive after running the app:config:dump command.

  • We added support for an earlier version of symfony/yaml to fix conflicts with some packages, which are not yet compatible with the latest version.

Older releases

See the release notes archive for version 2002.0.8 and earlier.