Update the Magento Functional Testing Framework

This topic was updated due to the 2.3.0 MFTF release.

Find out your version of the MFTF. If your version is 2.2, see the MFTF 2.2 documentation.

Magento tests and the framework are stored in different repositories.

Magento tests are stored in the same repository as the Magento code base. When you pull changes in the Magento code, you’re potentially pulling corresponding tests as well.

The MFTF is installed separately as a dependency using Composer. When pulling the latest Magento code, update the corresponding Composer dependencies in the magento2 root directory. This ensures that the MFTF is up to date.

Update the MFTF from 2.3.x

To update the MFTF to the latest patch:

  1. Verify that the Magento WYSIWYG settings and Security settings are set appropriately.
  2. Check details about backward incompatible changes in the Changelog and update your new or customized tests.
  3. Get the latest framework version using Composer:

     composer update
    
  4. Generate the updated tests:

     vendor/bin/robo generate:tests
    

Update the MFTF from 2.2

To update the MFTF from the previous minor version:

  1. When you update Magento, verify that the Magento WYSIWYG settings and Security settings are set appropriately.
  2. Starting at the magento2/ root directory remove the vendor/ directory:
    rm -rf vendor/
    
  3. Get the latest framework version from the Composer dependencies:
    composer install
    
  4. Remove the vendor/ directory at magento2/dev/tests/acceptance/:
    rm -rf dev/tests/acceptance/vendor/
    
  5. Install dependencies at that level:
    composer install -d dev/tests/acceptance/
    
  6. Run the upgrade:tests using the new command line tool:
    vendor/bin/mftf upgrade:tests app
    
  7. If you are using Phpstorm, update the urn catalog:
    vendor/bin/mftf generate:urn-catalog .idea/
    
  8. Update your own tests, including data, metadata, and so on, if they contain tags that are unsupported in the newer version. Check details about backward incompatible changes and review new MFTF release documentation in the Changelog.
  9. Generate newly pulled tests:
    vendor/bin/mftf generate:tests