Introduction to the Magento Functional Testing Framework version 2.3

The latest MFTF release is 2.3.11.

Find out your version of the MFTF. If your version is 2.2, see the MFTF 2.2 documentation.

The Magento Functional Testing Framework (MFTF) aims to replace the Functional Testing Framework in future releases. MFTF improves:

  • Traceability for clear logging and reporting capabilities.
  • Modularity to run tests based on installed modules and extensions.
  • Customizability for existing tests.
  • Readability using clear and declarative XML test steps.
  • Maintainability based on simple test creation and overall structure.

Because MFTF tests are written in XML, you no longer need to learn PHP to write tests.

We are actively developing functional tests. Refer to <magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/ for examples.

Audience

This MFTF guide is intended for Magento developers and software engineers, such as QA specialists, PHP developers, and system integrators.

Goals

The purpose of MFTF is to:

  • Facilitate functional testing and minimize the effort it takes to perform regression testing.
  • Make it easier to support the extension and customization of tests via XML merging.

Scope

MFTF will enable you to:

  • Test user interactions with web applications in testing.
  • Write functional tests located in <magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/.
  • Cover basic functionality using out-of-the-box tests. You can test extended functionality using custom tests.
  • Automate regression testing.

Use cases

As a Magento developer, test changes, such as extended search functionality, a new form attribute, or new product tags.

As a software engineer, perform regression testing before release to ensure that Magento works as expected with new functionality.

Find your MFTF version

There are two options to find out your MFTF version:

  • using the MFTF CLI
  • using the Composer CLI

MFTF CLI

cd <magento_root>/
vendor/bin/mftf --version

Composer CLI

cd <magento_root>/
composer show magento/magento2-functional-testing-framework

Contents of dev/tests/acceptance

tests
    _data                       // Additional files required for tests (e.g. pictures, CSV files for import/export, etc.)
    _output                     // The directory is generated during test run. It contains testing reports.
    _suite                      // Test suites.
    _bootstrap.php              // The script that executes essential initialization routines.
    functional.suite.dist.yml   // The Codeception functional test suite configuration (generated while running 'bin/mftf build:project')
utils                           // The test-running utilities.
.env.example                    // Example file for environmental settings.
.credentials.example            // Example file for credentials to be used by the third party integrations (generated while running 'bin/mftf build:project'; should be filled with the appropriate credentials in the corresponding sandboxes).
.gitignore                      // List of files ignored by git.
.htaccess.sample                // Access settings for the Apache web server to perform the Magento CLI commands.
RoboFile.php                    // TO BE DEPRECATED SINCE MFTF 3.0. The MFTF CLI commands configuration for Robo task runner.
codeception.dist.yml            // Codeception configuration (generated while running 'bin/mftf build:project')

MFTF output

  • Generated PHP Codeception tests
  • Codeception results and console logs
  • Screenshots and HTML failure report
  • Allure formatted XML results
  • Allure report dashboard of results

MFTF tests

The MFTF supports two different locations for storing the tests and test artifacts:

  • <magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/ is the directory to create new tests.
  • <magento_root>/vendor/<vendor_name>/<module_name>/Test/Mftf/ is the directory with the out of the box tests (fetched by the Composer).

All tests and test data from these locations are merged in the order indicated in the above list.

The file structure under the both path cases is the same:

<Path>
├── ActionGroup
│   └── ...
├── Data
│   └── ...
├── Metadata
│   └── ...
├── Page
│   └── ...
├── Section
│   └── ...
└── Test
    └── ...

MFTF on Github

Follow the MFTF project and contribute on Github.