Getting started with the Magento Functional Testing Framework
This topic was updated after 2.0.2 MFTF release.
Make sure that you’ve set up the following software:
- PHP versions supported by Magento instance under test
- Composer v1.3.x+
- Java v1.8.x+
- Selenium Server Standalone v3.6.0+
- ChromeDriver v2.33+
Make sure that the following settings in Magento are set as described.
A Selenium web driver cannot enter data to fields with WYSIWYG.
This option disables the WYSIWYG and enables the web driver to process these fields as simple text areas.
- Log in to the Magento Admin as an administrator.
- Click Stores > Configuration > General > Content Management > WYSIWYG Options.
- Set Enable WYSIWYG Editor to Disabled Completely.
- Click Save Config.
Enable the Admin Account Sharing setting to avoid unpredictable logout during testing session. And disable the Add Secret Key in URLs setting to open pages using direct URLs.
- Follow Stores > Configuration > Advanced > Admin > Security.
- Set Admin Account Sharing to Yes.
- Set Add Secret Key to URLs to No.
Set up the framework
Follow these steps to set up the MFTF on your system.
Step 1. Clone the magento2 source code repository
$ git clone https://github.com/magento/magento2.git
$ git clone firstname.lastname@example.org:magento/magento2.git
Step 2. Install dependencies
$ cd magento2/dev/tests/acceptance $ composer install
Step 3. Build the project
magento2/dev/tests/acceptance, run the following command:
$ vendor/bin/robo build:project
To avoid typing
vendor/bin every time, add to PATH your
<absolute path to acceptance dir>/vendor/bin value.
When added, you should be able to run commands:
Step 4. Edit environment settings
magento2/dev/tests/acceptance directory, edit the
.env file to match your system.
The following list describes parameters, required to launch tests.
MAGENTO_BASE_URLmust contain a domain name of the Magento instance that will be tested. Example:
MAGENTO_BACKEND_NAMEmust contain a relative pass of the Admin area. Example:
MAGENTO_ADMIN_USERNAMEmust contain a user name required for authorization in the Admin area. Example:
MAGENTO_ADMIN_PASSWORDmust contain a user password required for authorization in the Admin area. Example:
The following self-descriptive variables have default values (included).
SELENIUM_HOST=127.0.0.1 SELENIUM_PORT=4444 SELENIUM_PROTOCOL=http SELENIUM_PATH=/wd/hub
Only change or specify
SELENIUM_* values if you are not running Selenium locally, or if you have changed your Selenium Server configuration.
They come together to form the path to where Selenium Server is running from like:
Step 5. Make
command.php visible in the Magento testing environment
In your Magento installation, navigate to the
magento2/dev/tests/acceptance directory and run the following command to allow MFTF to send Magento CLI commands to your Magento instance.
$ cp .htaccess.sample .htaccess
If you do not have access to your Magento installation and cannot perform the above, tests that run Magento CLI commands will not be able to run.
Step 6. Generate existing tests
magento2/dev/tests/acceptance directory, run the following command to generate tests as PHP classes from XML files:
$ vendor/bin/robo generate:tests
Step 7. Run tests
Run the Selenium server
Download a Selenium web driver for your web browser into the same directory where the Selenium server is located.
Add the directory with the web driver to PATH.
Run the Selenium server in the terminal:
$ java -jar <path_to_selenium_directory>/selenium-server-standalone-<version>.jar
Run all tests
$ vendor/bin/codecept run
Step 7. Generate reports
Install Allure, a tool that generates testing reports in HTML. Testing reports are generated in CLI during testing.
If you want to see the reports in GUI, run:
$ vendor/bin/robo allure2:report