Merging in the Magento Functional Testing Framework

This topic was updated due to the 2.0.2 MFTF release.

The MFTF allows you to merge test components defined in XML files such as Tests, Pages, Sections, and Data. You can create, delete, or update the component. It is useful for supporting rapid test creation for extensions and customizations.

You can specify changes needed to an existing file and have that merged. This produces a modification of the original that incorporates the specified changes (the ‘delta’).

General

Merging operates on XML tag level. It is triggered by our parser when there are two (or more) entities with the same name. Therefore, your update (XML node with changes) must have the same attribute name as its base node (target object to be changed).

For example:

  • all tests with <test name="SampleTest> will be merged into one
  • all pages with <page name="SamplePage> will be merged into one
  • all sections with <section name="SampleAction"> will be merged into one
  • all data entities with <entity name="sampleData" type="sample"> will be merged into one

Although a file name doesn’t influence merging, we recommend to use same file names in merging updates, for convenience of search.

Merging in Tests

Add a test

There is no way to add another <test> using merging functionality. Adding of a <test> is done by creating a new *Test.xml file, or adding a <test> node to an existing *Test.xml file.

Remove a test

Tests cannot be removed while merging.

If a test must be skipped due to a module completely invalidating a functionality, you can add the test to the skip group.

Learn more about running tests with different options using robo or codecept commands.

Example:

Skip the AdminLoginTest test in .../Backend/Test/AdminLoginTest.xml while merging with .../Foo/Test/AdminLoginTest.xml:

.../Backend/Test/AdminLoginTest.xml code:

<tests ...>
    <test name="AdminLoginTest">
            <annotations>
                <features value="Admin Login"/>
                <stories value="Login on the Admin Login page"/>
                <title value="You should be able to log into the Magento Admin backend."/>
                <description value="You should be able to log into the Magento Admin backend."/>
                <severity value="CRITICAL"/>
                <testCaseId value="MAGETWO-71572"/>
                <group value="example"/>
                <group value="login"/>
            </annotations>
        <amOnPage url="" stepKey="amOnAdminLoginPage"/>
        <fillField selector="" userInput="" stepKey="fillUsername"/>
        <fillField selector="" userInput="" stepKey="fillPassword"/>
        <click selector="" stepKey="clickOnSignIn"/>
        <closeAdminNotification stepKey="closeAdminNotification"/>
        <seeInCurrentUrl url="" stepKey="seeAdminLoginUrl"/>
    </test>
</tests>

Create .../Foo/Test/AdminLoginTest.xml:

<tests ...>
    <test name="AdminLoginTest">
            <annotations>
                <group value="skip"/>
            </annotations>
    </test>
</tests>

The resulted AdminLoginTest will correspond to the following code:

<test name="AdminLoginTest">
        <annotations>
            <features value="Admin Login"/>
            <stories value="Login on the Admin Login page"/>
            <title value="You should be able to log into the Magento Admin backend."/>
            <description value="You should be able to log into the Magento Admin backend."/>
            <severity value="CRITICAL"/>
            <testCaseId value="MAGETWO-71572"/>
            <group value="example"/>
            <group value="login"/>
            <group value="skip"/>
        </annotations>
    <amOnPage url="" stepKey="amOnAdminLoginPage"/>
    <fillField selector="" userInput="" stepKey="fillUsername"/>
    <fillField selector="" userInput="" stepKey="fillPassword"/>
    <click selector="" stepKey="clickOnSignIn"/>
    <closeAdminNotification stepKey="closeAdminNotification"/>
    <seeInCurrentUrl url="" stepKey="seeAdminLoginUrl"/>
</test>

Update a test

Any change must include information about where it should be inserted relative to other test steps in scope of test.

This is done using the before or after element. The action can only specify either a before or after correspondingly. See the examples given above.

Add a test step

Use case: Add checkOption before click (stepKey="clickLogin") and seeInCurrentUrl after the click in the LogInAsAdminTest test (in .../Backend/Test/LogInAsAdminTest.xml) while merging with .../Foo/Test/LogInAsAdminTest.xml

.../Backend/Test/LogInAsAdminTest.xml code:

<tests ...>
    <test name="LogInAsAdminTest">
        <amOnPage url="" stepKey="navigateToAdmin"/>
        <fillField selector="" userInput="admin" stepKey="fillUsername"/>
        <fillField selector="" userInput="password" stepKey="fillPassword"/>
        <click selector="" stepKey="clickLogin"/>
        <see userInput="Lifetime Sales" stepKey="seeLifetimeSales"/>
    </test>
</tests>

Create .../Foo/Test/LogInAsAdminTest.xml:

<tests ...>
    <test name="LogInAsAdminTest">
        <checkOption selector="" stepKey="checkRememberMe" before="clickLogin"/>
        <seeInCurrentUrl url="admin/admin/dashboard/" stepKey="seeAdminUrl" after="clickLogin"/>
    </test>
</tests>

The resulted LogInAsAdminTest will correspond to the following code:

<test name="LogInAsAdminTest">
    <amOnPage url="" stepKey="navigateToAdmin"/>
    <fillField selector="" userInput="admin" stepKey="fillUsername"/>
    <fillField selector="" userInput="password" stepKey="fillPassword"/>
    <checkOption selector="" stepKey="checkRememberMe"/>
    <click selector="" stepKey="clickLogin"/>
    <seeInCurrentUrl url="admin/admin/dashboard/" stepKey="seeAdminUrl"/>
    <see userInput="Lifetime Sales" stepKey="seeLifetimeSales"/>
</test>

Remove a test step

Use case: Remove see (stepKey="seeLifetimeSales") from the LogInAsAdminTest test (in .../Backend/Test/LogInAsAdminTest.xml) while merging with .../Foo/Test/LogInAsAdminTest.xml

.../Backend/Test/LogInAsAdminTest.xml code:

<tests ...>
    <test name="LogInAsAdminTest">
        <amOnPage url="" stepKey="navigateToAdmin"/>
        <fillField selector="" userInput="admin" stepKey="fillUsername"/>
        <fillField selector="" userInput="password" stepKey="fillPassword"/>
        <click selector="" stepKey="clickLogin"/>
        <see userInput="Lifetime Sales" stepKey="seeLifetimeSales"/>
    </test>
</tests>

Create .../Foo/Test/LogInAsAdminTest.xml:

<tests ...>
    <test name="LogInAsAdminTest">
        <remove keyForRemoval="seeLifetimeSales"/>
    </test>
</tests>

The resulted LogInAsAdminTest will correspond to the following code:

<test name="LogInAsAdminTest">
    <amOnPage url="" stepKey="navigateToAdmin"/>
    <fillField selector="" userInput="admin" stepKey="fillUsername"/>
    <fillField selector="" userInput="password" stepKey="fillPassword"/>
    <click selector="" stepKey="clickLogin"/>
</test>

Update a test step

Use case: Change selector in fillField (stepKey="fillPassword") of the LogInAsAdminTest test (in .../Backend/Test/LogInAsAdminTest.xml) while merging with .../Foo/Test/LogInAsAdminTest.xml

<tests ...>
    <test name="LogInAsAdminTest">
        <amOnPage url="" stepKey="navigateToAdmin"/>
        <fillField selector="" userInput="admin" stepKey="fillUsername"/>
        <fillField selector="" userInput="password" stepKey="fillPassword"/>
        <click selector="" stepKey="clickLogin"/>
        <see userInput="Lifetime Sales" stepKey="seeLifetimeSales"/>
    </test>
</tests>

Create .../Foo/Test/LogInAsAdminTest.xml:

<tests ...>
    <test name="LogInAsAdminTest">
        <fillField selector="" userInput="password" stepKey="fillPassword"/>
    </test>
</tests>

The resulted LogInAsAdminTest will correspond to the following code:

<test name="LogInAsAdminTest">
    <amOnPage url="" stepKey="navigateToAdmin"/>
    <fillField selector="" userInput="admin" stepKey="fillUsername"/>
    <fillField selector="" userInput="password" stepKey="fillPassword"/>
    <click selector="" stepKey="clickLogin"/>
    <see userInput="Lifetime Sales" stepKey="seeLifetimeSales"/>
</test>

Pages merging

Using merging in pages, you can add or remove sections defining updates in your module. To merge pages, the page name must be the same as in base module, whish is set in the module attribute.

Add a section

Use case: The FooBackend module extends the Backend module and requires use of two new sections that must be covered with tests. Add BaseBackendSection and AnotherBackendSection to the BaseBackendPage.

.../Backend/Page/BaseBackendPage.xml code:

<pages ...>
    <page name="BaseBackendPage" url="admin" area="admin" module="Magento_Backend">
        <section name="BaseBackendSection"/>
        <section name="AnotherBackendSection"/>
    </page>
</pages>

Create .../FooBackend/Page/BaseBackendPage.xml:

<pages ...>
    <page name="BaseBackendPage" url="admin" area="admin" module="Magento_Backend">
        <section name="NewExtensionSection"/>
    </page>
</pages>

The resulted BaseBackendPage will correspond to the following code:

<page name="BaseBackendPage" url="admin" area="admin" module="Magento_Backend">
    <section name="BaseBackendSection"/>    
    <section name="AnotherBackendSection"/>
    <section name="NewExtensionSection"/>
</page>

Remove a section

Use case: The FooBackend module extends the Backend module and requires deletion of AnotherBackendSection section.

.../Backend/Page/BaseBackendPage.xml code:

<page name="BaseBackendPage" url="admin" area="admin" module="Magento_Backend">
    <section name="BaseBackendSection"/>    
    <section name="AnotherBackendSection"/>
</page>

Create .../FooBackend/Page/BaseBackendPage.xml:

<page name="BaseBackendPage" url="admin" area="admin" module="Magento_Backend">
    <section name="AnotherBackendSection" remove="true"/>
</page>

The resulted BaseBackendPage will correspond to the following code:

<page name="BaseBackendPage" url="admin" area="admin" module="Magento_Backend">
    <section name="BaseBackendSection"/>
</page>

Sections merging

If you need to add, remove, or update elements in sections, you can use merging functionality. All sections with the same file name, section name, and element name are merged during test generation.

Add an element

Use case: The FooBackend module extends the Backend module and requires a new element mergeElement in the AdminLoginFormSection. Add mergeElement to the AdminLoginFormSection.

.../Backend/Section/AdminLoginFormSection.xml code:

<sections ...>
    <section name="AdminLoginFormSection">
        <element name="username" type="input" selector="#username"/>
        <element name="password" type="input" selector="#login"/>
        <element name="signIn" type="button" selector=".actions .action-primary" timeout="30"/>
    </section>
</sections>

Create .../FooBackend/Section/AdminLoginFormSection.xml:

<sections ...>
    <section name="AdminLoginFormSection">
        <element name="mergeElement" type="input" selector="#selector"/>
    </section>
</sections>

The resulted AdminLoginFormSection will correspond to the following code:

<section name="AdminLoginFormSection">
    <element name="username" type="input" selector="#username"/>
    <element name="password" type="input" selector="#login"/>
    <element name="signIn" type="button" selector=".actions .action-primary" timeout="30"/>
    <element name="mergeElement" type="input" selector="#selector"/>
</section>

Remove an element

Use case: The FooBackend module extends the Backend module and requires deletion of username in the AdminLoginFormSection. Remove username from the AdminLoginFormSection.

.../Backend/Section/AdminLoginFormSection.xml code:

<sections ...>
    <section name="AdminLoginFormSection">
        <element name="username" type="input" selector="#username"/>
        <element name="password" type="input" selector="#login"/>
        <element name="signIn" type="button" selector=".actions .action-primary" timeout="30"/>
    </section>
</sections>

Create .../FooBackend/Section/AdminLoginFormSection.xml:

<sections ...>
    <section name="AdminLoginFormSection">
        <element name="username" type="input" remove="true"/>
    </section>
</sections>

The resulted AdminLoginFormSection will correspond to the following code:

<section name="AdminLoginFormSection">
    <element name="password" type="input" selector="#login"/>
    <element name="signIn" type="button" selector=".actions .action-primary" timeout="30"/>
</section>

Update an element

Use case: The FooBackend module extends the Backend module and requires change of selector in username in the AdminLoginFormSection. Update username in the AdminLoginFormSection.

.../Backend/Section/AdminLoginFormSection.xml code:

<sections ...>
    <section name="AdminLoginFormSection">
        <element name="username" type="input" selector="#username"/>
        <element name="password" type="input" selector="#login"/>
        <element name="signIn" type="button" selector=".actions .action-primary" timeout="30"/>
    </section>
</sections>

Create .../FooBackend/Section/AdminLoginFormSection.xml:

<sections ...>
    <section name="AdminLoginFormSection">
        <element name="username" type="input" selector="#newSelector"/>
    </section>
</sections>

The resulted AdminLoginFormSection will correspond to the following code:

<section name="AdminLoginFormSection">
    <element name="username" type="input" selector="#newSelector"/>
    <element name="password" type="input" selector="#login"/>
    <element name="signIn" type="button" selector=".actions .action-primary" timeout="30"/>
</section>

Data merging

You can add or update <data> elements within an [entity]. Removal of individual <data> tags is not supported. Entities and data with the same file name, entity name and type, and data key are merged during test generation.

Add data

Use case: The FooSample module extends the Sample module and requires a new data item thirdField in the _defaultSample entity. Add <data key="thirdField">field3</data> to the _defaultSample.

.../Sample/Data/SampleData.xml code:

<entities ...>
    <entity name="_defaultSample" type="testData">
        <data key="firstField">field1</data>
        <data key="secondField">field2</data>
    </entity>
</entities>

Create .../FooSample/Data/SampleData.xml:

<entities ...>
    <entity name="sampleData" type="testData">
        <data key="thirdField">field3</data>
    </entity>
</entities>

The resulted _defaultSample will correspond to the following code:

<entity name="_defaultSample" type="testData">
    <data key="firstField">field1</data>
    <data key="secondField">field2</data>
    <data key="thirdField">field3</data>
</entity>

Update data

Use case: The FooSample module extends the Sample module and requires change of data item firstField in the _defaultSample entity. Change firstField to <data key="firstField">overrideField</data> in the _defaultSample.

.../Sample/Data/SampleData.xml code:

<entities ...>
    <entity name="_defaultSample" type="testData">
        <data key="firstField">field1</data>
        <data key="secondField">field2</data>
    </entity>
</entity>

Create .../FooSample/Data/SampleData.xml:

<entities ...>
    <entity name="_defaultSample" type="testData">
        <data key="firstField">overrideField</data>
    </entity>
</entity>

The resulted _defaultSample will correspond to the following code:

<entity name="_defaultSample" type="testData">
    <data key="firstField">overrideField</data>
    <data key="secondField">field2</data>
</entity>