Annotations in the Magento Functional Testing Framework

This topic was updated due to the 2.0.2 MFTF release.

Annotations are essentially comments in the code. (In PHP, they all are marked by a preceding @ symbol.) Within test methods, annotations are contained within their own node.

Principles

  • All annotations are within an <annotations> element
  • Each element within corresponds to a supported annotation type
  • There is no distinction made in XML between Codeception annotations and Allure annotations
  • Each annotation contains only one value. If multiple annotation values are supported and required each value requires a separate annotation.

Example

<annotations>

       <features value="Category Creation"/>

       <title value="Create a Category via Admin"/>

       <group value="category"/>

</annotations>

Reference

description

Implementation of an Allure tag @Description

Metadata for report.

Attribute Type Use
value string required

Example

<description value="Add Catalog via Admin"/>

Generated PHP code:

@Description("Add Catalog via Admin")


features

Implementation of an Allure tag @Features

Sets a string that will be displayed as a Feature within the Allure report. Tests under the same feature are grouped together in the report.

Attribute Type Use
value string required

Example

<features value="Catalog"/>
<features value="Add/Edit"/>

Generated PHP code:

@Features({"Catalog", "Add/Edit"})


group

Implementation of a Codeception tag @group

Specifies a string to identify and collect tests together. Any test can be a part of multiple groups. The purpose of grouping is to create a set of test for a purpose (e.g. all Cart tests, all Slow tests) and run them together.

Attribute Type Use
value string required

Example

<group value="catalog"/>

Generated PHP code:

@group catalog


return

Implementation of a Codeception tag @return

Specifies what is returned from a test execution.

Attribute Type Use
value string required

Example

<return value="void"/>

Generated PHP code:

@return void


severity

Implementation of an Allure tag @Severity

Metadata for report.

Attribute Type Use Acceptable values
value string required "BLOCKER", "CRITICAL", "NORMAL", "MINOR", "TRIVIAL"

Example

<severity value="CRITICAL"/>

Generated PHP code:

@Severity(level = SeverityLevel::CRITICAL)


stories

Implementation of an Allure tag @Stories

Same functionality as features, within the Story report group.

Attribute Type Use
value string required

Example

<stories value="Add Catalog"/>
<stories value="Edit Catalog"/>

Generated PHP code:

@Stories({"Add Catalog", "Edit Catalog"})


testCaseId

Implementation of an Allure tag @TestCaseId

Specifies a ZephyrId for a test. If the linkage is set up correctly in Allure config, the test will have a hyperlink to the Zephyr test case in the report.

Lean more about setup instructions in Allure

Attribute Type Use
value string required

Example

<testCaseId value="#"/>

Generated PHP code:

@TestCaseId("#")


useCaseId

Implementation of a custom tag @UseCaseId

Specifies Use Case Id for a test.

Ignored by Allure configuration at the moment, as Allure implementation is not complete.

Attribute Type Use
value string required

Example

<useCaseId value="USECASE-1"/>

Generated PHP code:

@UseCaseId("USECASE-1")


title

Implementation of an Allure tag @Title

Metadata for report.

Attribute Type Use
value string required

Example

<title value="Add Catalog"/>

Generated PHP code:

@Title("Add Catalog")