Pro develop and deploy workflow

Everything in Magento Commerce (Cloud) is Git-driven. Your project is a Master Git branch cloned from a Magento 2 repository. Every active Git branch has an associated full environment. Depending on your Magento Commerce (Cloud) plan subscription, your deployment workflow may differ.

The general workflow for all development and deployment includes:

  • Push code to the remote Git branch
  • Build and deploy processes run
  • The environments updated with code, services, and configurations

The Pro plan gives you a large Integration environment for your development across eight active branches, a Staging environment, and a Production environment. For full details, see Pro architecture.

The following figure shows how it works at a high level:

High-level view of Pro architecture flow

You can manage all of Integration environments directly through the Project Web Interface. Access and manage all Integration, Staging, and Production environments through the store and Admin panel using provided URLs and SSH and CLI commands. For Integration environments, you can also use Magento Cloud command-line.

For existing Pro projects, you need to have your Project Web Interface updated to manage Staging and Production through the interface. For more information adding this management to existing Pro projects, see Add Staging and Production to Pro projects UI.

If you do not request this update, you must use CLI commands or tickets to modify settings, variables, routes, and more for Pro plan Staging and Production environments.

Pro environments and branches

For your environments, we recommend deploying and testing following a Development > Staging > Production workflow. The Integration environment acts as your extensive testing area for custom code, extensions, and 3rd party integrations. Deploying to Staging gives you Production features and additional services including Fastly in a safe environment for testing. Integration and Staging environments are only accessible by user accounts with strict access via SSH and URLs. These environments are not public facing. Finally, Production is your live, public environment.

For your branches, you can follow any methodology. One example follows an agile methodology such as scrum to create branches for every sprint.

From each sprint, you can have branches for every user story. All the stories become testable. You can continually merge to the sprint branch and validate that on a continuous basis. When the sprint ends, there is no testing bottleneck, and you can just merge to master and put the whole sprint into production.

For detailed information, see Pro architecture.

Development workflow

Development and deployment on Pro plans begins with your initial project. You create your project with the “blank site”, which is a Magento Commerce (Cloud) template code repo with a fully prepared store. This creates a master branch of Git code in your Integration environment.

The full process involves:

Develop and deploy workflow

You also have a few optional steps to help develop and test your code and store data:

This process assumes you would have your local developer workspace set up. Feel free to read over this process even if your local isn’t ready.

The following information assumes you have provisioned Staging and Production environments.

Clone and branch

When you created your project, a master branch was cloned using the Magento Commerce (Cloud) Git repository. To start branching and working with code, you will need to clone the master to your local.

The format of the Git clone command is:

git fetch origin
git pull origin <environment ID>

You can create a developer branch from master, which is an active branch in Integration. Anytime you need to develop custom code, add extensions, integrate with a 3rd party service, work in a develop branch. You will have up to seven active Git branches available. When you create and push your branch, a full environment is automatically available to test your code.

When you set up your local developer environment, we walk-through the software and tools you should install, including Git and the Magento Cloud CLI (a bit more robust than Git).

The format of the Git branch command is:

git checkout <branch name>

The format of the Magento Cloud CLI branch command is:

magento-cloud environment:branch <environment name> <parent environment ID>

Branch from Master

Develop code

It’s the time you have been waiting for…writing code. Using this base branch of Magento Commerce (Cloud) code, you can start installing extensions, configuring extension settings and your store options, creating multi-sites and stores, adding themes, and much more.

We recommend using a branching strategy with your development work. Using one branch to do all of your work all at once might make testing difficult. For example, you could follow continuous integration and sprint methodologies to work:

  • Add a few extensions and configure them with your first branch
  • Push this code, test, and merge to Staging then Production
  • Fully configure your services in services.yaml and add a theme
  • Push this code, test, and merge to Staging then Production
  • Integrate with a 3rd party service
  • Push this code, test, and merge to Staging then Production

Do not complete any configurations on your local yet.

And so on until you have your store fully built, configured, and ready to go live. But keep reading, we have even better options for your store and code configuration!

Develop code and push to deploy

Configure store

When you are ready to configure your store, have all code pushed to your Integration environment and access the Magento Admin. You should fully configure all store settings in the Integration environment Admin, not on your local. If you need the URL, see the Project Web Interface. The Store Admin URL is located on the branch page.

For the best information on configurations, we recommend reviewing Magento Commerce and your extension documentation. Here are some links and ideas to help you get kickstarted:

  • Basic configuration for store admin access, name, languages, currencies, branding, sites, store views and more
  • Theme for your look and feel of the site and stores including CSS and layouts
  • System configuration for roles, tools, notifications, and your encryption key for your database
  • Extension settings using their documentation

Beyond just store settings, you can further configure multiple sites and stores, configured services, and more. For details, see Configure Magento Commerce.

Now you need to get these settings into your code. We have a helpful command to do this, keep reading.

Generate configuration management files

If you are familiar with Magento, you may be concerned about how to get your configuration settings from your database in development to Staging and Production. Previously, you had to copy down on paper or Excel all of your configuration settings to enter them manually in another environment. Or you may have dumped your database and push that data to another environment.

Magento Commerce (Cloud) provides a set of two Configuration Management commands that export configuration settings from your environment into a file. These commands are only available for Magento Commerce (Cloud) 2.1.4 and later (not 2.2).

  • php bin/magento magento-cloud:scd-dump: Recommended. Exports only the configuration settings you have entered or modified from defaults into a configuration file.
  • php bin/magento app:config:dump: Exports every configuration setting, including modified and default, into a configuration file.

The generated file is located in app/etc/:

  • For 2.1.4 and later: config.local.php
  • For 2.2 and later: config.php

You will generate the file in the Integration environment where you configured Magento. We walk you through the process of generating the file, adding it to your Git branch, and deploying it.

Important notes on Configuration Management:

  • Any configuration setting included in the file is locked from editing, or read-only, in the deployed environment. This is one reason we recommend using scd-dump.

    For example, we will have you install a module for Fastly in your development environment. You will only configure this module in Staging and Production. Using scd-dump keeps those default fields editable.

  • This file can be long depending on the size of your deployment. The scd-dump command generates a far small file than app:config:dump.

Generate configuration management file

An additional feature of this command is part of Magento Commerce (Cloud) 2.2. Any values determined to be sensitive data, like sandbox credentials for a PayPal module, will be generated into another configuration file called env.php in app/etc/. This file remains in the exact environment it is created without traveling with your code. You will not add this file to your code repository. You can also create environment variables with CLI commands in all Magento Commerce (Cloud) versions.

Environment variables generate

For more information, see Configuration Management.

Push code and test

At this point, you should have a developed code branch with a configuration file (config.local.php or config.php) ready to test.

Everytime you push code from your local environment, a series of build and deploy scripts run. These scripts generate new Magento code and deploy it to the remote environment. For example, if you are pushing a development branch from your local to the remote Git branch, a matching environment updates services, code, and static content.

You can directly access this environment with a store URL, Magento Admin URL, and SSH to enter any needed CLI commands. These environments include a web server, database, and configured services. When ready, you can start deploying and testing in Staging.

For more information, see Deployment workflow.

Optional: Install sample data

If you need some example data when developing your store, you can install our sample data. This data simulates an active Magento store, including customers, products, and other data. This sample data works best with a “blank site” Magento Commerce (Cloud) template installation when creating your project.

We recommend installing sample data in your local Integration branches and environments. If you use this data in Staging or Production, make sure to clear out the information and products before going live.

For instructions, see Install optional sample data.

Install optional sample data

Optional: Pull production data

We recommend adding all of your products, catalogs, site content, and so on (not configurations) directly in Production. By adding this data in Production, you immediately update prices, coupons, inventory stock, strategize your sales and future offerings, and much more for your customers. This data does not include extension configurations. You will set those in your development branch on your local.

As you develop features, add extensions, and design themes, having real data to work with is helpful. At any time, you can create a database dump from Production and push that to your Staging environment, possibly Integration environments as you like.

To help export Production data as test data to use in Staging and Integration environments, Run the support utilities:

  • CLI commands (Recommended) to export a protected backup of customer and store data using your Magento encryption key
  • Data Collection tool for generating and exporting data

To migrate this data, see Migrate and deploy static files and data.

Pull and sanitize production data

Prior to pushing the data to another environment, you should consider sanitizing your data. You have a couple of options including using support utilities or developing a script to scrub out customer data.

Important: We don’t recommend pushing a database from the Integration or Staging environment to Production. This data will overwrite your Production live data including sales, orders, new and updated customers, and much more.

Deployment workflow

As we detailed in the architecture information, Magento Commerce (Cloud) is Git driven. Deploying Magento Commerce (Cloud) is part of your Git push processes for branches.

When you push branched code from your local to the remote Git branch, a series of build and deploy scripts begin.

Build scripts:

  • Your site on the target environment continue running during a build
  • Check and run Magento Commerce (Cloud) patches and hotfixes
  • Compile your code with a build and deploy log
  • Check for Configuration Management, if there static content deploy occurs during this phase
  • Create or use a slug of unchanged code to speed up the process
  • Provision all backend services and applications

Deploy scripts:

  • Puts your site on the target environment in Maintenance mode
  • Deploys static content if not completed during Build
  • Installs or updates Magento Commerce (Cloud)
  • Configure routing for traffic

When fully completed, your store comes back online, live, with all of your updated code and configurations.

To learn more, see Deployment process.

Push to Staging and test

You should always push all of your code in iterations to your Staging environment for full testing. The first time you use this environment, you will need to configure a few services including Fastly, Blackfire Profiler, and New Relic APM. We also recommend configuring payment gateways, shipping, notifications, and other vital services with sandbox or testing credentials.

Staging is a pre-production environment, providing all services and settings as close to Production as possible. Thoroughly test every service, verify your performance testing tools, perform UAT testing as an administrator and customers, until you feel your store is ready for Production.

To learn more, see Deploy your store.

Push to Master / Production

When ready to start launching or to push iterations of code live, push to Production. Treat configuration and testing of Production much as your Staging environment. The important difference in this environment is using live credentials. The moment you go live and launch, customers must be able to complete purchases and administrators should be able to manage your live store.

To learn more, see Deploy your store.

Go live

We provide a clear walk-through for going live and launching. It requires more steps than pressing a button. But when complete, your store can serve up products in your customized theme for sale immediately.

To learn more, check out Go live and launch.

Continuous integration

Following your branching and development methodologies, you can easily develop new features, configure changes, and add extensions to continuously develop and deploy updates.

Magento Commerce (Cloud) environments support continous integration for constant updates. This workflow supports releases multiple times a day or on a set schedule according to your business needs.

  • Create development branches with future features and changes
  • Test the code in your development environments
  • Deploy and test in Staging
  • Deploy to Production

For more information, see Continuous integration.