GitSwarm-EE 2017.2-1 Documentation

Running Composer and NPM scripts with deployment via SCP

This guide covers the building dependencies of a PHP project while compiling assets via an NPM script.

While is possible to create your own image with custom PHP and Node JS versions, for brevity, we will use an existing Docker image that contains both PHP and NodeJS installed.

image: tetraweb/php

The next step is to install zip/unzip packages and make composer available. We will place these in the before_script section:

before_script:
  - apt-get update
  - apt-get install zip unzip
  - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
  - php composer-setup.php
  - php -r "unlink('composer-setup.php');"

This will make sure we have all requirements ready. Next, we want to run composer update to fetch all PHP dependencies and npm install to load node packages, then run the npm script. We need to append them into before_script section:

before_script:
  # ...
  - php composer.phar update
  - npm install
  - npm run deploy

In this particular case, the npm deploy script is a Gulp script that does the following:

  1. Compile CSS & JS
  2. Create sprites
  3. Copy various assets (images, fonts) around
  4. Replace some strings

All these operations will put all files into a build folder, which is ready to be deployed to a live server.

How to transfer files to a live server?

You have multiple options: rsync, scp, sftp and so on. For now, we will use scp.

To make this work, you need to add a GitLab Secret Variable (accessible on gitlab.example/your-project-name/variables). That variable will be called STAGING_PRIVATE_KEY and it's the private ssh key of your server.

Security tip

Create a user that has access only to the folder that needs to be updated!

After you create that variable, you need to make sure that key will be added to the docker container on run:

before_script:
  # - ....
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  - mkdir -p ~/.ssh
  - eval $(ssh-agent -s)
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

In order, this means that:

  1. We check if the ssh-agent is available and we install it if it's not;
  2. We create the ~/.ssh folder;
  3. We make sure we're running bash;
  4. We disable host checking (we don't ask for user accept when we first connect to a server; and since every job will equal a first connect, we kind of need this)

And this is basically all you need in the before_script section.

How to deploy things?

As we stated above, we need to deploy the build folder from the docker image to our server. To do so, we create a new job:

stage_deploy:
  artifacts:
    paths:
      - build/
  only:
    - dev
  script:
    - ssh-add <(echo "$STAGING_PRIVATE_KEY")
    - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp"
    - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/_tmp
    - ssh -p22 server_user@server_host "mv htdocs/wp-content/themes/live htdocs/wp-content/themes/_old && mv htdocs/wp-content/themes/_tmp htdocs/wp-content/themes/live"
    - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old"

What's going on here?

  1. only:dev means that this build will run only when something is pushed to the dev branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want)
  2. ssh-add ... we will add that private key you added on the web UI to the docker container
  3. We will connect via ssh and create a new _tmp folder
  4. We will connect via scp and upload the build folder (which was generated by a npm script) to our previously created _tmp folder
  5. We will connect again to ssh and move the live folder to an _old folder, then move _tmp to live.
  6. We connect to ssh and remove the _old folder

What's the deal with the artifacts? We just tell GitLab CI to keep the build directory (later on, you can download that as needed).

Why we do it this way?

If you're using this only for stage server, you could do this in two steps:

- ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/live/*"
- scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/live

The problem is that there will be a small period of time when you won't have the app on your server.

So we use so many steps because we want to make sure that at any given time we have a functional app in place.

Where to go next?

Since this was a WordPress project, I gave real life code snippets. Some ideas you can pursuit:

Our final .gitlab-ci.yml will look like this:

image: tetraweb/php

before_script:
  - apt-get update
  - apt-get install zip unzip
  - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
  - php composer-setup.php
  - php -r "unlink('composer-setup.php');"
  - php composer.phar update
  - npm install
  - npm run deploy
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  - mkdir -p ~/.ssh
  - eval $(ssh-agent -s)
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

stage_deploy:
  artifacts:
    paths:
    - build/
  only:
    - dev
  script:
    - ssh-add <(echo "$STAGING_PRIVATE_KEY")
    - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp"
    - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/_tmp
    - ssh -p22 server_user@server_host "mv htdocs/wp-content/themes/live htdocs/wp-content/themes/_old && mv htdocs/wp-content/themes/_tmp htdocs/wp-content/themes/live"
    - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old"