London, UK 
6 Snow Hill, London, EC1A 2AY, UK

+442081331119

The Hague, The Netherlands 
WTC The Hague, 
Prinses Beatrixlaan 582 
2595 BM Den Haag

+31 (0) 70 240 0021

Sofia, Bulgaria 
141 Tsarigradsko Shose Blvd, VIP Security Building, Floor 2, Sofia 1784, Bulgaria

  • YouTube
  • Black Facebook Icon
  • Black Twitter Icon

© 2019 HeleCloud™

How to synchronise AWS ECS Services

November 28, 2019

 

Author: Stoyan Gramatikov, AWS Cloud Infrastructure Engineer

 

In this blog post, I’ll outline how to implement a custom AWS ECS solution. One that allows you to overcome a common challenge often experienced by businesses on AWS: not having a built-in synchronisation mechanism that can implement dependencies between AWS ECS services. 

 

Amazon Elastic Container Service (ECS) is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS.

 

As an AWS Advanced Consulting Partner, HeleCloud works with its customers to overcome traditional business barriers through the adoption of cloud services and solutions. Recently, we helped a large electronic financial markets firm to deliver the key components of its secure and robust trade automation platform. 

 

To do this, we [HeleCloud architecture team] selected AWS ECS as it allowed us to manage containers and allowed developers to run applications in the Cloud without having to configure an environment for the code to run it. Despite using a microservices architecture, which was implemented using Docker containers and run on Kubernetes, there remained a challenge over the synchronisation of dependencies.

 

In (and out of) sync 

Most of the applications developed for the customer, required interactions between different services that were dependent on each other. These dependencies required starting services in a well-known order, specific for each application. Whilst AWS ECS had instruments to synchronise dependencies between containers within a service, using the keyword “dependsOn” within task definitions, total synchronisation between different services remained a serious challenge.

 

In order to create a successful synchronisation solution, the answer must center around modifying the dependent image. This can be achieved by plugging in a piece of script code that is executed before any other application code in the container. Once the dependency is fulfilled, it makes the contained executing its application code.

 

The solution eliminates the need for you to install and operate your own container orchestration software, manage and scale a cluster of virtual machines, or schedules containers on those virtual machines as if it was created from the original image.

 

 

Two bash scripts are doing the job:

  1. sync-waiter-make-config.sh – bash script creating the image specific plug-in script and creating a new modified image;
     

  2. sync-waiter-this_YYYY-MM-DD_hh:mm:ss.nanosecs.sh – plug-in bash code, autogenerated by sync-waiter-make-config.sh.

The plug-in has two parameters, provided as environment variables:

  1. Variable name: CHECK_URLs – expects a list of URLs separated by spaces;
     

  2. Variable name: CHECK_INTERVAL_SECONDS – number of seconds between retries to access an URL.

The algorithm to create new image with plugged-in bash code:

  1. If your docker registry hosting the original image requires authentication, make sure you logged in to it before going to the next step;
     

  2. Run sync-waiter-make-config.sh by passing one parameter with original image URL.
    For example: 
    sync-waiter-make-config.sh quay.io/alfresco/alfresco-digital-workspace:1.2.0
     

  3. The script installs “curl” tool. This part of the script might need to be modified, depending on the type of OS in your original image;
     

  4. The script reads the ENTRYPOINT and CMD of the original image and saves them;
     

  5. The script generates a new image with name as the original one with capital “S” appended at the end. For example: 
    quay.io/alfresco/alfresco-digital-workspace:1.2.OS
     

  6. The new image ENTRYPOINT and CMD are modified so that firstly plug-in is executed and after the dependencies are fulfilled the original application code is executed.

The algorithm for executing the modified container:

  1. Bash plug-in is executed first on starting the container from modified image;
     

  2. Plug-in reads the environment variables values of CHECK_URLs and CHECK_INTERVAL_SECONDS;
     

  3. Plug-in checks the connectivity to the first URL;
     

  4. If there is successful check of the URL plug-in goes to check the next one;
     

  5. If check fails plug-in waits CHECK_INTERVAL_SECONDS seconds and retries;
     

  6. After all the URLs are successfully checked the plug-in executes ENTRYPOINT and CMD code of the original image.

When implementing this solution, it is important to remember that it is focused only on direct network connectivity dependencies. For any other types of dependencies, such as shared storage, bash scripts must be updated. Secondly, the solution hasn’t been tested on all the docker image types. It exposes only the idea of overcoming the given problem. For some images, you may need to modify the part of Bash scripts creating the “Dockerfile”, especially in “curl” installation piece.

 

Whilst Bash scripting may seem daunting, it is an extremely useful and powerful part of system administration and development. Utilizing its capabilities in this way can help achieve synchronization between AWS ECS services.

 

The Bash Script

 

#!/usr/bin/env bash

#

# Developed by Stoyan Gramatikov @ Helecloud LTD., Sofia, Bulgaria

#

 

# Verify function parameters

if [ ${#} -ne 3 ] || ! echo "${1}" | grep -q ':\S' || [ "${1}x" == "x" ] || [ "${2}x" == "x" ] || [ "${3}x" == "x" ]

then

    {

        echo

        echo 'Usage  : '"${0}"' <image path> <DEFAULT_CHECK_URLs> <DEFAULT_CHECK_INTERVAL_SECONDS>'

        echo 'Example: '"${0}"' "docker.io/alfresco/alfresco-content-repository-aws:6.1.1.1" "https://www.yahoo.com https://www.google.com" "30"'

        echo

    } 1>&2

 

    exit 128

fi

 

#################################### BEGIN ###################################

########################## PACKAGE this_sync_waiter ##########################

 

# Set default parameters

function this_sync_waiter_set_default_parameters()

{

    if [ ${#} -ne 2 ]

    then

        {

            echo

            echo 'Usage   : '"${FUNCNAME[0]}"' <default CHECK_URLs> <default CHECK_INTERVAL_SECONDS>'

            echo 'Example : '"${FUNCNAME[0]}"' "https://www.yahoo.com https://www.google.com" "30"'

            echo

        } 1>&2

 

return 128

    fi

 

    local default_check_urls="${1}"

    local default_check_interval_seconds="${2}"

 

    if [ "${CHECK_URLs}x" == "x" ]

    then

        CHECK_URLs="${default_check_urls}"

    fi

 

    if [ "${CHECK_INTERVAL_SECONDS}x" == "x" ]

    then

        CHECK_INTERVAL_SECONDS="${default_check_interval_seconds}"

    fi

}

 

# Print parameters

function this_sync_waiter_print_parameters()

{

    echo "CHECK_URLs=${CHECK_URLs}"

    echo "CHECK_INTERVAL_SECONDS=${CHECK_INTERVAL_SECONDS}"

}

 

# this_sync_waiter_export function

function this_sync_waiter_export()

{

    declare -f this_sync_waiter_set_default_parameters

    declare -f this_sync_waiter_print_parameters

    declare -f this_sync_waiter_main

}

 

# this_sync_waiter_main function

function this_sync_waiter_main()

{

    if [ ${#} -ne 2 ]

    then

        {

            echo

            echo 'Usage   : '"${FUNCNAME[0]}"' <default CHECK_URLs> <default CHECK_INTERVAL_SECONDS>'

            echo 'Example : '"${FUNCNAME[0]}"' "https://www.yahoo.com https://www.google.com" "30"'

            echo

        } 1>&2

 

return 128

    fi

 

    # Set default parameters if no values are set.

    this_sync_waiter_set_default_parameters "${@}"

 

    echo '### Parameters used ...'

    this_sync_waiter_print_parameters

 

    echo '### Start checking ...'

 

    local urls="$( echo "${CHECK_URLs}" )"

    local url=

    local i=

 

    for url in ${urls}

    do

        echo ; echo "### Checking URL: ${url}"

 

        i=0

        while true

        do

            let "i++"

            echo "--- Check number: ${i}"

            echo "--- Executing: curl -Ls ${url} > /dev/null"

            if curl -Ls "${url}" > /dev/null

            then

                echo "--- Successfully checked."

                break

            else

                echo "--- Waiting ${CHECK_INTERVAL_SECONDS} seconds ..."

                sleep "${CHECK_INTERVAL_SECONDS}"

            fi

        done

    done

 

    echo "### Continuing ..."

}

 

########################## PACKAGE this_sync_waiter ##########################

##################################### END ####################################

 

SRC_IMAGE="${1}"

DEFAULT_CHECK_URLs="${2}"

DEFAULT_CHECK_INTERVAL_SECONDS="${3}"

 

DST_IMAGE="${SRC_IMAGE}S"

SYNC_MARKER="$( date +%Y-%m-%d_%H:%M:%S.%N )"

SYNC_WAITER_THIS_SCRIPT="sync-waiter-this_${SYNC_MARKER}.sh"

DOCKERFILE_NAME="Dockerfile-this_${SYNC_MARKER}"

 

# Create this sync waiter script

function sync_waiter_create_this_script()

{

    echo -n "~~~ Creating ${SYNC_WAITER_THIS_SCRIPT} ... "

 

    {

        this_sync_waiter_export

 

        echo ; echo '# pre sync waiter main routine'

        echo ":"

 

        echo ; echo '# sync waiter main routine'

        echo "this_sync_waiter_main '${DEFAULT_CHECK_URLs}' '${DEFAULT_CHECK_INTERVAL_SECONDS}'"

 

        echo ; echo '# Add-on'

 

        local image_info="$( docker image inspect "${SRC_IMAGE}" )"

        local rc=${?}

 

        if [ ${rc} -ne 0 ] || [ "${image_info}x" == "x" ]

        then

            echo 'ERROR: Failed to get source image info: '"${SRC_IMAGE}"  1>&2

            return 2

        fi

 

        local entrypoint=$( echo "${image_info}" | jq '.[].Config.Entrypoint' )

        local cmd=$( echo "${image_info}" | jq '.[].Config.Cmd' )

 

        if echo ${entrypoint} | grep -q '^\s*\['

        then

            entrypoint=$( echo "${entrypoint}" | jq '.[]' )" "

        else

            entrypoint=''

        fi

 

        if echo ${cmd} | grep -q '^\s*\['

        then

            cmd=$( echo "${cmd}" | jq '.[]' )

        else

            cmd=''

        fi

 

        echo ${entrypoint}${cmd}

 

    }  > "${SYNC_WAITER_THIS_SCRIPT}"

 

    rc=${?}

 

    if [ ${rc} -eq 0 ]

    then

        echo 'Done.'

    fi

 

    return ${rc}

}

 

# Create Dockerfile

function sync_waiter_create_dockerfile()

{

    echo -n "~~~ Creating ${DOCKERFILE_NAME} ... "

 

    {

        echo "FROM ${SRC_IMAGE}"

 

        echo 'USER root'

        echo 'RUN ( apk add --update curl && rm -rf /var/cache/apk/* ) || ( yum install curl -y ) || ( apt-get update -y && apt-get install curl -y )'

 

        echo "COPY ${SYNC_WAITER_THIS_SCRIPT} /."

        echo "ENTRYPOINT [ \"/bin/sh\" ]"

        echo "CMD [ \"/${SYNC_WAITER_THIS_SCRIPT}\" ]"

 

    } > "${DOCKERFILE_NAME}"

 

    rc=${?}

 

    if [ ${rc} -eq 0 ]

    then

        echo 'Done.'

    fi

 

    return ${rc}

}

 

# sync_waiter_main routine

function sync_waiter_main()

{

    sync_waiter_create_dockerfile && sync_waiter_create_this_script && docker build -f "${DOCKERFILE_NAME}" -t "${DST_IMAGE}" .

    rc=${?}

 

    if [ ${rc} -eq 0 ]

    then

        echo

        echo "Synced image produced: ${DST_IMAGE}"

        echo

    fi

 

    return ${rc}

}

 

# Execute sync_waiter_main routine

sync_waiter_main

 

 

Any questions, please feel free to get in touch through my LinkedIn or take a look at how HeleCloud is helping businesses embrace Cloud technologies.

 

References

AWS ECS guide:

https://docs.aws.amazon.com/AmazonECS/latest/userguide/Welcome.html

Docker file reference:

https://docs.docker.com/engine/reference/builder/

Bash reference manual:

https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html

Share on Facebook
Share on Twitter
Please reload

Featured Posts
Archive
Please reload