ec2-ecs plugin

The EC2 Container Service (ECS)[^1] plugin provides instance deployment and operations using the Amazon Web Services EC2 Container Service.

Support for ECS in the current version of Skopos is EXPERIMENTAL. See its known limitations below.

Support for AWS ECS is currently experimental. We don't recommend using the ec2-ecs plugin for production deployments yet. For more information, please contact us at support@opsani.com.

Quick jump to Configuration.

[^1]: All product and company names are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

EC2 Roles and Permissions

To be able to call the ECS API, the access key (or role, if Skopos is running on EC2) should have the AmazonEC2ContainerServiceFullAccess managed role or equivalent permissions.

Clusters, Regions and Availability Zones

When deploying to ECS clusters, Skopos with the ec2-ecs plugin supports accessing ECS clusters and deploying to multiple availability zones for improved reliability. There is no additional Skopos- or plugin-specific configuration necessary, as long as the ECS cluster and load balancers are configured to work in multiple zones.

Pre-requisites

In order to deploy applications with the ec2-ecs plugin, the following resources SHOULD be provisioned prior to deployment:

  • AWS account
  • ECS cluster
  • ECS cluster container instances (confusingly, this refers to EC2 virtual machines which are the ECS cluster nodes)
  • roles, security groups and subnets
  • load balancers and, if applicable, target groups

The ec2-ecs plugin has been tested with the default ECS-optimized AWS machine image (AMI) and the built-in ECS container agent.

In addition, if using the optional support for service discovery (inter-service binding and communication):

  • a private Route 53 zone
  • internal load balancers for any services that are not already connected to load balancers
  • an A record with alias to the respective load balancer or target group (public or internal)

Note that when using the optional service discovery (service_discovery_mode: elb-manual), there is no need to create gateway objects for each service; only for the ones that provide services to users outside of the application.

Task Definitions and Services

The ec2-ecs plugin automatically creates Task Definitions for components in the application. It also deactivates task definitions that are no longer used (usually there will be 2-3 of the latest task definitions left active).

Note that in ECS, task definitions are per-account and shared between clusters. In order to avoid conflicts between unrelated applications, Skopos will prepend the cluster name when creating a task definition; the task definion name is constructed as follows:

<cluster>_<project>_<component>

where:

  • <cluster> is the ECS cluster name,
  • <project> is the Skopos project name (e.g., provided with the load command), and
  • <component> is the name of the component for which the task definition is being created.

The ec2-ecs plugin creates Services for each component.

Service names in ECS are scoped within the cluster, so the ec2-ecs plugin constructs the service name based only on the project and component:

<project>_<component>

For most of the task definition and service attributes, the ec2-ecs leaves the reasonable defaults. See the boto3 AWS documentation for the defaults, as well as for objects mentioned in the raw configuration settings below.

Limitations

The following list highlights key limitations of the ec2-ecs plugin support for running applications on ECS in the current version of Skopos. If your usage is restricted by these limitations and you would like to use Skopos with ECS, please contact us at support@opsani.com.

  • Single container per task. It is not possible to circumvent this limitation using the raw component settings.
  • Deployment rollback does not operate correctly with dynamic container tags, as ECS currently does not support container specification by digest. This limitation effectively prevents use of dynamic tags in production with the ec2-ecs plugin.
  • Not all types of deployment failures are detected quickly and error messages may not be surfaced sufficiently (workaround is using the overall failure timeout and Event section in the AWS ECS console)
  • Deploying upgrades may take extra 5 minutes per component, as Skopos waits for the old versions to be drained and destroyed (the ECS draining timeout does not seem configurable)
  • There are no pre-flight checks (except basic configuration settings)

In addition, the optional service discovery integration requires pre-provisioned load balancers and Route 53 entries.

AWS ECS Limitations

Note that when deploying to ECS clusters, Skopos inherits the following ECS limitations:

  • any resource limits applicable to your AWS account and region (e.g., # instances, # load balancers, etc.)
  • a single load balancer per component (however, multiple components can back a single load balancer and even a single target group)
  • changes in gateways and how components are attached to them cannot be performed on running services. This plugin automatically works around this ECS limitation in case the service is scaled to 0 instances prior to upgrade.
  • no support for image specifications using digest (e.g., image@sha256:a02596fdd012...)
  • upgrading services that cannot have multiple instances on the same ECS container host may deadlock or execute slowly if the number of replicas (tasks) is close to exactly matches the number of ECS container nodes and the default minimumHealthyPercent is used (100%).
  • at least one of mem and mem_reserve resource setting must be specified for every component (see below)

Component-specific attributes in the model

All ec2-ecs plugin attributes for components are optional except mem and mem_reserve - at least one must be defined for each component.

components:
  web:
    image: nginx:1.13.0
    plugins:
      ec2-ecs:
        role: xxx  # IAM role to give to each task
        group: xxx
        cpu: nn
        mem: nnn            # one of mem or mem_reserve must be specified
        mem_reserve: nnn    #  " " " "
        command: [ xxx ]
        args: [ xxx, xxx ]
        network_mode: bridge|none|host

        raw:
          taskDefinition:
            attributes... # incl. attributes[] and attributes{}
            containerDefinitions: # special handling into containerDefinitions[<each>]
              attributes... # incl. attributes[] and attributes{}
          service:
            attributes... # incl. attributes[] and attributes{}

Gateway-specific attributes in the model

components:
  web:
    image: nginx:latest
    exposes:
      ports: [ 80 ]
    ...

gateways:
  in:
    type: load_balancer
    exposes:
    - name: http
      port: 8080  # may be set to "" for application load balancers
      target_port: 80
    target: [ web ]

    plugins:
      ec2-ecs:
        raw:
          .... # merges into service's loadBalancers[0] record

Configuration

Here is the format of the plugin configuration record for the ec2-ecs plugin:

For the full environment description format, see the Target Environment Configuration document.

core_plugin: ec2-ecs

plugin_config:

  ec2-ecs:
    access_key_id:        # optional, not needed if using role from within EC2
    secret_acces_key:     # " " ""
    region: us-east-1     # optional but must match region of cluster
    cluster: default      # optional, defaults to `default`; ok to use short name or ARN
    timeout: 120          # how long to wait on update for something to make progress before giving up
    overall_timeout: 500  # how long to wait on update, total (regardless of progress), before giving up
    update_monitor: 5     # how long to wait after reaching final state before considering it a success

    discovery_zone: myapp.mycluster1.internal   # service discovery automatic zone assignment (dnsDomainSearch)
    discovery_mode: none|elb-manual             # service discovery to expect/auto-attach to per-service elb (int or ext, classic or app)

    lb:
      gateway1: classic_elb_name_or_arn # classic ELB with standard role ecsServiceRole
      gateway2:
        name: classic_elb_name_or_arn   # classic ELB
        role: xxx                       # role to give ECS for configuring ELB
      gateway3: target_group_arn        # application ELB with standard role ecsServiceRole
      gateway4:
        name: target_group_arn          # application ELB
        role: xxx                       # role to give ECS for configuring ELB

      service1:                         # implied load balancer for service1 (discovery_mode=elb-manual)
        name: elb_name_or_group_arn     # classic ELB name/arn or target group arn for app ELB
        role: xxx                       # role to give ECS for configuring ELB


    volumes:
      log_vol_name1: {}     # empty temp volume
      log_vol_name2: { from: "<abspath_or_volname" }  # bind to host dir or volume