What is Skopos?

Skopos™ is a modern continuous deployment system for container-based services, especially for DevOps teams and those using microservice architectures.

Skopos uses a plugin architecture and includes core plugins for performing operations on the following deployment environments: Docker Swarm, Docker (single host), Amazon ec2 (ASG), and Amazon ECS (experimental).

With Skopos you can:

  • Automatically deploy new images based on tags.
  • Deploy in respect of dependencies among services, and in respect of service attribution (singleton, stateful, stateless).
  • Perform rolling upgrades and automatically roll-back deployment failures.
  • Inject user defined actions into deployment plans: execute user scripts, call Skopos plugins, add manual gates or inject delays. Injected actions can make use of service instance data to act in respect of newly created, updated or destroyed instances.
  • Fully verify deployed services: Skopos probes package service healthchecks as containers so they can verify the operation of a target service on the same network used to consume that service.
  • Use the same compose file or application model with different environment bindings, e.g.: for development, staging and production, or for different geographical regions.
  • Get chatops notifications for deployment operations.
  • Discover running stacks and services.
  • Use the Skopos UI to:
    • Visually edit stacks/applications.
    • Use "VCR" deployment controls: start, pause, step-forward, resume, and abandon.
    • View an application model: this grapical model reflects state and deployment progress.
    • View an application deployment plan: this logical flow diagram reflects state and deployment progress and can be "zoomed-in" to show the logical flow of any individual service's deployment plan.

Install Skopos

Skopos consists of two components:

  • the Skopos engine, packaged in a container for simple installation. The Skopos engine exposes both API and web interfaces.
  • the Skopos control utility, a command line utility skopos, available for Linux, Mac OS and Windows. The utility may run on the same host where the Skopos engine runs or anywhere else with network access to that host.

To start the Skopos engine, run the following command on a Docker-enabled host (a swarm manager node or a single-host Docker, such as a laptop):

docker run -d -p 8100:8100 --restart=unless-stopped --name skopos \
   -v /var/run/docker.sock:/var/run/docker.sock \
   opsani/skopos

This command starts the Skopos engine and exposes its API and web-based user interface on port 8100 of the host.

Note: this command starts the Skopos engine as an HTTP server with no user authentication and is suitable only for use on networks that are not exposed to the public Internet. See also the following user guides:

  • Installation: additional start options, including running Skopos as a service and providing access to private Docker registries.
  • Running Skopos Securely: set up encryption or user authentication.

To download the skopos control utility on Linux:

wget https://s3.amazonaws.com/get-skopos/latest/linux/skopos
chmod +x skopos
sudo mv skopos /usr/local/bin

Or download the control utility for Mac OS X or Windows and install it on the executable path.

First Deployment

This simple first deployment verifies that Skopos is installed correctly and can be controlled with the skopos utility. It also provides a practical introduction to using Skopos.

The sample application lives in a github repository https://github.com/opsani/skopos-sample-twotier. It is a simple two-tier application with a front-end and a back-end service, both replicated. The front-end service exposes a web UI which displays a page showing which front/back instances served the request. The repository contains the following files:

  • compose-swarm.yaml: a v3 compose file suitable for deploying on Docker Swarm (it uses the Swarm routing mesh to load-balance the front-end). Front-end web UI externally exposed port is 80.
  • compose-docker.yaml: a v3 compose file suitable for deploying on Docker single-host (it uses nginx to load-balance the front-end). Front-end web UI externally exposed port is 8080.
  • env-v2.yaml: a Skopos target environment descriptor (TED file). This optional TED file modifies the application to deploy different versions of the front/back services. It can be used to test an application upgrade.

Deploy the sample app directly from github:

# on Docker Swarm
skopos stack deploy --c github://opsani/skopos-sample-twotier/compose-swarm.yaml two-tier-swarm

# on Docker single-host
skopos stack deploy --c github://opsani/skopos-sample-twotier/compose-docker.yaml --profile docker \
two-tier-docker

You can view the deployment progress in the Skopos web UI. Point a browser at port 8100 of the host on which Skopos is running: http://<ip_address>:8100/ and open the two-tier application. Deployment progress is visible in both the model and plan views.

Note: if you are working with a swarm cluster, you can try both of these deployments side-by-side. The first will deploy a swarm stack two-tier-swarm, while the second will deploy the two-tier-docker containers on the same swarm manager node on which Skopos is running.

Once the application is deployed, you can view its web UI with a browser on port 80 of any swarm node (port 8080 of Docker single-host). Refresh the browser to see which front/back instance served the request.

Note: the two-tier app web UI displays the front and back service IP addresses and instance IDs only when run on Amazon EC2.

To update the sample app:

# on Docker Swarm
skopos stack deploy --c github://opsani/skopos-sample-twotier/compose-swarm.yaml \
--env github://opsani/skopos-sample-twotier/env-v2.yaml two-tier-swarm

# on Docker single-host
skopos stack deploy --c github://opsani/skopos-sample-twotier/compose-docker.yaml \
--env github://opsani/skopos-sample-twotier/env-v2.yaml --profile docker two-tier-docker

You can see the rolling upgrade progress in the Skopos UI. You can also check the update progress by refreshing the browser window for the sample application UI: as the update progresses, different versions of front/back will serve requests, some with a blue theme (the old version) and some with a red theme (the new version).

To teardown the sample app:

# on Docker Swarm
skopos stack rm two-tier-swarm

# on Docker Swarm
skopos stack rm two-tier-docker

New User Workflows

You can start using Skopos in one of three simple ways, depending on where you are in your development cycle:

  • Import a Docker Compose File - use this approach if you have an existing Docker compose file you want to start from, whether or not the services are already running. Skopos supports both v2 and v3+ compose files.
  • Discover Existing Stacks or Services - this approach works if you already have a running containerized application, whether it was deployed manually or using a Docker compose file.
  • Design in the Skopos Editor - use the Skopos editor to design an application from scratch. The visual editor can also be used to modify any application loaded into Skopos, e.g., to work with an existing compose file.

Import a Docker Compose File

Skopos works both with its native model descriptors and with Docker compose files. You can import compose files using the web UI or by providing a compose file to the Skopos command-line load, run or stack deploy operations.

On import, Skopos automatically creates a visual layout for the application based on the dependencies among its services. Specifying explicit dependencies between services in the compose file will help create a better graphical layout; however, you can easily modify the layout after import using the Skopos Editor.

Using the web UI

  1. Use a browser to access the Skopos user interface, e.g. at port 8100 of the host on which Skopos is running: http://<ip_address>:8100/
  2. Click the Import button and:
  3. Click the Choose File button to select a compose file to import.
  4. Enter a name for the application to create.
  5. Use the Run as Docker Swarm services checkbox to indicate whether the services found in the compose file should be run as Swarm services or simple Docker containers (both options work with both v2 and v3+ compose files).
  6. Click the Import button.

Using the command-line interface (CLI)

Note: if the Skopos engine is running on a different host from the CLI, or is using a port other than the default 8100, then you will need to add the --bind <host>:<port_number> option to the example commands below.

Deploy a Stack

To deploy directly from a compose file use the stack deploy command:

skopos stack deploy -c <compose-file> STACK

This command emulates the familiar docker stack deploy syntax:

  • <compose-file> is the path to the compose file to import
  • STACK is the name of the stack/application to deploy

By default v2 compose files import using the docker core plugin and target Docker single-host as the deployment environment. Likewise, v3+ compose files import using the docker-swarm core plugin and target Docker Swarm as the deployment environment. This behavior can be controlled with the --profile option, e.g., use --profile docker or --profile swarm.

You can also deploy from compose using the run command, which has slightly different syntax and default options:

skopos run --project <project> <compose-file>
  • <project> is the name of the stack/application to deploy.

Load a Stack

To load a compose file into Skopos without deploying it, use the stack load command:

skopos stack load -c <compose-file> STACK

You can also use the load command, which has slightly different syntax and options:

skopos load --project <project> <compose-file>

Once an application is loaded, you can start a deployment using either the web UI or the CLI.

Command Line Help

For detailed information please see the Skopos command line reference. You can also get contextual help from the CLI itself, e.g.:

  • skopos --help
  • skopos stack --help
  • skopos stack deploy --help

Discover Existing Stacks or Services

The Skopos discover operation examines the Docker host or Swarm cluster on which Skopos is running. It loads each discovered running application into Skopos as described below.

  1. Use a browser to access the Skopos user interface, e.g. at port 8100 of the host on which Skopos is running: http://<ip_address>:8100/
  2. Click the Discover button to begin. Discovery takes about 10-30 seconds, but will take longer on first use (to pull container images for the discovery operation) or when run on a large Swarm cluster.

For best results, make sure that running applications are in use (traffic passing among various services) just prior to discovery. This will help identify dependencies and capture connections between services, which, in turn, will produce a more intuitive graphical layout.

Discovery populates the application list with all discovered applications, and will overwrite any same-named application which is already loaded into Skopos:

  • Each running stack is loaded as a Skopos application (e.g., stacks deployed from Docker compose v3).
  • Each running compose project is loaded as a Skopos application (e.g., projects deployed from Docker compose v2).
  • Remaining services/instances are grouped into one or more applications by virtue of being attached to a common private network. These applications have a prefix discover_net and are not directly transformable/deployable by Skopos (e.g., discover_net_bridge)
  • An application named discover_all is always loaded. This is a catch-all application which shows all discovered services/instances. It is not directly transformable/deployable by Skopos.

After discovery, you can click on the discovered applications to see their architectures. Within an application, click on the {} button to review the details of the extracted application model (in YAML), modify and/or save the generated files. You can also edit discovered application in the UI.

For more information on the discovery process, including manually executing a long-running discovery to extract rich connection detail, see the Skopos Discovery guide.

Design in the Skopos Editor

The Skopos visual application editor can be used to edit a loaded application, or to design a new application from scratch.

Here is an excellent short video demonstrating building an application from scratch in the Skopos editor:   Build and Deploy Docker Stacks without YAML.

To edit a Skopos application, first open it from the application list. Then, click on the Edit button to open the editor. Within the editor:

  • Click on the Save button to save changes and return to the model view.
  • Click on the Discard button to discard changes and return to the model view.
  • Click on Current Target to view the application in its current state (without any edits).
  • Click on Canvas to switch back to the editor canvas after viewing the current target.
  • Click on a component, gateway or connection to open the editor dialog for that object on the right side of the screen. There is also such a dialog for the application as a whole which is accessed by clicking on a blank area of the canvas.
  • Drag-and-drop objects using the mouse.
  • To delete an object: click on the object to select it and either press the delete key or click on the Delete button of the editor dialog for that object.
  • To create a new connection: click on the connection start point (blue circle on the right side of a component or gateway) and drag the new connection to its end point (blue circle on the left side of a component or gateway).
  • To create a new component (service) or gateway, click on the Add button in the upper left of the canvas.

_Note: Skopos uses a gateway object to visually represent the exposed or port-mapped ports of a component. You can edit a gateway to modify its externally exposed port(s), or click on a connection from a gateway to modify the mapping of the externally exposed port(s) to the internal port of the connected component.

Next Steps

Tutorial Applications

Explore Skopos through any of our step-by-step tutorial applications:

  • Docker Pet Voting Application: a scalable variant of the Docker example Pet Voting Application which deploys to Docker (single host). This application demonstrates the injection of user scripts into the deployment process.
  • Swarm Pet Voting Application: a Swarm version of the Pet Voting Application which uses User Defined Quality Gates to verify the deployment.
  • CI/CD Pipeline: an example CI/CD pipeline with Jenkins and Skopos which demonstrates automation from application code commit through image build to deployment.
  • Galera Database Cluster: an example MariaDB Galera cluster which deploys to Docker Swarm and demonstrates:
    • cluster bootstrap and seed removal
    • Galera node failover in the event of swarm node failure

These tutorial apps also illustrate Skopos's approach to infrastructure-as-code, including using deployment artifacts directly from version control.

Deeper Dive

Check out these user guides:

Explore the Skopos command line. You can:

  • load, deploy, update, destroy or discover stacks/applications
  • list or get status for stacks/applications
  • control an application deployment: start, pause, resume or abandon a deployment, or re-generate the deployment plan for a stack/application
  • execute transient deployments (these do not require a running Skopos engine)
  • login or logout of Skopos

Explore Skopos probes. Probes package service healthchecks as containers so they can be instantiated in-application. Probes faithfully reproduce the position of a service's client in order to verify a service in the same manner as it is consumed.

Get Help or Give Feedback

We give good support. If you would like help getting started with Skopos, or are having any difficulty with its use, please e-mail us at support@opsani.com.

We also love feedback. If you try Skopos - or decide not to try it - please send us your feedback on what you liked and what we can improve. Just e-mail us at support@opsani.com.