Local Deployment¶

Dependencies¶

Build¶

From the project folder, run:

./tools/build.sh

The will create the docker images tagged with latest which are used by docker-compose.

Run¶

Run docker compose:

docker-compose -p n1es -f tools/docker-compose.yml up

This will start the following containers:

nginx frontend (named n1es_nginx_1)
gunicorn/flask backend (named n1es_backend_1)
celery backend worker (named n1es_worker_1)
postgres database (named n1es_db_1)
redis job queue (named n1es_redis_1)
minio object store
jaeger opentracing

The REST api for the service is exposed on port 8851 of the nginx container, which docker will map to a high numbered port on your host.

The address of the nginx endpoint can be found with:

docker port n1es_nginx_1 "8851"

For example to GET the service status:

$ export ENTITY_SERVICE=`docker port n1es_nginx_1 "8851"`
$ curl $ENTITY_SERVICE/api/v1/status
{
    "status": "ok",
    "number_mappings": 0,
    "rate": 1
}

The service can be taken down by hitting CTRL+C. This doesn’t clear the DB volumes, which will persist and conflict with the next call to docker-compose … up unless they are removed. Removing these volumes is easy, just run:

docker-compose -p n1es -f tools/docker-compose.yml down -v

in between calls to docker-compose … up.

Monitoring¶

A celery monitor tool flower is also part of the docker-compose file - this graphical interface allows administration and monitoring of the celery tasks and workers. Access this via the monitor container.

Testing with docker-compose¶

An additional docker-compose config file can be found in ./tools/ci.yml, this can be added in to run along with the rest of the service:

docker-compose -p n1estest -f tools/docker-compose.yml -f tools/ci.yml  up -d

docker logs -f n1estest_tests_1

docker-compose -p n1estest -f tools/docker-compose.yml -f tools/ci.yml down

Docker Compose Tips¶

Local Scaling¶

You can run additional worker containers by scaling with docker-compose:

docker-compose -f tools/docker-compose.yml scale es_worker=2

A collection of development tips.

Volumes¶

You might need to destroy the docker volumes used for the object store and the postgres database:

docker-compose -f tools/docker-compose.yml rm -s -v [-p <project-name>]

Restart one service¶

Docker compose can modify an existing deployment, this can be particularly effective when you modify and rebuild the backend and want to restart it without changing anything else:

docker-compose -f tools/docker-compose.yml up -d --no-deps es_backend

Scaling¶