DinD

#devops #docker #dind #cicd

Docker in Docker

Docker has transformed the way we build, deploy, and maintain applications by providing unrivalled containerisation capabilities. One exciting concept in the Docker ecosystem is to run Docker within a container, which is appropriately termed Docker in Docker (DinD). This technique can be quite beneficial in a variety of settings, including CI/CD pipelines, testing, and isolation. In this post, we'll go over how to run Docker in Docker, including a full step-by-step approach to help you learn this sophisticated subject.

Steps

Pull the DinD Image: docker pull docker:dind
Launch the DinD Container: docker run --privileged --name dind-container -d docker:dind
Access the DinD Container: docker exec -it dind-container sh
Test Docker Within the DinD Container: docker info
Run Containers Inside the DinD Container: docker run -it --rm alpine
Clean Up: docker stop dind-container
And: docker rm dind-container

Start a daemon instance

$ docker run --privileged --name test-docker -d \
    --network some-network --network-alias docker \
    -e DOCKER_TLS_CERTDIR=/certs \
    -v test-docker-certs-ca:/certs/ca \
    -v test-docker-certs-client:/certs/client \
    docker:dind

Runtime Settings Considerations

For production Docker instances, consider adjusting the following runtime configuration parameters based on the standard systemd docker.service configuration.

$ docker run --privileged --name test-docker -d \
    ... \
    --ulimit nofile=-1 \
    --ulimit nproc=-1 \
    --ulimit core=-1 \
    --pids-limit -1 \
    --oom-score-adj -500 \
    docker:dind

Some of these will not be supported based on the settings on the host's dockerd, such as --ulimit nofile=-1, causing problems that look like error setting rlimit type 7. operation not permitted, and some may inherit sane values from the host dockerd instance or may not apply to your use of Docker-in-Docker. For example, you should probably set --oom-score-adj to a value higher than dockerd on the host so that your Docker-in-Docker instance is killed before the host Docker instance.

Rootless

Just like the regular dind images, --privileged is required for Docker-in-Docker to function properly

Example: $ docker run -d --name test-docker --privileged docker:dind-rootless

To verify the daemon has finished generating TLS certificates and is listening successfully.

$ docker logs --tail=3 test-docker
time="xxx" level=info msg="Daemon has completed initialization"
time="xxx" level=info msg="API listen on /run/user/1000/docker.sock"
time="xxx" level=info msg="API listen on [::]:2376"

Using docker-entrypoint.sh which auto-sets DOCKER_HOST appropriately

$ docker exec -it test-docker docker-entrypoint.sh sh
/ $ docker info --format '{{ json .SecurityOptions }}'
["name=seccomp,profile=default","name=rootless"]

Data storage

Here are few options for data storage:

Allow Docker to store your data on the host system's disc using internal volume management. This is the default, and it is simple and straightforward for the user. The negative is that files may be difficult to find for tools and applications that operate directly on the host system, i.e. outside of containers.
Create a data directory on the host system and mount it to a directory visible from within the container. This places the files in a well-known location on the host system, allowing utilities and applications to easily access them. The negative is that the user must ensure that the directory exists and that e.g. directory permissions and other security mechanisms on the host system are set up correctly. The Docker documentation is a wonderful place to start for learning about the many storage options and variations, and there are numerous blogs and forum posts that discuss and provide advise on this topic.

Here's an example of using a data directory.

Create a data directory on a suitable volume on your host system, /home/suyi/docker-dir
Start docker container: docker run --privileged --name test-docker -v /home/suyi/docker-dir:/opt/ -d docker:dind The -v /home/suyi/docker-dir:/opt/ part of the command mounts the /home/suyi/docker-dir directory from the underlying host system as /opt/ inside the container, where Docker by default will write its data files.

Running Docker in Docker may be an extremely useful tool in a variety of development and testing environments. However, it's crucial to remember that DinD presents its own set of problems and security concerns. It is not recommended for production use owing to security concerns and performance overhead. However, for isolated testing and experimentation, DinD might be a beneficial addition to your Docker toolkit.

Ref: https://hub.docker.com/_/docker/
Ref: https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

Debug School