Docker: Overview

Docker is a platform for building, shipping, and running applications.

Key Tools

Docker Hub - public repositories with Docker Images
Docker Desktop - local client with graphical interface and CLI for running and interacting with Docker Images, Docker Containers, etc.
Docker Compose - tool for defining and organizing multiple Container deployments/applications.

Images and Containers

Images
- Are built.
- Specified by configuration files.
- Definitions can be combined and composed.
- Container Images represent an encapsulated resource with all its dependencies.
- Images are templates for Containers.
Containers
- Are run.
- They represent an Image at runtime, environment, its state, etc.

Virtual Machines and Hypervisors

Virtual Machines	Hypervisors	Docker
Separate, dedicated, Operating System Kernels.	Shared underlying Operating System Kernel.	Uses Virtualization through an independent Hypervisor or Virtual Machine framework.
Dedicated system resources (CPU's, RAM, HDD).	Shared system resources.	Either.
Launches an entire simulated Operating System with machine state, etc.	Light-weight, virtualized, containerized, encapsulated, and insolated environments.	Either.
VirtualBox, Apple Virtualization	Hyper-V, Docker VMM, the Windows Subsystem for Linux (`WSL2`, which runs on a subset of Hyper-V)	Docker Desktop

Note: Docker can be configured in either fashion (is compatible with either the Virtual Machine or Hypervisor approach), but typically uses some kind of Virtual(ized) Machine, and is primarily for building, shipping, and flexibly running application.

Resources and Links

Docker: Basic Commands

# Build from dockerfile
## Use this over `docker build - < Dockerfile`
## Note that the dockerfile copies in ANY files in this directory
docker build .

# Docker metrics and processes
docker images --all
docker info
## Get the CONTAINER ID <aa9f01c38d04>
docker stats

# Cleanup
## Remove image
docker rmi -f IMAGE_ID
## Remove container
docker rm CONTAINER_NAME
docker stop CONTAINER_NAME
docker system prune --volumes

Refer to: https://github.com/Thoughtscript/docker

Also: https://github.com/Thoughtscript/postgres_json_practice/blob/master/1%20-%20dockerfile/docker.sh

Resources and Links

Code samples:

Docker: On Mac ARM

Some recent changes for use on Mac.

Rosetta 2

Tested on an Apple M3 laptop with macOS 15.1.1 (24B91)

Newer Macs (equipped with Apple's newish ARM CPU's) will require installing Rosetta 2 - a binary translator for converting x86 and ARM instructions.
Since Docker virtualizes x86 operations, it must be installed on Mac now to use Docker:
```
softwareupdate --install-rosetta
```

Failing to do so will result in the following error:

Rosetta is only intended to run on Apple Silicon with a macOS host using Virtualization.framework with Rosetta mode enabled.

Make sure to update and restart Docker Desktop.
Verify that the checkbox Settings > General > Virtual Machine Options > Use Rosetta for x86_64/amd64 emulation on Apple Silicon is selected.

With instructions for the above: https://www.docker.com/blog/docker-desktop-4-25/

Docker Commands

Since Docker on Apple's ARM CPU requires Rosetta 2 (and Rosetta 2 in turn requires Compose V2), the following Compose V2 command syntax is now enforced:

docker-compose up is now docker compose up.

More on this change: https://docs.docker.com/compose/releases/migrate/

Resources and Links

Docker: dockerfile

FROM postgres:13.0

# Execute init scripts
## These only have to be copied into /docker-entrypoint-initdb.d/
COPY init_json_sql.sql /docker-entrypoint-initdb.d/

FROM python:3.8.2

RUN echo "Creating working dir and copying files"
RUN mkdir /app
WORKDIR /app
COPY . .

# update pip globally within the container
RUN python3 -m pip install --upgrade pip
# update requirements by directory
RUN cd ml && python3 -m pip install -r requirements.txt
# run the machine learning scripts to save off the annModels within the image
# the logs for these scripts will now show in Docker Desktop
RUN cd ml && python3 ml-conjunction.py && python3 ml-disjunction.py && python3 ml-implication.py && python3 ml-negation.py && python3 ml-nand.py

# this is apparently a required dependency of SQLAlchemy
RUN apt-get update && apt-get install -y default-mysql-client default-libmysqlclient-dev
RUN cd server && python3 -m pip install -r requirements.txt
# host and ports are set in server/main.py but they could be passed below instead
# these are required to bind the ips and ports correctly
CMD [ "bash", "run.sh" ]

Useful Dockerfile Commands

FROM
- Is typically set at the top of the file but multiple FROM can be used throughout.
- Specifies that some prebuilt/existing Docker Image should be downloaded and used as a baseline upon which the other Commands are run.
WORKDIR
- Specifies the current Working Directory (within the Image and Container directory structure) where Commands are run.
RUN
- Specifies a Bash Command or Expression to be executed as the Image is being built.
- The results of RUN Commands are saved within the Image.
VOLUME
- Defines a Volume Mount.
- (Note: These can be Bind Mountings that are transient and confined to the local Container.)
COPY
- Simplified and cp-equivalent instruction.
CMD
- A Command executed and run whenever the Docker Container runs/starts.

Resources and Links

https://docs.docker.com/engine/reference/builder/

Code samples:

Docker: Images

Layers

Docker Images are assembled and built up using multiple Layers:

Each Layer is cached and changes made only to the top-most relevant one(s).
They are Read-Only once built and changes made to the Docker Image baseline are done so in a new Read-Write Layer that sits on top of the Docker Image baseline.

Docker: Storage

Docker Volumes

Docker Volumes are persistant data stores for Containers.

In Docker Compose, a Volume is declared in its own block, then associated with each Service (source, typically the Volume name) along with a destination path (a file path or directory within the Volume) where the persisted data will reside.

services:
  mongo:
    image: bitnami/mongodb:7.0.9
    ports:
      - "27017:27017"
    volumes:
      - 'mongodb_data:/bitnami/mongodb'
    environment:
      - MONGODB_ROOT_USER=rootuser
      - MONGODB_ROOT_PASSWORD=rootpass
      - MONGODB_USERNAME=testuser
      - MONGODB_PASSWORD=testpass
      - MONGODB_DATABASE=testdatabase
      # This is required on Apple Silicon https://github.com/docker/for-mac/issues/6620
      # https://github.com/bitnami/containers/issues/40947#issuecomment-1927013148
      - EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU=1
    networks:
      - testnet

  node:
    build:
      context: ./node
      dockerfile: dockerfile
    ports:
      - '8888:8888'
    depends_on:
      - mongo
    networks:
      - testnet
    restart: unless-stopped

  react:
    build:
      context: ./react
      dockerfile: dockerfile
    ports:
      - '443:443'
      - '1234:1234'
    depends_on:
      - node
    networks:
      - testnet
    restart: unless-stopped

  angular:
    build:
      context: ./angular
      dockerfile: dockerfile
    ports:
      - '4200:4200'
    depends_on:
      - node
    networks:
      - testnet

volumes:
  mongodb_data:
    driver: local

networks:
  testnet:
    driver: bridge

https://github.com/Thoughtscript/mearn_2024/blob/main/docker-compose.yml

Bind Mounts

Bind Mounts are Volumes that are Mounted from a specific location on the host machine into the Docker Image and Container.

Example: host directory ./static is bound to Docker Container file path: /opt/app/static.

# docker compose config
services:
  frontend:
    image: node:lts
    volumes:
      # Bind mount example
      - type: bind
        source: ./static
        target: /opt/app/static
volumes:
  myapp:

https://docs.docker.com/engine/storage/bind-mounts/

Dockerfile Volumes

A slight variation on the topics above. dockerfile Volumes can define a Mount Point at a specific location. For example, like so:

FROM ubuntu

USER myuser

RUN mkdir /myvol
VOLUME /myvol
RUN chown -R myuser /myvol

This can be used in tandem with chown priviliges and ECS_CONTAINERS_READONLY_ACCESS to restrict what's writeable within a Container to exactly the VOLUME. (AWS ECS will allow a VOLUME to be writeable even if the rest of the Docker Image and Container aren't.)

https://docs.aws.amazon.com/config/latest/developerguide/ecs-containers-readonly-access.html

https://docs.docker.com/reference/dockerfile/#volume

Local Files

var/lib/docker - default Docker directory used to store data for Containers, Docker Images, and Volumes.
var/lib/docker/volumes - location from where Docker Mounts a Volume.
Data is removed via: docker system prune -a.

Docker Storage Drivers

Docker Storage Drivers facilitate the Layered architecture and caching used when building Docker Images and running Containers.

Resources and Links

Code samples:

https://github.com/Thoughtscript/mearn_2024/blob/main/docker-compose.yml