We started our article series with the installation of PostgreSQL and PgAdmin on Docker. Next, we will examine volume management, which is another important topic in Docker. For this, we will start by running the PostgreSQL 11 image. Do not start with another version because in the rest of the article, we will update our data to PostgreSQL 11.7 without corrupting it. Therefore, the data directory used by the container will not be affected and can be used by the two minor versions of PostgreSQL. Here, we can also think of the data directory as the folder that we will follow and that holds our common files/data (for these two PostgreSQL versions). The -v tag given in the command below indicates the volume expression, and we define the superuser password as an environment variable with -e.
[profelis@profelis ~]$ docker run –name pg11 -d -e POSTGRES_PASSWORD=pass -v pg11data:/var/lib/postgresql/data postgres:11
Unable to find image ‘postgres:11’ locally
11: Pulling from library/postgres
5e35bd43cf78: Pull complete
0df82dab3c88: Pull complete
0670a6d74375: Pull complete
bd2be22379ef: Pull complete
9eb829e5b266: Pull complete
95d7f9aaa5bd: Pull complete
de97a90167ed: Pull complete
4b50288dfea5: Pull complete
b71c4d54a488: Pull complete
e1828d0d4b60: Pull complete
dce8713bb72b: Pull complete
37db80a505c4: Pull complete
936725e631d9: Pull complete
86ab6b577e4f: Pull complete
Digest: sha256:32f2b2b831bcc5318183eb09915c97552f7c2a9657b225f9ea0d23ed727271bb
Status: Downloaded newer image for postgres:11
2b891069b8bb9bfcb24952f4759ad66f9ad37134e35c6b0c41dc85e4242cd779
[profelis@profelis ~]$ docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b891069b8bb postgres:11 “docker-entrypoint.s…” 6 seconds ago Up 4 seconds 5432/tcp pg11
In the first article, we had examined the lifecycle of containers running in Docker, but we had left how they use system resources or behave on the network to subsequent articles. In fact, Docker isolates both the processes and disk usage areas of all containers running within it from each other. While separate operating system processes are executed within each container, they are run in separate disk spaces.
So where does a Docker container write its data, or does it write at all? Who defines this and where? To find the answers to these questions, one must first understand the concept of volumes in Docker. Yes, Docker containers also write to the operating system they run on “default” It uses a disk space, and we can either check this disk space or use another disk space that we have defined.
First of all, Docker containers It is mortal! This means that when Docker containers are shut down, the data created inside them is also deleted. When Docker generates a new container from an image, the changes made by the user no longer exist. There are some predefined settings in every Docker image, and the file containing them Dockerfile appears as. To represent the space to be used on the disk, located in the Dockerfile volume and its definition is one of these settings and if it is explicitly defined in the container generated from the image, data on disk permanent contains a pre-definition of an area where it can be stored as. Even if the container is stopped and deleted, the data created in the container can continue to exist in the Docker file system thanks to the defined volume. In summary, while “epileptik” (note: contextually means mortal/ephemeral, translating as ephemeral/mortal based on standard Docker terminology) Docker containers destroy their data along with themselves when stopped, they also offer the option to store data on disk upon the user's request. In such cases, a volume must be defined for the data we want to persist in the container.
Just as we can decide on the volume's name and location, we can also leave these assignment tasks to Docker. However, Docker generally does not give “human-friendly” names to automatically named volumes. If we want to easily manage this process and its aftermath, we can define the disk space the container will use ourselves, thereby determining both the disk space to be used and the name Docker will use for the volume. Docker allows two different usages for defining disk space: Data Volume and Bind Mount. In this article, we will only give examples through Data Volume.
The image file for our PostgreSQL 11 version was pulled from the Docker Registry and run. If the running container image docker container inspect’if we query the disk space used by the container Volumes we will see in the title. Here, additionally Mounts The source and destination tags in this section are also important for seeing which directory in the operating system is mapped (linked) to which directory in the container.
[profelis@profelis ~]$ docker container inspect pg11
…
“Volumes”: {
“/var/lib/postgresql/data”: {}
},
…
“Mounts”: [
{
“Type”: “volume”,
“Name”: “pg11data”,
“Source”: “/var/lib/docker/volumes/pg11data/_data”,
“Destination”: “/var/lib/postgresql/data”,
“Driver”: “local”,
“Mode”: “z”,
“RW”: true,
“Propagation”: “”
}
],
If a container is run without using the -v flag, Docker will run this container by creating a data directory with a hash name under /var/lib/docker/volumes/. For example, let's create another PostgreSQL server for testing purposes and not define a volume. In this case, as we mentioned above, we will encounter Docker's “non-human” naming convention, which makes it difficult for us to track volumes.
[omen@omen docker]$ docker run -d -e POSTGRES_PASSWORD=pass –name pg_test postgres:11
2aee185beb35ecd78406bf8578301ffef11d9fdea69854bc51c16c03b8e521d5
[omen@omen docker]$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b891069b8bb postgres:11 “docker-entrypoint.s…” 5 seconds ago Up 4 seconds 5432/tcp pgtest
[omen@omen docker]$ docker volume ls
DRIVER VOLUME NAME
local f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32b9dd4f9ad8a2a
[omen@omen docker]$ docker volume inspect f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32b9dd4f9ad8a2a
[
{
“CreatedAt”: “2020-04-21T07:10:16+03:00”,
“Driver”: “local”,
“Labels”: null,
“Mountpoint”: “/var/lib/docker/volumes/f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc3
2b9dd4f9ad8a2a/_data”,
“Name”: “f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32b9dd4f9ad8a2a”,
“Options”: null,
“Scope”: “local”
}
]To see what is in this volume, we switch to the privileged user, and when we inspect the inside of the volume stored by Docker, we encounter our PostgreSQL data directory.
[omen@omen docker]$ sudo su –
[omen ~]# cd /var/lib/docker/volumes/f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32
b9dd4f9ad8a2a/_data/
[omen _data]# ls -l
total 120
drwx—— 6 999 adm 4096 Apr 17 19:54 base
drwx—— 2 999 adm 4096 Apr 21 03:38 global
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_commit_ts
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_dynshmem
-rw——- 1 999 adm 4535 Apr 17 19:52 pg_hba.conf
-rw——- 1 999 adm 1636 Apr 17 19:52 pg_ident.conf
drwx—— 4 999 adm 4096 Apr 21 03:42 pg_logical
drwx—— 4 999 adm 4096 Apr 17 19:52 pg_multixact
drwx—— 2 999 adm 4096 Apr 21 03:37 pg_notify
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_replslot
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_serial
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_snapshots
drwx—— 2 999 adm 4096 Apr 21 03:37 pg_stat
drwx—— 2 999 adm 4096 Apr 21 07:18 pg_stat_tmp
drwx------ 2 999 adm 4096 Apr 17 19:52 pg_subtrans
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_tblspc
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_twophase
-rw——- 1 999 adm 3 Apr 17 19:52 PG_VERSION
drwx—— 3 999 adm 4096 Apr 17 19:52 pg_wal
drwx—— 2 999 adm 4096 Apr 17 19:52 pg_xact
-rw——- 1 999 adm 88 Apr 17 19:52 postgresql.auto.conf
-rw——- 1 999 adm 23963 Apr 17 19:52 postgresql.conf
-rw------- 1 999 adm 36 Apr 21 03:37 postmaster.opts
-rw——- 1 999 adm 94 Apr 21 03:37 postmaster.pidOver time, the use of some containers may cease. Docker container rm We said that when we delete a container using the command, the volume used by this container is not deleted. The shortest way to clean up all defined volumes that take up space on the disk and are not used by any container docker volume prune is the command. This way, all unused data volumes can be deleted with a single command to free up space on the disk.
[omen _data]# docker volume ls
DRIVER VOLUME NAME
local f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32b9dd4f9ad8a2a
[omen _data]# docker container rm -f wonderful_bassi
wonderful_bassi
[omen _data]# docker volume ls
DRIVER VOLUME NAME
local f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32b9dd4f9ad8a2a
[omen _data]# Docker Volume Prune
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Volumes:
f12cc8cf7eee61af18715ba70e16c9ea8fd167a175c3fbfbc32b9dd4f9ad8a2a
Total reclaimed space: 40.54MB
[omen _data]# docker volume ls
DRIVER VOLUME NAME
Returning to our first example, we had our pg11 container and this -v pg11data:/var/lib/postgresql/data Because we started it with the tag, we had created a data volume named pg11data. Docker maps this directory on the disk to the inside of the container /var/lib/postgresql/data directed to the folder. Thus, for all data generated in the PostgreSQL container, on our disk /var/lib/docker/volumes/pg11data We indicated that we want to use its directory. We can monitor this folder, back it up, or attach a new container to it so that even if this image crashes, it can resume from where it left off with a new PostgreSQL.
[profelis@profelis ~]$ docker volume ls
DRIVER VOLUME NAME
local pg11data
Bir konteyneri çalıştırmadan da imajın kendi datalarını hangi dizinde tutacağını ilgili imajın Dockerfile’ını inceleyerek görebiliriz. Bunun için Docker Hub’da Postgres imaj sayfasında tags sekmesine geçerek üzerinde çalıştığımız sürüm etiketine tıklayalım. Şu linkten PostgreSQL:11 imajının Dockerfile’ına ulaşabilirsiniz. Dosyanın sonlarına doğru VOLUME satırında PostgreSQL imajı konteynerleştirildiğinde verilerini saklayacağı dizini belirtiyor.
Bir data volume tanımlayarak başlattığımız PostgreSQL 11 konteynerine girelim ve PostgreSQL kümesine yeni bir veritabanı açarak tablolar ve veriler ekleyelim. Sonrasında ise bu konteyneri durdurarak silelim. Bakalım verilere ne olacak?
[profelis@profelis ~]$ docker exec -it pg11 bash
root@2b891069b8bb:/# su – postgres
postgres@2b891069b8bb:~$ psql
psql (11.7 (Debian 11.7-2.pgdg90+1))
Type “help” for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype |
———–+———-+———-+————+————+
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
(3 rows)
postgres=# create database profelis;
CREATE DATABASE
postgres=# \c profelis ;
You are now connected to database “profelis” as user “postgres”.
profelis=# create table departman(id serial, name text);
CREATE TABLE
profelis=# insert into departman values (1, ‘muhasebe’), (2, ‘teknik’), (3, ‘yonetim’);
INSERT 0 3
profelis=# select * from departman;
id | name
—-+———-
1 | muhasebe
2 | teknik
3 | yonetim
(3 rows)
profelis=# \q
postgres@2b891069b8bb:~$ logout
root@2b891069b8bb:/# exit
[profelis@profelis ~]$ docker container stop pg11
pg11
[profelis@profelis ~]$ docker container rm pg11
pg11
PostgreSQL 11 durduruldu ve diskten silindi. Şimdi PostgreSQL 11.7 konteynerini, aynı klasörü data volume olarak gösterecek şekilde başlatacağız. Yani ilk veritabanımızda oluşturduğumuz verileri, PostgreSQL’i upgrade ettiğimizde hala görmeyi bekliyoruz.
[profelis@profelis ~]$ docker run –name pg117 -d -e POSTGRES_PASSWORD=pass -v pg11data:/var/lib/postgresql/data postgres:11.7
Unable to find image ‘postgres:11.7’ locally
11.7: Pulling from library/postgres
Digest: sha256:32f2b2b831bcc5318183eb09915c97552f7c2a9657b225f9ea0d23ed727271bb
Status: Downloaded newer image for postgres:11.7
514d87da4305b895c044df3612b801dfb079ac84f2333055b0105fd8c09aab3c
[profelis@profelis ~]$ docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b891069b8bb postgres:11 “docker-entrypoint.s…” 43 seconds ago Up 42 seconds 5432/tcp pg117
[profelis@profelis ~]$ docker exec -ti pg117 bash
root@514d87da4305:/# su – postgres
postgres@514d87da4305:~$ psql
psql (11.7 (Debian 11.7-2.pgdg90+1))
Type “help” for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | privileges
———–+———-+———-+————+————+
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
profelis | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
(4 rows)
postgres=# \c profelis
You are now connected to database “profelis” as user “postgres”.
profelis=# select * from departman;
id | name
—-+———-
1 | muhasebe
2 | teknik
3 | yonetim
(3 rows)
Burada farkındaysanız PostgreSQL üzerinde küçük bir güncelleme işlemini Docker konteynerlerinin bize sunduğu yapı sayesinde yapmış olduk. PostgreSQL özelinde büyük sürüm değişikliklerini (PostgreSQL 10’dan 11’e gibi) bu şekilde yapmamız mümkün değil. Çünkü büyük sürüm değişikliklerinde PostgreSQL’in dahili dosya depolama formatı da (muhtemelen) değiştirildiği için yalnız veri dizini değil, PostgreSQL’in yapısı da farklılık gösterecektir. PostgreSQL’de bu tür ana sürüm güncellemelerinde resmi dokümantasyonu da esas alarak farklı araçlar ve yöntemlerle sürüm güncelleme sürecini tamamlayabilirsiniz.
Yazının sonuna geldiğimizde artık Docker’ın diskte kullandığı alanı nasıl bulacağımızı ve PostgreSQL özelinde bu alandaki verileri nasıl kullanabileceğimizi biliyoruz. Sonraki yazıda Docker network komutlarına bakacağız. Yazı dizimizi takip edin!
