marko/docker-development-youtube-series
1
0
Fork 0
mirror of https://github.com/marcel-dempers/docker-development-youtube-series.git synced 2025-06-06 17:01:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

postgres k8s wip

Browse Source
This commit is contained in:
marcel-dempers 2022-10-29 13:55:46 +11:00
parent 6a30902a63
commit 27ff6f801c
17 changed files with 2147 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
storage/databases/postgresql/3-replication/README.md
View File

@ -68,7 +68,7 @@ Hence it is "writing ahead". </br>
More documentation for configuration [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) and [max_wal_senders](https://www.postgresql.org/docs/current/runtime-config-replication.html)
```
wal_level = hot_standby
wal_level = replica
max_wal_senders = 3
```
@ -184,3 +184,8 @@ CREATE TABLE customers (firstname text, customer_id serial, date_created timesta
That's it for chapter three! </br>
Now we understand how to [run PostgreSQL](../1-introduction/README.md), how to [configure PostgreSQL](../2-configuration/README.md) and how to setup replication for better availability.
## Summary
<img src="./summary.png" alt="Summary">

2
storage/databases/postgresql/3-replication/postgres-1/config/postgresql.conf
View File

@ -27,7 +27,7 @@ lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = hot_standby
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /mnt/server/archive/%f && cp %p /mnt/server/archive/%f'
max_wal_senders = 3

BIN
storage/databases/postgresql/3-replication/summary.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 30 KiB

148
storage/databases/postgresql/4-k8s-basic/README.md Normal file
View File

@ -0,0 +1,148 @@
# Running PostgreSQL in Kubernetes (Primary and Standby)
## Create a cluster
```
kind create cluster --name postgresql --image kindest/node:v1.23.5
kubectl get nodes
NAME STATUS ROLES AGE VERSION
postgresql-control-plane Ready control-plane,master 31s v1.23.5
```
## Deploy our first PostgreSQL instance
Deploy a namespace to hold our resources:
```
kubectl create ns postgresql
```
Create our secret for our first PostgreSQL instance:
```
kubectl -n postgresql create secret generic postgresql `
--from-literal POSTGRES_USER="postgresadmin" `
--from-literal POSTGRES_PASSWORD='admin123' `
--from-literal POSTGRES_DB="postgresdb" `
--from-literal REPLICATION_USER="replicationuser" `
--from-literal REPLICATION_PASSWORD='replicationPassword'
```
Deploy our first instance which will act as our primary:
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-1.yaml
```
## Check our installation
```
kubectl -n postgresql get pods
# check the initialization logs (should be clear)
kubectl -n postgresql logs postgres-1-0 -c init
# check the database logs
kubectl -n postgresql logs postgres-1-0
kubectl -n postgresql exec -it postgres-1-0 -- bash
# login to postgres
psql --username=postgresadmin postgresdb
# see our replication user created
\du
#create a table
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
#show the table
\dt
# quit out of postgresql
\q
# check the data directory
ls -l /data/pgdata
# check the archive
ls -l /data/archive
```
## Deploy our Standby Server
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-2.yaml
```
runuser -u postgres -- pg_ctl reload
## Failover
Now lets say `postgres-1` fails. </br>
PostgreSQL does not have built-in automated failver and recovery and requires tooling to perform this. </br>
When `postgres-1` fails, we would use a utility called [pg_ctl](https://www.postgresql.org/docs/current/app-pg-ctl.html) to promote our stand-by server to a new primary server. </br>
Then we have to build a new stand-by server just like we did in this guide. </br>
We would also need to configure replication on the new primary, the same way we did in this guide. </br>
Let's stop the primary server to simulate failure:
```
kubectl -n postgresql delete sts postgres-1
# notice the failure in replication from postgres-1
kubectl -n postgresql logs postgres-2-0
```
Then log into `postgres-2` and promote it to primary:
```
kubectl -n postgresql exec -it postgres-2-0 -- bash
psql --username=postgresadmin postgresdb
# confirm we cannot create a table as its a stand-by server
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
# quit out of postgresql
\q
# run pg_ctl as postgres user (cannot be run as root!)
runuser -u postgres -- pg_ctl promote
# confirm we can create a table as its a primary server
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
```
## Setup Replication on the new Primary
```
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
```
Reconfigure the `postgres-2` instance:
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-2.yaml
# check our new instance
kubectl -n postgresql get pods
kubectl -n postgresql logs postgres-2-0 -c init
kubectl -n postgresql logs postgres-2-0
kubectl -n postgresql exec -it postgres-2-0 -- bash
```
That's it for chapter three! </br>
Now we understand how to [run PostgreSQL](../1-introduction/README.md), how to [configure PostgreSQL](../2-configuration/README.md) and how to setup replication for better availability.

188
storage/databases/postgresql/4-k8s-basic/yaml/postgres-1.yaml Normal file
View File

@ -0,0 +1,188 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-1
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-1
spec:
serviceName: postgres-1
replicas: 1
selector:
matchLabels:
app: postgres-1
template:
metadata:
labels:
app: postgres-1
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "off"
- name: PRIMARY_SERVER_ADDRESS
value: ""
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-1
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-1
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-1

188
storage/databases/postgresql/4-k8s-basic/yaml/postgres-2.yaml Normal file
View File

@ -0,0 +1,188 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-2
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-2
spec:
serviceName: postgres-2
replicas: 1
selector:
matchLabels:
app: postgres-2
template:
metadata:
labels:
app: postgres-2
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "off"
- name: PRIMARY_SERVER_ADDRESS
value: "postgres-1"
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-2
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-2
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-2

189
storage/databases/postgresql/4-k8s-basic/yaml/postgres-3.yaml Normal file
View File

@ -0,0 +1,189 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-4
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
#replication
wal_level = replica
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-4
spec:
serviceName: postgres-4
replicas: 1
selector:
matchLabels:
app: postgres-4
template:
metadata:
labels:
app: postgres-4
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "on"
- name: PRIMARY_SERVER_ADDRESS
value: "postgres-2"
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-4
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-4
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-4

0
storage/databases/postgresql/4-kubernetes/README.md
View File

148
storage/databases/postgresql/5-k8s-replication/README.md Normal file
View File

@ -0,0 +1,148 @@
# Running PostgreSQL in Kubernetes (Primary and Standby)
## Create a cluster
```
kind create cluster --name postgresql --image kindest/node:v1.23.5
kubectl get nodes
NAME STATUS ROLES AGE VERSION
postgresql-control-plane Ready control-plane,master 31s v1.23.5
```
## Deploy our first PostgreSQL instance
Deploy a namespace to hold our resources:
```
kubectl create ns postgresql
```
Create our secret for our first PostgreSQL instance:
```
kubectl -n postgresql create secret generic postgresql `
--from-literal POSTGRES_USER="postgresadmin" `
--from-literal POSTGRES_PASSWORD='admin123' `
--from-literal POSTGRES_DB="postgresdb" `
--from-literal REPLICATION_USER="replicationuser" `
--from-literal REPLICATION_PASSWORD='replicationPassword'
```
Deploy our first instance which will act as our primary:
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-1.yaml
```
## Check our installation
```
kubectl -n postgresql get pods
# check the initialization logs (should be clear)
kubectl -n postgresql logs postgres-1-0 -c init
# check the database logs
kubectl -n postgresql logs postgres-1-0
kubectl -n postgresql exec -it postgres-1-0 -- bash
# login to postgres
psql --username=postgresadmin postgresdb
# see our replication user created
\du
#create a table
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
#show the table
\dt
# quit out of postgresql
\q
# check the data directory
ls -l /data/pgdata
# check the archive
ls -l /data/archive
```
## Deploy our Standby Server
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-2.yaml
```
runuser -u postgres -- pg_ctl reload
## Failover
Now lets say `postgres-1` fails. </br>
PostgreSQL does not have built-in automated failver and recovery and requires tooling to perform this. </br>
When `postgres-1` fails, we would use a utility called [pg_ctl](https://www.postgresql.org/docs/current/app-pg-ctl.html) to promote our stand-by server to a new primary server. </br>
Then we have to build a new stand-by server just like we did in this guide. </br>
We would also need to configure replication on the new primary, the same way we did in this guide. </br>
Let's stop the primary server to simulate failure:
```
kubectl -n postgresql delete sts postgres-1
# notice the failure in replication from postgres-1
kubectl -n postgresql logs postgres-2-0
```
Then log into `postgres-2` and promote it to primary:
```
kubectl -n postgresql exec -it postgres-2-0 -- bash
psql --username=postgresadmin postgresdb
# confirm we cannot create a table as its a stand-by server
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
# quit out of postgresql
\q
# run pg_ctl as postgres user (cannot be run as root!)
runuser -u postgres -- pg_ctl promote
# confirm we can create a table as its a primary server
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
```
## Setup Replication on the new Primary
```
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
```
Reconfigure the `postgres-2` instance:
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-2.yaml
# check our new instance
kubectl -n postgresql get pods
kubectl -n postgresql logs postgres-2-0 -c init
kubectl -n postgresql logs postgres-2-0
kubectl -n postgresql exec -it postgres-2-0 -- bash
```
That's it for chapter three! </br>
Now we understand how to [run PostgreSQL](../1-introduction/README.md), how to [configure PostgreSQL](../2-configuration/README.md) and how to setup replication for better availability.

188
storage/databases/postgresql/5-k8s-replication/yaml/postgres-1.yaml Normal file
View File

@ -0,0 +1,188 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-1
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-1
spec:
serviceName: postgres-1
replicas: 1
selector:
matchLabels:
app: postgres-1
template:
metadata:
labels:
app: postgres-1
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "off"
- name: PRIMARY_SERVER_ADDRESS
value: ""
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-1
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-1
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-1

188
storage/databases/postgresql/5-k8s-replication/yaml/postgres-2.yaml Normal file
View File

@ -0,0 +1,188 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-2
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-2
spec:
serviceName: postgres-2
replicas: 1
selector:
matchLabels:
app: postgres-2
template:
metadata:
labels:
app: postgres-2
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "off"
- name: PRIMARY_SERVER_ADDRESS
value: "postgres-1"
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-2
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-2
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-2

189
storage/databases/postgresql/5-k8s-replication/yaml/postgres-3.yaml Normal file
View File

@ -0,0 +1,189 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-4
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
#replication
wal_level = replica
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-4
spec:
serviceName: postgres-4
replicas: 1
selector:
matchLabels:
app: postgres-4
template:
metadata:
labels:
app: postgres-4
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "on"
- name: PRIMARY_SERVER_ADDRESS
value: "postgres-2"
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-4
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-4
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-4

148
storage/databases/postgresql/6-k8s-reference/README.md Normal file
View File

@ -0,0 +1,148 @@
# Running PostgreSQL in Kubernetes (Primary and Standby)
## Create a cluster
```
kind create cluster --name postgresql --image kindest/node:v1.23.5
kubectl get nodes
NAME STATUS ROLES AGE VERSION
postgresql-control-plane Ready control-plane,master 31s v1.23.5
```
## Deploy our first PostgreSQL instance
Deploy a namespace to hold our resources:
```
kubectl create ns postgresql
```
Create our secret for our first PostgreSQL instance:
```
kubectl -n postgresql create secret generic postgresql `
--from-literal POSTGRES_USER="postgresadmin" `
--from-literal POSTGRES_PASSWORD='admin123' `
--from-literal POSTGRES_DB="postgresdb" `
--from-literal REPLICATION_USER="replicationuser" `
--from-literal REPLICATION_PASSWORD='replicationPassword'
```
Deploy our first instance which will act as our primary:
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-1.yaml
```
## Check our installation
```
kubectl -n postgresql get pods
# check the initialization logs (should be clear)
kubectl -n postgresql logs postgres-1-0 -c init
# check the database logs
kubectl -n postgresql logs postgres-1-0
kubectl -n postgresql exec -it postgres-1-0 -- bash
# login to postgres
psql --username=postgresadmin postgresdb
# see our replication user created
\du
#create a table
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
#show the table
\dt
# quit out of postgresql
\q
# check the data directory
ls -l /data/pgdata
# check the archive
ls -l /data/archive
```
## Deploy our Standby Server
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-2.yaml
```
runuser -u postgres -- pg_ctl reload
## Failover
Now lets say `postgres-1` fails. </br>
PostgreSQL does not have built-in automated failver and recovery and requires tooling to perform this. </br>
When `postgres-1` fails, we would use a utility called [pg_ctl](https://www.postgresql.org/docs/current/app-pg-ctl.html) to promote our stand-by server to a new primary server. </br>
Then we have to build a new stand-by server just like we did in this guide. </br>
We would also need to configure replication on the new primary, the same way we did in this guide. </br>
Let's stop the primary server to simulate failure:
```
kubectl -n postgresql delete sts postgres-1
# notice the failure in replication from postgres-1
kubectl -n postgresql logs postgres-2-0
```
Then log into `postgres-2` and promote it to primary:
```
kubectl -n postgresql exec -it postgres-2-0 -- bash
psql --username=postgresadmin postgresdb
# confirm we cannot create a table as its a stand-by server
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
# quit out of postgresql
\q
# run pg_ctl as postgres user (cannot be run as root!)
runuser -u postgres -- pg_ctl promote
# confirm we can create a table as its a primary server
CREATE TABLE customers (firstname text, customer_id serial, date_created timestamp);
```
## Setup Replication on the new Primary
```
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
```
Reconfigure the `postgres-2` instance:
```
kubectl -n postgresql apply -f storage/databases/postgresql/4-kubernetes/yaml/postgres-2.yaml
# check our new instance
kubectl -n postgresql get pods
kubectl -n postgresql logs postgres-2-0 -c init
kubectl -n postgresql logs postgres-2-0
kubectl -n postgresql exec -it postgres-2-0 -- bash
```
That's it for chapter three! </br>
Now we understand how to [run PostgreSQL](../1-introduction/README.md), how to [configure PostgreSQL](../2-configuration/README.md) and how to setup replication for better availability.

188
storage/databases/postgresql/6-k8s-reference/yaml/postgres-1.yaml Normal file
View File

@ -0,0 +1,188 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-1
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-1
spec:
serviceName: postgres-1
replicas: 1
selector:
matchLabels:
app: postgres-1
template:
metadata:
labels:
app: postgres-1
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "off"
- name: PRIMARY_SERVER_ADDRESS
value: ""
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-1
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-1
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-1

188
storage/databases/postgresql/6-k8s-reference/yaml/postgres-2.yaml Normal file
View File

@ -0,0 +1,188 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-2
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
#replication
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-2
spec:
serviceName: postgres-2
replicas: 1
selector:
matchLabels:
app: postgres-2
template:
metadata:
labels:
app: postgres-2
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "off"
- name: PRIMARY_SERVER_ADDRESS
value: "postgres-1"
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-2
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-2
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-2

189
storage/databases/postgresql/6-k8s-reference/yaml/postgres-3.yaml Normal file
View File

@ -0,0 +1,189 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-4
data:
pg_hba.conf: |+
# TYPE DATABASE USER ADDRESS METHOD
host replication replicationuser 0.0.0.0/0 md5
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
host all all all scram-sha-256
postgresql.conf: |+
data_directory = '/data/pgdata'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'
port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'
#locale settings
lc_messages = 'en_US.utf8' # locale for system error message
lc_monetary = 'en_US.utf8' # locale for monetary formatting
lc_numeric = 'en_US.utf8' # locale for number formatting
lc_time = 'en_US.utf8' # locale for time formatting
default_text_search_config = 'pg_catalog.english'
archive_mode = on
archive_command = 'test ! -f /data/archive/%f && cp %p /data/archive/%f'
#replication
wal_level = replica
max_wal_senders = 3
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-4
spec:
serviceName: postgres-4
replicas: 1
selector:
matchLabels:
app: postgres-4
template:
metadata:
labels:
app: postgres-4
spec:
initContainers:
- name: init
image: postgres:15.0
command: [ "bash", "-c" ]
args:
- |
if [ ${STANDBY_MODE} == "on" ];
then
# initialize from backup if data dir is empty
if [ -z "$(ls -A ${PGDATA})" ]; then
export PGPASSWORD=${REPLICATION_PASSWORD}
pg_basebackup -h ${PRIMARY_SERVER_ADDRESS} -p 5432 -U ${REPLICATION_USER} -D ${PGDATA} -Fp -Xs -R
fi
else
#create archive directory
mkdir -p /data/archive && chown -R 999:999 /data/archive
#create a init template
echo "CREATE USER #REPLICATION_USER REPLICATION LOGIN ENCRYPTED PASSWORD '#REPLICATION_PASSWORD';" > init.sql
# add credentials
sed -i 's/#REPLICATION_USER/'${REPLICATION_USER}'/g' init.sql
sed -i 's/#REPLICATION_PASSWORD/'${REPLICATION_PASSWORD}'/g' init.sql
mkdir -p /docker-entrypoint-initdb.d/
cp init.sql /docker-entrypoint-initdb.d/init.sql
fi
env:
- name: STANDBY_MODE
value: "on"
- name: PRIMARY_SERVER_ADDRESS
value: "postgres-2"
- name: PGDATA
value: "/data/pgdata"
- name: REPLICATION_USER
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_USER
optional: false
- name: REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: REPLICATION_PASSWORD
optional: false
volumeMounts:
- mountPath: /docker-entrypoint-initdb.d
name: initdb
- name: data
mountPath: /data
readOnly: false
containers:
- name: postgres
image: postgres:15.0
args: ["-c", "config_file=/config/postgresql.conf"]
env:
- name: PGDATA
value: "/data/pgdata"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_USER
optional: false
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_PASSWORD
optional: false
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql
key: POSTGRES_DB
optional: false
ports:
- containerPort: 5432
name: database
volumeMounts:
- name: initdb
mountPath: /docker-entrypoint-initdb.d
- name: config
mountPath: /config
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: config
configMap:
name: postgres-4
defaultMode: 0755
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 100Mi
---
apiVersion: v1
kind: Service
metadata:
name: postgres-4
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
name: postgres
selector:
app: postgres-4

2
storage/databases/postgresql/README.md
View File

@ -65,7 +65,7 @@ host replication replicationUser 0.0.0.0/0 md5
```
docker exec -it postgres-1 bash
wal_level = hot_standby
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /mnt/server/archive/%f && cp %p /mnt/server/archive/%f'
max_wal_senders = 3