marko/docker-development-youtube-series
1
0
Fork 0
mirror of https://github.com/marcel-dempers/docker-development-youtube-series.git synced 2025-06-06 17:01:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

nginx demo and features

Browse Source
This commit is contained in:
marcel-dempers 2022-11-14 13:24:50 +11:00
parent 4095c3050a
commit dc163cf341
8 changed files with 615 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

277
kubernetes/ingress/controller/nginx/README.md
View File

@ -125,16 +125,279 @@ If you are running in the cloud, you will get a real IP address. </br>
kubectl -n ingress-nginx port-forward svc/ingress-nginx-controller 443
```
We can reach our controller on [https://localhost/](https://localhost/)
We can reach our controller on [https://localhost/](https://localhost/) </br>
It's important to understand that Ingress runs on two ports `80` and `443` </br>
NGINX Ingress creates a fake certificate which is served for default `HTTPS` traffic on port `443`. </br>
If you look in the browser you will notice the name of the certificate `Common Name (CN) Kubernetes Ingress Controller Fake Certificate`
## Features
* Routing DOMAIN
* SSL terminating & passthrough
Now before we take a look at the features we'll need two web applications that we can use as our test harness, `service-a` and `service-b` </br>
* routing URL
In this demo, i have a deployment that runs a pod and a service that exposes the pod on port 80. </br>
This is a typical scenario where you have a micrservice you want to expose publicly. </br>
customization (configmap)
### Deploy Service A & B
* location snippet
* log formating
Will deploy these two apps to the default namespace:
```
kubectl apply -f ./kubernetes/ingress/controller/nginx/features/service-a.yaml
kubectl apply -f ./kubernetes/ingress/controller/nginx/features/service-b.yaml
```
Test our service : `kubectl port-forward svc/service-a 80`
Our services accept traffic on:
* `http://localhost/` which goes to the root `/`
* `http://localhost/path-a.html` which goes to the root `/path-a.html`
* `http://localhost/path-b.html` which goes to the root `/path-b.html`
* `http://localhost/<any-other-path>.html` which goes to the root `404`
### Routing by Domain
The most common way to route traffic with ingress is by domain:
* https://public.service-a.com/ --> Ingress --> k8s service --> http://service-a/
* https://public.service-b.com/ --> Ingress --> k8s service --> http://service-b/
To showcase this, let's deploy an ingress for service-a and service-b that routes by domain. </br>
<i>Note: we don't own public domain `public.service-a.com` so we're using a `/etc/hosts` file</i>
Example Ingress:
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
spec:
ingressClassName: nginx
rules:
- host: public.service-a.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
```
<i>Note: we don't own public domain `public.my-services.com` so we're using a `/etc/hosts` file</i>
Deploy our ingresses:
```
kubectl apply -f ./kubernetes/ingress/controller/nginx/features/routing-by-domain.yaml
```
Now we can access service-a and service-b on:
* https://public.service-a.com/
* https://public.service-b.com/
### Routing by Path
Another popular routing strategy is to use a shared domain and route based on the HTTP path. For example: </br>
* https://public.my-services.com/path-a --> Ingress --> k8s service --> http://service-a/path-a
* https://public.my-services.com/path-b --> Ingress --> k8s service --> http://service-b/path-b
This way public path `/path-a` will hit our application on `/path-a` </br>
Example Ingress:
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-a
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
```
Deploy our ingresses:
```
kubectl apply -f ./kubernetes/ingress/controller/nginx/features/routing-by-path.yaml
```
Now notice the following routing:
* https://public.my-services.com/ --> Ingress (404)
* https://public.my-services.com/path-a --> Ingress --> k8s service --> http://service-a/
* https://public.my-services.com/path-b --> Ingress --> k8s service --> http://service-b/
No matter what path you place on the front end, as long as the path matches `/path-a` or `/path-b`
it will be routed to the correct service on `/` </br>
It's important to note that no extra paths or querystrings will NOT be passed to the upstream </br>
We can see this by looking at our NGINX Ingress controller logs as the controller will write the path it sees as well as the upstream service where it sent the request
```
kubectl -n ingress-nginx logs -l app.kubernetes.io/instance=ingress-nginx
```
### App Root
Sometimes applications have different root paths and don't simply serve traffic on `/` </br>
For example, the base path may be `http://localhost/home` </br>
To tell the Ingress controller that our application root path is `/home`, we can set the annotation `nginx.ingress.kubernetes.io/app-root: /home` </br>
This means the controller will be aware that all traffic that matches `path-a` should go to `/home` on service-a. </br>
### URL Rewrite
We saw earlier when we routed by path, that we could pass `/path-a` to service-a and `/path-b` to service-b. </br>
However, the traffic would always go to `/` so we lost any trailing URL, parameters and querystring. </br>
Not very useful. </br>
To allow the Ingress controller to pass paths to the upstream you need to look into [Rewrite Configuration](https://kubernetes.github.io/ingress-nginx/examples/rewrite/)
Example Ingress:
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-a(/|$)(.*)
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
```
Deploy our ingresses:
```
kubectl apply -f ./kubernetes/ingress/controller/nginx/features/routing-by-path-rewrite.yaml
```
Now notice the following routing:
* https://public.my-services.com/ --> Ingress (404)
* https://public.my-services.com/path-a* --> Ingress --> k8s service --> http://service-a/*
* https://public.my-services.com/path-b* --> Ingress --> k8s service --> http://service-b/*
```
kubectl -n ingress-nginx logs -l app.kubernetes.io/instance=ingress-nginx
```
It's important to study the logs of the Ingress Controller to learn what path it saw, where it routed to
```
127.0.0.1 - - [13/Nov/2022:02:17:47 +0000] "GET /path-a/path.html HTTP/2.0" 404 19 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" 485 0.000 [default-service-a-80] [] 10.244.0.8:80 19 0.000 404 206ed4b88b712564fc073c3adb845dff
```
In the above case, the controller saw ` /path-a/path.html` , routed to service-a and we can see what our service-a saw, by looking at its logs:
```
kubectl logs -l app=service-a
10.244.0.7 - - [13/Nov/2022:02:28:36 +0000] "GET /path-a.html HTTP/1.1" 200 28 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
```
### SSL terminating & passthrough
As we noticed by logs, its default for the Ingress controller to offload SSL. </br>
We can see this because when it routes to upstreams, it routes to our service on port 80 </br>
Ingress offloads the TLS connection and creates a new connection with its upstream. </br>
This is a common approach to offload TLS on the edge as internal traffic is generally unencrypted in private
networks especially in large microservice environments where security is tightened in other manners so TLS is not needed all the way through. </br>
We can enable SSL pass through with the annotation: `nginx.ingress.kubernetes.io/ssl-passthrough`. </br>
SSL Passthrough is disabled by default and requires starting the controller with the --enable-ssl-passthrough flag. </br>
### IP Whitelist
We can add a layer of protection to our services that are exposed by an ingress. </br>
One popular way is IP whitelisting. </br>
This can be done with a [whitelist source range annotation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#whitelist-source-range) for example: </br>
`nginx.ingress.kubernetes.io/whitelist-source-range: <ip,ip,ip>`</br>
You can set this globally if you want using the [Customization ConfigMap](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#whitelist-source-range). </br>
We'll take a look at this customization in a bit. </br>
### Authentication
You can add a layer of protection to services exposed by ingress by several [Authentication methods](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#authentication). </br>
A simple example is basic Authentication where the client supplied a `username\password` to access our service. </br>
This is controlled by annotations:
* `nginx.ingress.kubernetes.io/auth-type: basic`
* `nginx.ingress.kubernetes.io/auth-secret: server-a-secret`
* `nginx.ingress.kubernetes.io/auth-secret-type: auth-file`
Create a username and password:
```
apk add apache2-utils
htpasswd -c auth service-a-user
kubectl create secret generic server-a-secret --from-file=auth
```
Deploy our ingresses:
```
kubectl apply -f ./kubernetes/ingress/controller/nginx/features/basic-auth.yaml
```
### Server snippet
Every ingress is technically an NGINX server block with a NGINX proxy pass. </br>
We can even customise this server block with a [Server Snippet annotation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#server-snippet)
### Customization
As mentioned before, the NGINX Ingress controller can be customized quite heavily with the [ConfigMap](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/)
We can customize log format to JSON as well for example:
```
log-format-escape-json: "true"
log-format-upstream: '{"time":"$time_iso8601","remote_addr":"$remote_addr","proxy_protocol_addr":"$proxy_protocol_addr","proxy_protocol_port":"$proxy_protocol_port","x_forward_for":"$proxy_add_x_forwarded_for","remote_user":"$remote_user","host":"$host","request_method":"$request_method","request_uri":"$request_uri","server_protocol":"$server_protocol","status":$status,"request_time":$request_time,"request_length":$request_length,"bytes_sent":$bytes_sent,"upstream_name":"$proxy_upstream_name","upstream_addr":"$upstream_addr","upstream_uri":"$uri","upstream_response_length":$upstream_response_length,"upstream_response_time":$upstream_response_time,"upstream_status":$upstream_status,"http_referrer":"$http_referer","http_user_agent":"$http_user_agent","http_cookie":"$http_cookie","http_device_id":"$http_x_device_id","http_customer_id":"$http_x_customer_id"}'
```
Apply the changes and restart Ingress:
```
kubectl apply -f ./kubernetes/ingress/controller/nginx/manifests/nginx-ingress.${APP_VERSION}.yaml
```
kubectl -n ingress-nginx logs -l app.kubernetes.io/instance=ingress-nginx

43
kubernetes/ingress/controller/nginx/features/basic-auth.yaml Normal file
View File

@ -0,0 +1,43 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: server-a-secret
nginx.ingress.kubernetes.io/auth-secret-type: auth-file
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-a(/|$)(.*)
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-b
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-b(/|$)(.*)
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
---

36
kubernetes/ingress/controller/nginx/features/routing-by-domain.yaml Normal file
View File

@ -0,0 +1,36 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
spec:
ingressClassName: nginx
rules:
- host: public.service-a.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-b
spec:
ingressClassName: nginx
rules:
- host: public.service-b.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
---

40
kubernetes/ingress/controller/nginx/features/routing-by-path-rewrite.yaml Normal file
View File

@ -0,0 +1,40 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-a(/|$)(.*)
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-b
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-b(/|$)(.*)
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
---

40
kubernetes/ingress/controller/nginx/features/routing-by-path.yaml Normal file
View File

@ -0,0 +1,40 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-a
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-a
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-b
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: public.my-services.com
http:
paths:
- path: /path-b
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
---

92
kubernetes/ingress/controller/nginx/features/service-a.yaml Normal file
View File

@ -0,0 +1,92 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: service-a
data:
path-a.html: |
"/path-a.html" on service-a
path-b.html: |
"/path-b.html" on service-a
index.html: |
"/" on service-a
404.html: |
service-a 404 page
---
apiVersion: v1
kind: ConfigMap
metadata:
name: service-a-nginx.conf
data:
nginx.conf: |
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
sendfile on;
server {
listen 80;
server_name localhost;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: service-a
labels:
app: service-a
spec:
replicas: 1
selector:
matchLabels:
app: service-a
template:
metadata:
labels:
app: service-a
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
volumeMounts:
- name: html
mountPath: "/usr/share/nginx/html/"
- name: config
mountPath: "/etc/nginx/"
volumes:
- name: html
configMap:
name: service-a
- name: config
configMap:
name: service-a-nginx.conf
---
apiVersion: v1
kind: Service
metadata:
name: service-a
spec:
selector:
app: service-a
ports:
- protocol: TCP
port: 80
targetPort: 80

92
kubernetes/ingress/controller/nginx/features/service-b.yaml Normal file
View File

@ -0,0 +1,92 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: service-b
data:
path-a.html: |
"/path-a.html" on service-b
path-b.html: |
"/path-b.html" on service-b
index.html: |
"/" on service-b
404.html: |
service-b 404 page
---
apiVersion: v1
kind: ConfigMap
metadata:
name: service-b-nginx.conf
data:
nginx.conf: |
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
sendfile on;
server {
listen 80;
server_name localhost;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: service-b
labels:
app: service-b
spec:
replicas: 1
selector:
matchLabels:
app: service-b
template:
metadata:
labels:
app: service-b
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
volumeMounts:
- name: html
mountPath: "/usr/share/nginx/html/"
- name: config
mountPath: "/etc/nginx/"
volumes:
- name: html
configMap:
name: service-b
- name: config
configMap:
name: service-b-nginx.conf
---
apiVersion: v1
kind: Service
metadata:
name: service-b
spec:
selector:
app: service-b
ports:
- protocol: TCP
port: 80
targetPort: 80

2
kubernetes/ingress/controller/nginx/manifests/nginx-ingress.1.5.1.yaml
View File

@ -31,6 +31,8 @@ metadata:
namespace: ingress-nginx
data:
allow-snippet-annotations: "true"
log-format-escape-json: "true"
log-format-upstream: '{"time":"$time_iso8601","remote_addr":"$remote_addr","proxy_protocol_addr":"$proxy_protocol_addr","proxy_protocol_port":"$proxy_protocol_port","x_forward_for":"$proxy_add_x_forwarded_for","remote_user":"$remote_user","host":"$host","request_method":"$request_method","request_uri":"$request_uri","server_protocol":"$server_protocol","status":$status,"request_time":$request_time,"request_length":$request_length,"bytes_sent":$bytes_sent,"upstream_name":"$proxy_upstream_name","upstream_addr":"$upstream_addr","upstream_uri":"$uri","upstream_response_length":$upstream_response_length,"upstream_response_time":$upstream_response_time,"upstream_status":$upstream_status,"http_referrer":"$http_referer","http_user_agent":"$http_user_agent","http_cookie":"$http_cookie","http_device_id":"$http_x_device_id","http_customer_id":"$http_x_customer_id"}'
---
# Source: ingress-nginx/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1