Ingress Nginx

Ingress Nginx

Posted by BlueFat on Thursday, July 16, 2020

https://kubernetes.github.io/ingress-nginx/

Install

https://kubernetes.github.io/ingress-nginx/deploy/

helm repo add ingress-nginx  https://kubernetes.github.io/ingress-nginx 
helm search repo ingress-nginx
helm pull ingress-nginx/ingress-nginx 
tar xf ingress-nginx-4.3.0.tgz 
cd ingress-nginx

# 修改后
vim values.yaml

# false -> true
hostNetwork: true
# ClusterFirst -> ClusterFirstWithHostNet
dnsPolicy: ClusterFirstWithHostNet
# Deployment -> DaemonSet
kind: DaemonSet
# LoadBalancer -> ClusterIP
type: ClusterIP
# 指定标签 代码行292-294
nodeSelector: 
  nodeSelector:
  kubernetes.io/os: linux
     ingress: "true"

# 修改镜像
registry.aliyuncs.com/google_containers/kube-webhook-certgen:v20220916-gd32f8c343
registry.aliyuncs.com/google_containers/defaultbackend-amd64
registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0

kubectl create ns ingress-nginx
kubectl label node master01 ingress=true
kubectl label node master02 ingress=true
kubectl label node master03 ingress=true
helm install ingress-nginx -n ingress-nginx .

容忍

# 223行 注意有多个tolerations,或者全加
  tolerations: 
    - key: ""
      operator: "Exists"
      effect: "NoSchedule"

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

http proxy

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-demo
  namespace: default
spec:
  ingressClassName: nginx
  rules:
    - host: demo.sundayhk.com
      http:
        paths:
          - pathType: Prefix
            backend:
              service:
                name: nginx-demo
                port:
                  number: 80
            path: /

Annotations 即时更新 ConfigMap 一般需要手动更新

Redirect

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#permanent-redirect

nginx.ingress.kubernetes.io/permanent-redirect: https://www.baidu.com
nginx.ingress.kubernetes.io/permanent-redirect-code: '308'  # 可选,rancher ui不需加引号

Rewrite

https://kubernetes.github.io/ingress-nginx/examples/rewrite/

SSL

HOST=test.sundayhk.com
KEY_FILE=$HOST.key
CERT_FILE=$HOST.pem
CERT_NAME=$HOST
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ${KEY_FILE} -out ${CERT_FILE} -subj "/CN=${HOST}/O=${HOST}"

kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-prod
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: web.sundayhk.com
    http:
      paths:
      - backend:
          service:
            name: web-prod
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - web.sundayhk.com
    secretName: web.sundayhk.com

禁止http自动跳转https

nginx.ingress.kubernetes.io/ssl-redirect: "false"

ingress https直通 kubernetes-dashboard使用service ssl

#
nginx.ingress.kubernetes.io/ssl-passthrough: "true"

Limit

黑白名单:

  • Annotations: 只对指定的ingress生效
  • ConfigMap: 全局生效

configmap 默认为滚动升级不安全,设置为手动删除pod才生效 Ingress imagePolicy: OnDelete

白名单

whitelist-source-range

nginx.ingress.kubernetes.io/whitelist-source-range: 127.0.0.1,192.168.10.250

黑名单

server-snippet

单个域名ingress限制IP

nginx.ingress.kubernetes.io/server-snippet: deny 192.168.1.171; allow all;

所有域名ingress限制 在ingress-nginx configmap配置

block-cidrs

data:
  block-cidrs: ip/cidrs
kubectl edit configmap -n ingress-nginx ingress-nginx-controller 

# data 配置keys: values
apiVersion: v1
data:
  allow-snippet-annotations: "true"
  block-cidrs: 192.168.10.0/24

Rate-limiting

rate-limiting

Rewrite

rewrite

Cancary

Nginx Annotations 支持以下 4 种 Canary 规则:

  • nginx.ingress.kubernetes.io/canary-by-header:基于 Request Header 的流量切分,适用于灰度发布以及 A/B 测试。当 Request Header 设置为 always时,请求将会被一直发送到 Canary 版本;当 Request Header 设置为 never时,请求不会被发送到 Canary 入口;对于任何其他 Header 值,将忽略 Header,并通过优先级将请求与其他金丝雀规则进行优先级的比较。
  • nginx.ingress.kubernetes.io/canary-by-header-value:要匹配的 Request Header 的值,用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时,它将被路由到 Canary 入口。该规则允许用户自定义 Request Header 的值,必须与上一个 annotation (即:canary-by-header) 一起使用。
  • nginx.ingress.kubernetes.io/canary-weight:基于服务权重的流量切分,适用于蓝绿部署,权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求,权重为 100 意味着所有请求都将被发送到 Canary 入口。
  • nginx.ingress.kubernetes.io/canary-by-cookie:基于 cookie 的流量切分,适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时,它将被路由到 Canary 入口;当 cookie 值设置为 never时,请求不会被发送到 Canary 入口;对于任何其他值,将忽略 cookie 并将请求与其他金丝雀规则进行优先级的比较。

注意:金丝雀规则按优先顺序进行如下排序:

canary-by-header - > canary-by-cookie - > canary-weight
nginx.ingress.kubernetes.io/canary    "true" or "false"
nginx.ingress.kubernetes.io/canary-by-header    string
nginx.ingress.kubernetes.io/canary-by-header-value    string
nginx.ingress.kubernetes.io/canary-by-header-pattern    string
nginx.ingress.kubernetes.io/canary-by-cookie    string
nginx.ingress.kubernetes.io/canary-weight    number

正式版本

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-prod
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: web.sundayhk.com
    http:
      paths:
      - backend:
          service:
            name: web-prod
            port:
              number: 80
        path: /
        pathType: Prefix

灰度版本

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-gray
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: canary # value 自定义
    nginx.ingress.kubernetes.io/canary-by-header-value: "true" # value 自定义
spec:
  ingressClassName: nginx
  rules:
  - host: web.sundayhk.com
    http:
      paths:
      - backend:
          service:
            name: web-gray
            port:
              number: 80
        path: /
        pathType: Prefix

测试

curl http://web.sundayhk.com
hello web prod@!!!

curl -H "canary: true" http://web.sundayhk.com
hello web gray@xxx

阿里云 Nginx Ingress高级用法 Nginx Ingress异常问题排查