一文讀懂Ingress-Nginx的實(shí)踐

更新時(shí)間：2024年11月04日 09:34:58 作者：StevenZeng學(xué)堂

Ingress-Nginx是Kubernetes中管理HTTP和HTTPS流量的重要工具,本文深入探討Ingress-Nginx工作原理、配置及最佳實(shí)踐,文中通過(guò)示例代碼介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友們下面隨著小編來(lái)一起學(xué)習(xí)學(xué)習(xí)吧

?? 摘要: Ingress-Nginx 是 Kubernetes 中用于管理 HTTP 和 HTTPS 流量的強(qiáng)大工具。通過(guò)配置 Ingress 資源，用戶(hù)可以靈活地路由流量到后端服務(wù)，增強(qiáng)了集群的可訪問(wèn)性和可擴(kuò)展性。本文將深入探討 Ingress-Nginx 的工作原理、基本配置及最佳實(shí)踐，并通過(guò)實(shí)際案例來(lái)驗(yàn)證其功能。

1 概念

1.1 什么是Ingress？

Ingress是Kubernetes的一種 API 對(duì)象，用于管理集群內(nèi)服務(wù)的外部訪問(wèn)。Ingress 可以提供從集群外部到集群內(nèi)Service的 HTTP 和 HTTPS 路由，并可以基于域名、路徑等規(guī)則將請(qǐng)求轉(zhuǎn)發(fā)到集群內(nèi)的Service上。

下面是 Ingress 的一個(gè)簡(jiǎn)單示例，可將外部的流量導(dǎo)流到K8S集群的同一 Service，再由Service將流量發(fā)送到后端的Pod：

1.1.1 主要功能：

HTTP/HTTPS 路由: Ingress 允許外部 HTTP 和 HTTPS 請(qǐng)求根據(jù)預(yù)定義的路由規(guī)則訪問(wèn)集群內(nèi)部的服務(wù)。路由規(guī)則可以基于主機(jī)名、URL 路徑等進(jìn)行配置。
反向代理: Ingress 控制器通常會(huì)充當(dāng)反向代理，將外部請(qǐng)求路由到集群內(nèi)部的服務(wù)。Ingress 不會(huì)公開(kāi)任意端口或協(xié)議，通常使用Service.Type=NodePort或Service.Type=LoadBalancer類(lèi)型的服務(wù)。
域名支持: 通過(guò) Ingress，可以為集群中的服務(wù)配置域名，從而提供更友好的訪問(wèn)方式。例如，可以將 foo.example.com 指向集群內(nèi)的 foo-service 服務(wù)。
負(fù)載均衡: Ingress 控制器可以將流量分配給多個(gè)服務(wù)實(shí)例，實(shí)現(xiàn)負(fù)載均衡，從而提高服務(wù)的可用性和性能。在云環(huán)境中，Ingress 通常會(huì)與云提供商的負(fù)載均衡器集成，以確保流量能夠從外部訪問(wèn)集群。
TLS/SSL 支持: Ingress 支持 HTTPS 協(xié)議，允許為服務(wù)配置 TLS 證書(shū)，以確保傳輸?shù)陌踩?。TLS 證書(shū)可以使用 Kubernetes 的 Secret 對(duì)象進(jìn)行管理。
虛擬主機(jī): Ingress 可以配置虛擬主機(jī)，使得多個(gè)服務(wù)可以通過(guò)不同的域名或子域名進(jìn)行訪問(wèn)。

1.2 Ingress的組件

Ingress 控制器: 為了使 Ingress 資源正常工作，集群必須運(yùn)行一個(gè) Ingress 控制器。類(lèi)似Kubernetes集群的kube-apiserver服務(wù)，負(fù)責(zé)管理和執(zhí)行 Ingress 資源定義的路由規(guī)則。常見(jiàn)的 Ingress 控制器包括 NGINX Ingress Controller、Traefik、HAProxy 等，還有公有云廠商的Ingress 控制器包括Azure的AKS Application Gateway、阿里云的Alibaba Cloud MSE Ingress。
Ingress 對(duì)象: 用戶(hù)自定義的 Kubernetes 對(duì)象，描述了外部流量訪問(wèn)集群內(nèi)Service的規(guī)則集。

1.3 什么是ingress-nginx

Ingress nginx是 Kubernetes 中最常用的 Ingress 控制器之一，由 NGINX 官方提供支持（備注：nginx和k8s官方各自維護(hù)了一套 nginx ingress controller）。

NGINX Ingress Controller 是 NGINX 和 NGINX Plus 的Ingress Controller實(shí)現(xiàn)，可以對(duì)Websocket、gRPC、TCP 和 UDP 應(yīng)用程序進(jìn)行負(fù)載平衡。它支持標(biāo)準(zhǔn)Ingress功能，例如基于內(nèi)容的路由和 TLS/SSL 終止。

1.4 ingress-nginx優(yōu)點(diǎn)和限制

優(yōu)點(diǎn)	限制
經(jīng)過(guò)市場(chǎng)廣泛使用，社區(qū)支持活躍。	需要對(duì) NGINX 配置有一定的理解，以便充分利用其功能。
與 NGINX 的成熟生態(tài)系統(tǒng)兼容，提供穩(wěn)定和高性能的 HTTP/HTTPS 處理。	在非常復(fù)雜的路由場(chǎng)景中，配置可能變得繁瑣。
豐富的配置選項(xiàng)和靈活的路由機(jī)制。

1.5 版本兼容性矩陣

ingress-nginx 項(xiàng)目支持的版本意味著官方已經(jīng)完成了 E2E 測(cè)試，并且它們正在通過(guò)列出的版本。 Ingress-Nginx 版本可能適用于舊版本，但該項(xiàng)目不做出這種保證。

支持	Ingress-NGINX版本	k8s支持版本	Alpine 版本	Nginx版本	Helm Chart 版本
??	v1.11.2	1.30, 1.29, 1.28, 1.27, 1.26	3.20.0	1.25.5	4.11.2
??	v1.11.1	1.30, 1.29, 1.28, 1.27, 1.26	3.20.0	1.25.5	4.11.1
??	v1.11.0	1.30, 1.29, 1.28, 1.27, 1.26	3.20.0	1.25.5	4.11.0
??	v1.10.4	1.30, 1.29, 1.28, 1.27, 1.26	3.20.0	1.25.5	4.10.4
??	v1.10.3	1.30, 1.29, 1.28, 1.27, 1.26	3.20.0	1.25.5	4.10.3
??	v1.10.2	1.30, 1.29, 1.28, 1.27, 1.26	3.20.0	1.25.5	4.10.2
??	v1.10.1	1.30, 1.29, 1.28, 1.27, 1.26	3.19.1	1.25.3	4.10.1
??	v1.10.0	1.29, 1.28, 1.27, 1.26	3.19.1	1.25.3	4.10.0

2 實(shí)踐： Ingress nginx部署

本文k8s環(huán)境是v1.29.7, ingress-nginx是v1.11.1

?? 注意：提前將ingress-nginx的鏡像下載并上傳到私有倉(cāng)庫(kù)

[root@k8s-master1 ingress-nginx]# grep -n -r image: ingress-nginx.yaml
447:        image:  registry.k8s.io/ingress-nginx/controller:v1.11.1 @sha256:e6439a12b52076965928e83b7b56aae6731231677b01e81818bce7fa5c60161a
548:        image:  registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.1 @sha256:36d05b4077fb8e3d13663702fa337f124675ba8667cbd949c03a8e8ea6fa4366

2.1 使用helm部署ingress-nginx

官方推薦helm方式部署，如果環(huán)境沒(méi)有helm需要先下載： helm下載地址

2.1.1 安裝和配置Helm

安裝helm

tar -zxvf helm-v3.15.3-linux-amd64.tar.gz --strip-components 1 -C /usr/local/bin linux-amd64/helm

添加ingress-nginx的repo

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

更新repo信息

helm repo update

搜索ingress-nginx的Chart

[root@k8s-master1 ingress-nginx]# helm search repo ingress-nginx --version 4.11.1
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                      
ingress-nginx/ingress-nginx     4.11.1          1.11.1          Ingress controller for Kubernetes using NGINX a...

? 說(shuō)明： --version 4.11.1：指定Chart版本搜索

下載ingress-nginx的Chart

mkdir helm && cd helm
helm pull ingress-nginx/ingress-nginx --version 4.11.1

解壓Charts

# 解壓Chart
tar zxvf ingress-nginx-4.11.1.tgz

# 進(jìn)入Chart目錄
cd ingress-nginx

2.1.2 配置和創(chuàng)建Ingress-Nginx

調(diào)整Values.yaml的參數(shù)

鏡像倉(cāng)庫(kù)和鏡像

  21   image:
  22     ## Keep false as default for now!
  23     chroot: false
  24     #注釋原倉(cāng)庫(kù)地址
  25     #registry: registry.k8s.io
  26     #替換成阿里云倉(cāng)庫(kù)地址
  27     registry: registry.cn-hangzhou.aliyuncs.com
  28     image: ingress-nginx-steven/controller
  32     tag: "v1.11.1"
  33     # 注釋sha256校驗(yàn)碼
  34     #digest: sha256:e6439a12b52076965928e83b7b56aae6731231677b01e81818bce7fa5c60161a

 ---
 805       image:
 806         # 注釋原倉(cāng)庫(kù)地址
 807         #registry: registry.k8s.io
 808         registry: registry.cn-hangzhou.aliyuncs.com
 809         image: ingress-nginx-steven/kube-webhook-certgen
 813         tag: v1.4.1
 814         # 注釋sha256校驗(yàn)碼
 815         #digest: sha256:36d05b4077fb8e3d13663702fa337f124675ba8667cbd949c03a8e8ea6fa4366

使用本地網(wǎng)絡(luò)和dns

# 啟用主機(jī)網(wǎng)絡(luò) 
103   hostNetwork: true
 ...
# 啟用主機(jī)dns服務(wù)
78   dnsPolicy: ClusterFirstWithHostNet

修改Ingress模式和指定port

 # 默認(rèn)是LoadBalancer
 484     #type: LoadBalancer
 485     type: NodePort
 ...
 540     appProtocol: true
 541     nodePorts:
 542       # -- Node port allocated for the external HTTP listener. If left empty, the service co     ntroller allocates one from the configured node port range.
 543       http: "30080"
 544       # -- Node port allocated for the external HTTPS listener. If left empty, the service c     ontroller allocates one from the configured node port range.
 545       https: "30443"

創(chuàng)建前，執(zhí)行以下命令檢查配置是否修改成功：

helm install ingress-nginx . --dry-run=client --namespace=ingress-nginx

創(chuàng)建ingress-nignx

helm install ingress-nginx . --namespace=ingress-nginx --create-namespace

查看是否創(chuàng)建成功

[root@k8s-master1 ingress-nginx]# helm list --namespace ingress-nginx
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS  CHART                    APP VERSION
ingress-nginx   ingress-nginx   1               2024-08-27 21:44:34.137344711 +0800 CST deployedingress-nginx-4.11.1     1.11.1

卸載ingress-nginx

[root@k8s-master1 ingress-nginx]# helm uninstall ingress-nginx -n ingress-nginx
release "ingress-nginx" uninstalled

2.2 使用yaml文件部署ingress-nginx

1) 參考官方y(tǒng)aml文件

2) 也可以參考以下配置

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resourceNames:
  - ingress-nginx-leader
  resources:
  - leases
  verbs:
  - get
  - update
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "false"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  strategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.11.1
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-nginx-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        - --enable-metrics=false
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx-steven/controller:v1.11.1
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          readOnlyRootFilesystem: false
          runAsNonRoot: true
          runAsUser: 101
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.11.1
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx-steven/kube-webhook-certgen:v1.4.1
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65532
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.11.1
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx-steven/kube-webhook-certgen:v1.4.1
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65532
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None
 官方y(tǒng)aml文件

根據(jù)官方y(tǒng)aml，修改鏡像地址：

[root@k8s-master1 ingress-nginx]# grep -n  image: ingress-nginx.yaml
448:        image:  harbor.zx/hcie/controller:v1.11.1 
549:        image:  harbor.zx/hcie/kube-webhook-certgen:v1.4.1 
602:        image:  harbor.zx/hcie/kube-webhook-certgen:v1.4.1

修改ingress的端口：

352   ports:
353   - appProtocol: http
354     name: http
355     port: 80
356     protocol: TCP
357     targetPort: http
358      nodePort: 30080 
359   - appProtocol: https
360     name: https
361     port: 443
362     protocol: TCP
363     targetPort: https
364      nodePort: 30443

修改ingress的代理方式：

369   #type: LoadBalancer  # 默認(rèn)是負(fù)載均衡，如果是云環(huán)境建議使用默認(rèn)參數(shù)
370   type: NodePort

部署ingress-nginx：

kubectl apply -f ingress-nginx.yaml

部署后輸出結(jié)果：

namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

2.3 部署后查看ingress狀態(tài)

部署后，執(zhí)行以下命令：

kubectl get svc,pod -n ingress-nginx -owide

查看Pod和SVC的狀態(tài)：

NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE   SELECTOR
service/ingress-nginx-controller              NodePort     10.245.187.253   <none>         80:30080/TCP,443:30443/TCP    18m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission   ClusterIP   10.245.245.14    <none>        443/TCP                      18m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx

NAME                                            READY   STATUS      RESTARTS   AGE   IP              NODE          NOMINATED NODE   READINESS GATES
pod/ingress-nginx-admission-create-h6s42        0/1     Completed   0          18m   172.16.126.2    k8s-worker2   <none>           <none>
pod/ingress-nginx-admission-patch-79kv5         0/1     Completed   0          18m   172.16.126.3    k8s-worker2   <none>           <none>
pod/ingress-nginx-controller-746fcdfcdd-6gt4n   1/1     Running     0          18m   172.16.194.85    k8s-worker1    <none>           <none>

? 說(shuō)明：

NodePort: Service的類(lèi)型是在ingress-nginx.yaml中自定義。
80:30080/TCP, 443:30443/TCP: 這兩個(gè)端口也是在ingress-nginx.yaml中自定義，后面訪問(wèn)使用這兩個(gè)端口。
ingress-nginx-controller的Pod為ingress-nginx-controller-746fcdfcdd-6gt4n運(yùn)行在k8s-worker1節(jié)點(diǎn)上。后續(xù)訪問(wèn)ingress代理的服務(wù)，都要用k8s-worker1節(jié)點(diǎn)的IP做DNS解析。

執(zhí)行以下命令，查看IngressClass：

[root@k8s-master1 ingress-nginx]# kubectl get IngressClass
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx   <none>       25m

? 說(shuō)明:

Ingress類(lèi)型：nginx是ingress-nginx默認(rèn)創(chuàng)建的；
當(dāng)創(chuàng)建Ingress對(duì)象時(shí)，需要指定ingressClass字段(.spec.ingressClassName)
可以對(duì)ingressClass添加ingressclass.kubernetes.io/is-default-class 注解，這樣 IngressClass 會(huì)被視為默認(rèn)的 Ingress 類(lèi)。當(dāng)某個(gè) IngressClass 資源將此注解設(shè)置為 true 時(shí)，沒(méi)有指定類(lèi)的新 Ingress 資源將被分配到此默認(rèn)類(lèi)。

2.4 創(chuàng)建實(shí)例測(cè)試 Ingress

2.4.1 部署ingress對(duì)象

最后，我們打算使用1個(gè)3副本的deployment組成nginx-service，然后創(chuàng)建一個(gè)ingress，用于代理nginx的服務(wù)。yaml配置如下：

# 部署3個(gè)nginx實(shí)例
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: registry.cn-hangzhou.aliyuncs.com/hcie/nginx:1.26.1-alpine
        ports:
        - containerPort: 80
---
# 部署nginx的Service
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx-pod
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
---
# 創(chuàng)建ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-http
spec:
  # 上面的IngressClass名稱(chēng)
  ingressClassName: "nginx"
  rules:
  # 域名，稍后要通過(guò)這個(gè)域名訪問(wèn)服務(wù)
  - host: nginx.steven.com
    http:
      paths:
      # 路徑，訪問(wèn)域名時(shí)后面添加的路徑
      - path: /
        pathType: Prefix
        backend:
          # 要代理的服務(wù)和服務(wù)的端口
          service:
            name: nginx-service
            port: 
              number: 80

部署deployment、service、ingress

kubectl apply -f ingress-test.yaml

執(zhí)行以下命令，查看部署結(jié)果：

[root@k8s-master1 ingress-nginx]# kubectl get deployment,svc,ingress
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-test   3/3     3            3           19s

NAME                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/nginx-service   ClusterIP   10.245.127.127   <none>        80/TCP    19s

NAME                                     CLASS   HOSTS            ADDRESS          PORTS   AGE
ingress.networking.k8s.io/ingress-http   nginx   nginx.test.com   10.245.187.253   80      19s

查看下ingress-http詳細(xì)信息：

[root@k8s-master1 ingress-nginx]# kubectl describe ingress ingress-http
Name:             ingress-http
Labels:           <none>
Namespace:        default
Address:          10.245.187.253
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host            Path  Backends
  ----            ----  --------
  nginx.test.com
                  /   nginx-service:80 (<none>)
Annotations:      <none>
Events:
  Type    Reason  Age                    From                      Message
  ----    ------  ----                   ----                      -------
  Normal  Sync    2m49s (x2 over 2m56s)  nginx-ingress-controller  Scheduled for sync

查看ingress-controller的日志信息：

[root@k8s-master1 ingress-nginx]# kubectl -n ingress-nginx logs -f ingress-nginx-controller-746fcdfcdd-6gt4n
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.11.1
  Build:         7c44f992012555ff7f4e47c08d7c542ca9b4b1f7
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.25.5

-------------------------------------------------------------------------------

W0826 03:58:30.736251       7 client_config.go:659] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0826 03:58:30.736660       7 main.go:205] "Creating API client" host="https://10.245.0.1:443"
I0826 03:58:37.887271       7 main.go:248] "Running in Kubernetes cluster" major="1" minor="29" git="v1.29.7" state="clean" commit="4e4a18878ce330fefda1dc46acca88ba355e9ce7" platform="linux/amd64"
I0826 03:58:38.432286       7 main.go:101] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0826 03:58:38.490727       7 ssl.go:535] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0826 03:58:38.516493       7 nginx.go:271] "Starting NGINX Ingress controller"
I0826 03:58:38.538981       7 event.go:377] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"0dba2870-a96b-4b5e-a908-b985139e5d63", APIVersion:"v1", ResourceVersion:"624593", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0826 03:58:39.719275       7 nginx.go:317] "Starting NGINX process"
I0826 03:58:39.720802       7 leaderelection.go:250] attempting to acquire leader lease ingress-nginx/ingress-nginx-leader...
I0826 03:58:39.723329       7 nginx.go:337] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0826 03:58:39.724636       7 controller.go:193] "Configuration changes detected, backend reload required"
I0826 03:58:39.748482       7 leaderelection.go:260] successfully acquired lease ingress-nginx/ingress-nginx-leader
I0826 03:58:39.748953       7 status.go:85] "New leader elected" identity="ingress-nginx-controller-746fcdfcdd-6gt4n"
 I0826 03:58:39.826280       7 controller.go:213] "Backend successfully reloaded"
I0826 03:58:39.826390       7 controller.go:224] "Initial sync, sleeping for 1 second" 
I0826 03:58:39.826978       7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-746fcdfcdd-6gt4n", UID:"68964da5-0da2-4ca7-84f3-742e3d1646ed", APIVersion:"v1", ResourceVersion:"624720", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration

從最后幾條日志來(lái)看，我們創(chuàng)建的ingress已經(jīng)與ingress-nginx成功關(guān)聯(lián)上了，下面可以進(jìn)行訪問(wèn)測(cè)試了。

2.4.2 訪問(wèn)測(cè)試

2.4.2.1 在HOSTS文件里添加解析記錄：

?? 推薦：如果安裝了火絨，可以使用內(nèi)置安全工具，方便修改。

添加記錄：

192.168.3.44 nginx.steven.com

訪問(wèn)測(cè)試：

2.4.3 pod負(fù)載均衡測(cè)試

修改nginx的index.html

[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-dsqkg -- sh
/ # echo "nginx1"> /usr/share/nginx/html/index.html
/ # exit
[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-h7khh -- sh
/ # echo "nginx2"> /usr/share/nginx/html/index.html
/ # exit
[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-z2v6l -- sh
/ # echo "nginx3"> /usr/share/nginx/html/index.html
/ # exit

再次訪問(wèn)測(cè)試，默認(rèn)是輪詢(xún)方式，負(fù)載均衡主要是service的作用。

3 DaemonSet方式部署Ingress-nginx

DaemonSet 確保 Ingress Controller 的副本在集群中的每個(gè)節(jié)點(diǎn)上運(yùn)行。這么做的原因如下：

它確保入口控制器保持服務(wù)流量高可用，即使某個(gè)節(jié)點(diǎn)發(fā)生故障。
它允許 Ingress Controller 在集群中的所有節(jié)點(diǎn)之間均勻分配流量，從而提高性能。
它允許 Ingress Controller 綁定到主機(jī)的網(wǎng)絡(luò)命名空間，這對(duì)于某些功能（例如外部 DNS）是必需的。

3.1 配置DaemonSet

3.1.1 helm部署方式

# 切換到Chart目錄
cd helm/ingress-nginx
vim values.yaml

修改ingress-nginx-controller的kind字段

 222   #kind: Deployment
 223   kind: DaemonSet

更新helm配置

helm upgrade ingress-nginx . -n ingress-nginx

查看部署結(jié)果

[root@k8s-master1 ingress-nginx]# kubectl get pod,svc -owide -n ingress-nginx
NAME                                 READY   STATUS    RESTARTS   AGE    IP          NODE          NOMINATED NODE   READINESS GATES
pod/ingress-nginx-controller-2bljr   1/1     Running   0          61s    10.10.0.1   k8s-master1   <none>           <none>
pod/ingress-nginx-controller-2s59q   1/1     Running   0          3m1s   10.10.0.4   k8s-worker1   <none>           <none>
pod/ingress-nginx-controller-dppwt   1/1     Running   0          61s    10.10.0.5   k8s-worker2   <none>           <none>
pod/ingress-nginx-controller-gzshs   1/1     Running   0          61s    10.10.0.3   k8s-master3   <none>           <none>
pod/ingress-nginx-controller-hbj9d   0/1     Running   0          61s    10.10.0.2   k8s-master2   <none>           <none>

NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE    SELECTOR
service/ingress-nginx-controller             NodePort    10.245.133.125   <none>        80:30080/TCP,443:30443/TCP   3m2s   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission   ClusterIP   10.245.236.128   <none>        443/TCP                      3m2s   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx

看到每個(gè)節(jié)點(diǎn)都部署了一個(gè)ingress-controller的pod

3.1.2 yaml文件部署方式

修改這行即可

396 #kind: Deployment
397 kind: DaemonSet

驗(yàn)證方法如4.2節(jié)，這里就不重復(fù)敘述了。

4 結(jié)論

Ingress-Nginx 是 Kubernetes 中強(qiáng)大而靈活的流量管理工具，能夠有效地路由外部請(qǐng)求到內(nèi)部服務(wù)。通過(guò)本文的講解與實(shí)踐示例，您應(yīng)該能夠快速上手并在自己的集群中實(shí)現(xiàn) Ingress-Nginx 的配置與管理。繼續(xù)探索 Ingress-Nginx 的更多高級(jí)功能，將使您的應(yīng)用架構(gòu)更加高效和安全。