在AWS使用EKS中
本文链接地址:在AWS使用EKS慢慢的回味
使用AWS的EKS来托管Kubernetes是比较复杂,按照如下的方法可以创建出一个满足大部分使用环境的EKS。
Content:5设置EKS的存储EFS5。1创建接入EFS的策略(Root用户操作)5。2创建访问EFS的角色(Root用户操作)5。3为OpenIDConnect创建IdentityProvider(Root用户操作)5。4在EKS中创建服务账户(IAM用户)5。5创建EFSCSI插件(IAM用户)5。6创建EFS文件系统(Root用户操作)5。7创建Kubernetes里面的存储类(IAM用户)6部署Jenkins来测试(IAM用户)6。1部署Jenkins6。2验证结果
5设置EKS的存储EFS5。1创建接入EFS的策略(Root用户操作)
自定义一策略:TestEKSAccessEFSPolicy{Version:20121017,Statement:〔{Effect:Allow,Action:〔elasticfilesystem:DescribeAccessPoints,elasticfilesystem:DescribeFileSystems〕,Resource:},{Effect:Allow,Action:〔elasticfilesystem:CreateAccessPoint〕,Resource:,Condition:{StringLike:{aws:RequestTagefs。csi。aws。comcluster:true}}},{Effect:Allow,Action:elasticfilesystem:DeleteAccessPoint,Resource:,Condition:{StringEquals:{aws:ResourceTagefs。csi。aws。comcluster:true}}}〕}5。2创建访问EFS的角色(Root用户操作)
创建角色TestEKSAccessEFSRole并分配策略TestEKSAccessEFSPolicy。
在信任关系Trustrelationships中,修改如下内容。
替换oidc。eks。useast1。amazonaws。comid98F61019E9B399FA9B7A43A19B56DF14为你EKS的OpenIDConnectproviderURL。
{Version:20121017,Statement:〔{Effect:Allow,Principal:{Federated:arn:aws:iam::675892200046:oidcprovideroidc。eks。useast1。amazonaws。comid98F61019E9B399FA9B7A43A19B56DF14},Action:sts:AssumeRoleWithWebIdentity,Condition:{StringEquals:{oidc。eks。useast1。amazonaws。comid98F61019E9B399FA9B7A43A19B56DF14:sub:system:serviceaccount:kubesystem:efscsicontrollersa}}}〕}
5。3为OpenIDConnect创建IdentityProvider(Root用户操作)
填入提供URL和审计URLsts。amazonaws。com,点击Getthumbprint,然后单击Addprovider。
5。4在EKS中创建服务账户(IAM用户)
创建文件efsserviceaccount。yaml,包含如下内容,然后kubectlapplyfefsserviceaccount。yaml创建账户,注意修改accountid。apiVersion:v1kind:ServiceAccountmetadata:name:efscsicontrollersanamespace:kubesystemlabels:app。kubernetes。ioname:awsefscsidriverannotations:eks。amazonaws。comrolearn:arn:aws:iam::675892200046:roleTestEKSAccessEFSRole5。5创建EFSCSI插件(IAM用户)
执行如下命令获取EFS插件的安装yaml文件:driver。yamlkubectlkustomizegithub。comkubernetessigsawsefscsidriverdeploykubernetesoverlaysstableecr?refrelease1。3driver。yaml
上面已经创建了服务账号,所以driver。yaml文件里面的efscsicontrollersa段可以去掉。
接着运行命令kubectlapplyfdriver。yaml创建CSI插件。apiVersion:v1kind:ServiceAccountmetadata:name:efscsicontrollersanamespace:kubesystemlabels:app。kubernetes。ioname:awsefscsidriverannotations:eks。amazonaws。comrolearn:arn:aws:iam::675892200046:roleTestEKSAccessEFSRoleapiVersion:v1kind:ServiceAccountmetadata:labels:app。kubernetes。ioname:awsefscsidrivername:efscsinodesanamespace:kubesystemapiVersion:rbac。authorization。k8s。iov1kind:ClusterRolemetadata:labels:app。kubernetes。ioname:awsefscsidrivername:efscsiexternalprovisionerrolerules:apiGroups:resources:persistentvolumesverbs:getlistwatchcreatedeleteapiGroups:resources:persistentvolumeclaimsverbs:getlistwatchupdateapiGroups:storage。k8s。ioresources:storageclassesverbs:getlistwatchapiGroups:resources:eventsverbs:listwatchcreatepatchapiGroups:storage。k8s。ioresources:csinodesverbs:getlistwatchapiGroups:resources:nodesverbs:getlistwatchapiGroups:coordination。k8s。ioresources:leasesverbs:getwatchlistdeleteupdatecreateapiVersion:rbac。authorization。k8s。iov1kind:ClusterRoleBindingmetadata:labels:app。kubernetes。ioname:awsefscsidrivername:efscsiprovisionerbindingroleRef:apiGroup:rbac。authorization。k8s。iokind:ClusterRolename:efscsiexternalprovisionerrolesubjects:kind:ServiceAccountname:efscsicontrollersanamespace:kubesystemapiVersion:appsv1kind:Deploymentmetadata:labels:app。kubernetes。ioname:awsefscsidrivername:efscsicontrollernamespace:kubesystemspec:replicas:2selector:matchLabels:app:efscsicontrollerapp。kubernetes。ioinstance:kustomizeapp。kubernetes。ioname:awsefscsidrivertemplate:metadata:labels:app:efscsicontrollerapp。kubernetes。ioinstance:kustomizeapp。kubernetes。ioname:awsefscsidriverspec:containers:args:endpoint(CSIENDPOINT)logtostderrv2deleteaccesspointrootdirfalseenv:name:CSIENDPOINTvalue:unix:varlibcsisocketspluginproxycsi。sockimage:602401143452。dkr。ecr。uswest2。amazonaws。comeksawsefscsidriver:v1。3。8imagePullPolicy:IfNotPresentlivenessProbe:failureThreshold:5httpGet:path:healthzport:healthzinitialDelaySeconds:10periodSeconds:10timeoutSeconds:3name:efspluginports:containerPort:9909name:healthzprotocol:TCPsecurityContext:privileged:truevolumeMounts:mountPath:varlibcsisocketspluginproxyname:socketdirargs:csiaddress(ADDRESS)v2featuregatesTopologytrueextracreatemetadataleaderelectionenv:name:ADDRESSvalue:varlibcsisocketspluginproxycsi。sockimage:602401143452。dkr。ecr。uswest2。amazonaws。comekscsiprovisioner:v2。1。1imagePullPolicy:IfNotPresentname:csiprovisionervolumeMounts:mountPath:varlibcsisocketspluginproxyname:socketdirargs:csiaddresscsicsi。sockhealthport9909image:602401143452。dkr。ecr。uswest2。amazonaws。comekslivenessprobe:v2。2。0imagePullPolicy:IfNotPresentname:livenessprobevolumeMounts:mountPath:csiname:socketdirhostNetwork:truenodeSelector:kubernetes。ioos:linuxpriorityClassName:systemclustercriticalpriorityClassName:systemclustercriticalserviceAccountName:efscsicontrollersavolumes:emptyDir:{}name:socketdirapiVersion:appsv1kind:DaemonSetmetadata:labels:app。kubernetes。ioname:awsefscsidrivername:efscsinodenamespace:kubesystemspec:selector:matchLabels:app:efscsinodeapp。kubernetes。ioinstance:kustomizeapp。kubernetes。ioname:awsefscsidrivertemplate:metadata:labels:app:efscsinodeapp。kubernetes。ioinstance:kustomizeapp。kubernetes。ioname:awsefscsidriverspec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:matchExpressions:key:eks。amazonaws。comcomputetypeoperator:NotInvalues:fargatecontainers:args:endpoint(CSIENDPOINT)logtostderrv2env:name:CSIENDPOINTvalue:unix:csicsi。sockimage:602401143452。dkr。ecr。uswest2。amazonaws。comeksawsefscsidriver:v1。3。8imagePullPolicy:IfNotPresentlivenessProbe:failureThreshold:5httpGet:path:healthzport:healthzinitialDelaySeconds:10periodSeconds:2timeoutSeconds:3name:efspluginports:containerPort:9809name:healthzprotocol:TCPsecurityContext:privileged:truevolumeMounts:mountPath:varlibkubeletmountPropagation:Bidirectionalname:kubeletdirmountPath:csiname:plugindirmountPath:varrunefsname:efsstatedirmountPath:varamazonefsname:efsutilsconfigmountPath:etcamazonefslegacyname:efsutilsconfiglegacyargs:csiaddress(ADDRESS)kubeletregistrationpath(DRIVERREGSOCKPATH)v2env:name:ADDRESSvalue:csicsi。sockname:DRIVERREGSOCKPATHvalue:varlibkubeletpluginsefs。csi。aws。comcsi。sockname:KUBENODENAMEvalueFrom:fieldRef:fieldPath:spec。nodeNameimage:602401143452。dkr。ecr。uswest2。amazonaws。comekscsinodedriverregistrar:v2。1。0imagePullPolicy:IfNotPresentname:csidriverregistrarvolumeMounts:mountPath:csiname:plugindirmountPath:registrationname:registrationdirargs:csiaddresscsicsi。sockhealthport9809v2image:602401143452。dkr。ecr。uswest2。amazonaws。comekslivenessprobe:v2。2。0imagePullPolicy:IfNotPresentname:livenessprobevolumeMounts:mountPath:csiname:plugindirdnsPolicy:ClusterFirsthostNetwork:truenodeSelector:beta。kubernetes。ioos:linuxpriorityClassName:systemnodecriticalserviceAccountName:efscsinodesatolerations:operator:Existsvolumes:hostPath:path:varlibkubelettype:Directoryname:kubeletdirhostPath:path:varlibkubeletpluginsefs。csi。aws。comtype:DirectoryOrCreatename:plugindirhostPath:path:varlibkubeletpluginsregistrytype:Directoryname:registrationdirhostPath:path:varrunefstype:DirectoryOrCreatename:efsstatedirhostPath:path:varamazonefstype:DirectoryOrCreatename:efsutilsconfighostPath:path:etcamazonefstype:DirectoryOrCreatename:efsutilsconfiglegacyapiVersion:storage。k8s。iov1kind:CSIDrivermetadata:annotations:helm。shhook:preinstall,preupgradehelm。shhookdeletepolicy:beforehookcreationhelm。shresourcepolicy:keepname:efs。csi。aws。comspec:attachRequired:false
等一会,efscsicontroller应该就绪了。
5。6创建EFS文件系统(Root用户操作)
在AmazonEFS产品中,点击Createfilesystem开始创建:
选择Standard作为存储类,这样可用区里面的所有节点都可以访问。
创建完成后,等待Network可用,然后点击Manage按钮添加集群安全组。
5。7创建Kubernetes里面的存储类(IAM用户)
安装如下内容创建storageclass。yaml,并运行kubectlapplyfstorageclass。yaml来创建。
注意修改fileSystemId成你自己的,通过如下图查询。
kind:StorageClassapiVersion:storage。k8s。iov1metadata:name:efsscprovisioner:efs。csi。aws。comparameters:provisioningMode:efsapfileSystemId:fs04470c1ed1eab275cdirectoryPerms:700gidRangeStart:1000optionalgidRangeEnd:2000optionalbasePath:dynamicprovisioningoptional6部署Jenkins来测试(IAM用户)6。1部署Jenkins
注意设置存储类为efssc。
helmrepoaddjenkinscihttps:charts。jenkins。io
helminstallmyjenkinsjenkinscijenkinsversion4。1。17setpersistence。storageClassefssc6。2验证结果
等Jenkins启动后,可以采用端口转发来临时访问。〔awsclibogon〕kubectlportforwardsvcmyjenkinsaddress0。0。0。08081:8080Forwardingfrom0。0。0。0:80818080Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081Handlingconnectionfor8081