docker方式部署的jenkins解决docker-in-docker的几种方式
已收录 5950

分享几种用 dokcer 方式部署的 jenkins 中如何使用 docker 的解决方案
解决方案一：挂载 docker.sock，直接使用 docker 命令
docker-compose 中增加下面的挂载项


...
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
      - /etc/docker:/etc/docker

给/var/run/docker.sock 增加权限，否则可能报下面的错


permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/auth": dial unix /var/run/docker.sock: connect: permission denied

方式一：在 docker.service 中添加权限修改命令 ExecStartPost=/usr/bin/chmod 666 /var/run/docker.sock


...
[Service]
...
ExecStart=/usr/bin/dockerd
ExecStartPost=/usr/bin/chmod 666 /var/run/docker.sock
ExecReload=/bin/kill -s HUP $MAINPID
...

方式二：给容器内的 jenkins 用户增加 id=128 的组权限

启动 jenkins 容器之后进去可以看到 jenkins 的 id 为 128：


$ cat /etc/group | grep docker
[sudo] password for fly:
docker:x:128:fly

# 先移除旧容器
sudo docker rm -f jenkins
# 重新创建容器,启动参数添加：--group-add=128
sudo docker run -d -p 8080:8080 -p 50000:50000 \
    -v /data2/jenkins/jenkins_home:/var/jenkins_home \
    -v /etc/localtime:/etc/localtime:ro \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /etc/docker:/etc/docker \
    -v /usr/bin/docker:/usr/bin/docker \
    --restart=always  \
    --group-add=128 \
    --name jenkins \
    jenkins/jenkins:latest

方式三：使用 root 用户启动 jenkins


version: "3"
services:
  jenkins:
    ...
    user: root
    privileged: true

3. jenkinsfile 中直接调用 docker


pipeline {
...
  stages {
       ...
  stage(' docker build and push') {
            steps {
                sh '''
                    docker build  -f ./.cicd/Dockerfile -t ${HARBOR_URL}/${HARBOR_REPO}/${PROJECT_NAME}:${IMAGE_TAG}
                    docker login -u ${HARBOR_USER} -p ${HARBOR_PASSWORD} ${HARBOR_URL}
                    docker push ${HARBOR_URL}/${HARBOR_REPO}/${PROJECT_NAME}:${IMAGE_TAG}
                    docker rmi ${HARBOR_URL}/${HARBOR_REPO}/${PROJECT_NAME}:${IMAGE_TAG}
                '''
          }
    }
    ...
  }
}

解决方案二：调用远程 docker 服务
找一台机器安装好 docker 环境的服务器，可以是本机也可以是其他机器，开启 Remote API 访问端口，这样就能被远程调用了


$ cat /etc/docker/daemon.json
{
...
  "hosts": [
    "tcp://0.0.0.0:2375",
    "unix:///var/run/docker.sock"
   ]
}
$  systemctl restart docker

新版可能是需要修改 service 文件：修改 ExecStart 行，增加内容 -H tcp://0.0.0.0:2375


$ cat /usr/lib/systemd/system/docker.service
...
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
# 注意需要增加-H tcp://0.0.0.0:2375，原来的-H 不能去掉
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H fd:// --containerd=/run/containerd/containerd.sock
#ExecStartPost=/usr/bin/chmod 666 /var/run/docker.sock
...
$ systemctl daemon-reload
$ systemctl restart docker

测试是否可用


$ docker -H tcp://127.0.0.1:2375 version
Client: Docker Engine - Community
 Version:           26.1.2
 API version:       1.45
 Go version:        go1.21.10
 Git commit:        211e74b
 Built:             Wed May  8 14:01:00 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.2
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.10
  Git commit:       ef1912d
  Built:            Wed May  8 13:59:54 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.32
  GitCommit:        8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
jenkinsfile 中引用,所有命令前需要加上-H tcp://xx.xx.xx.xx:2375

pipeline {
...
    stages {
        ...
        stage('docker build&push'){
          steps {
                sh '''
                 docker -H tcp://192.168.31.79:2375 build -f ./.cicd/Dockerfile -t ${HARBOR_URL}/${HARBOR_REPO}/${PROJECT_NAME}:${IMAGE_TAG}
                 docker -H tcp://192.168.31.79:2375 login -u ${HARBOR_USER} -p ${HARBOR_PASSWORD} ${HARBOR_URL}
                 docker -H tcp://192.168.31.79:2375 push ${HARBOR_URL}/${HARBOR_REPO}/${PROJECT_NAME}:${IMAGE_TAG}
                 docker -H tcp://192.168.31.79:2375 rmi ${HARBOR_URL}/${HARBOR_REPO}/${PROJECT_NAME}:${IMAGE_TAG}
                '''
            }
        }
       ...
    }
}

解决方案三：使用 docker/k8s 插件加 buildah
另外还可以使用 buildah 来构建镜像，因为它不依赖 docker 等容器运行时，支持 dockerfile，build 出来的镜像也支持 docker 和其他镜像运行时使用。这种方式其实不能说是解决 docker-in-docker 的一种方式，因为本身 jenkins 就需要 docker 环境了，这种适合 jenkins 本身就在宿主机运行又不想使用 docker 的场景。

项目的 Dockerfile 中使用多阶段构建写法,示例


FROM node:20.16.0-alpine3.20 AS builder
USER root
ARG ENV_VAR
WORKDIR /src
COPY . .

RUN node --version \
  && npm -v \
  && cat /etc/os-release \
  && rm -f yarn.lock\
  && rm -f package-lock.json \
  && npm cache clean --force \
  && npm cache verify \
  && npm config get proxy \
  && npm config get https-proxy \
  && npm config rm proxy \
  && npm config rm https-proxy \
  && unset http_proxy \
  && unset https_proxy \
  && unset ftp_proxy \
  && unset no_proxy \
  && npm config set cache "~/.npm" \
  && npm config set prefix "~/.npm" \
  && npm config set strict-ssl false \
  && cat ~/.npmrc \
  && npm config set registry https://registry.npmmirror.com \
  && npm install --unsafe-perm=true --loglevel=verbose --allow-root \
  && npm run build:${ENV_VAR}

FROM nginx:1.24.0-alpine AS final
USER root
ADD .cicd/ca.crt /etc/pki/ca-trust/source/anchors/
RUN update-ca-trust

COPY .cicd/nginx.conf /etc/nginx/nginx.conf

WORKDIR /data/frontend
COPY --from=builder /src/dist .
EXPOSE 8081
CMD ["nginx", "-g", "daemon off;"]

在 jenkins master 上的 jenkins slave pod 中添加 buildah 的 container

kubernetes

1）安装 kubernetes 插件

2）在 cloud 中添加 buildah 的 container template

docker 方式部署的 jenkins 解决 docker-in-docker 的几种方式

docker 方式部署的 jenkins 解决 docker-in-docker 的几种方式

docker

1) 把/var/run/docker.sock /usr/bin/docker 挂载到 jenkins 容器中


version: "3"
services:
  jenkins:
...
    volumes:
      - $PWD/jenkins/data:/var/jenkins_home
      - $PWD/jenkins/.ssh:/var/jenkins_home/.ssh
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
      - /etc/docker:/etc/docker
...

2）安装 docker 插件

在 jenkins pipeline 中使用 buildah 构建镜像，只需要指定多阶段构建的 Dockerfile 即可，不用再增加而外的编译打包等 step。

k8s


pipeline {
    agent {label 'jenkins-slave'}

    ...

    stages {

        stage('拉取代码') {
          steps {
            container('git'){
              script {
                      checkout scmGit(branches: [[name: "${git_branch}"]], extensions: [], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_url}" ]])
              }
           }
          }
        }

        stage('build&push image'){
          steps {
                container('buildah'){ 
                    sh '''
                        pwd
                        ls

                        buildah --storage-driver=vfs bud --format=oci --tls-verify=false --no-cache -f ./.cicd/Dockerfile -t ${REGISTRY_URL}/${PROJECT_NAME}/${IMAGE_NAME}:${IMAGE_TAG} .

                        buildah images

                        buildah --storage-driver=vfs push --tls-verify=false ${REGISTRY_URL}/${PROJECT_NAME}/${IMAGE_NAME}:${IMAGE_TAG} docker://${REGISTRY_URL}/${PROJECT_NAME}/${IMAGE_NAME}:${IMAGE_TAG}
                    '''
                }
            }
        }

    ...
    } 
}

docker


pipeline {
    agent any
    ...
    stages {
        stage('拉取代码') {
              //注意：这里不加 args '--entrypoint=\'\'' 会报错：Alternatively you can force image entrypoint to be disabled by adding option `--entrypoint=''`.
                agent {
                      docker {
                        image 'bitnami/git:2.43.2'
                        args "--privileged"
                        args '--entrypoint=\'\''
                      }
                }

                steps {
                    script {
                        //检出代码
                        checkout scmGit(branches: [[name: "${GIT_BRANCH_OR_TAG}"]], extensions: [], userRemoteConfigs: [[credentialsId: "${GIT_AUTH}", url: "${GIT_URL}" ]])

                        //组装 IMAGE_TAG
                        sh '''
                            BRANCH_NAME=$(cd ${WORKSPACE} && git name-rev --name-only HEAD|awk -F"/" '{print $NF}'|tr 'A-Z' 'a-z')
                            DATE_TIME=$(date '+%Y%m%d%H%M%S'|sed 's/[[:space:]]//g')
                            COMMIT_ID=$(git log -n 1 --pretty=format:'%h'|sed 's/[[:space:]]//g')
                            export IMAGE_TAG=${BRANCH_NAME}-${DATE_TIME}-${COMMIT_ID}
                            echo ${IMAGE_TAG} > ${WORKSPACE}/imagetag.txt
                        '''

                    }
              }
        }

        stage('build&push image'){
         agent {
              docker { 
                image 'quay.io/buildah/stable:v1.35.4' 
                args '--privileged'
              }
         }

         steps {
              script {
                  withCredentials([usernamePassword(credentialsId: "${env.HARBOR_CREDENTIALS_ID}", usernameVariable: 'HARBOR_USER', passwordVariable: 'HARBOR_PASS')]) {
                  sh '''
                      pwd
                      ls -la
                      export IMAGE_TAG=$(cat ${WORKSPACE}/imagetag.txt)
                      echo ${IMAGE_TAG}

                      buildah login -u ${HARBOR_USER} -p ${HARBOR_PASS} ${DOCKER_REGISTRY}

                      buildah --storage-driver=vfs build --build-arg ENV_VAR=prod --format=oci --tls-verify=false --no-cache -f ${DOCKERFILE_PATH} -t ${DOCKER_REGISTRY}/${PROJECT_NAME}/${IMAGE_NAME}:${IMAGE_TAG} .

                      buildah images

                      buildah --storage-driver=vfs push --tls-verify=false ${DOCKER_REGISTRY}/${PROJECT_NAME}/${IMAGE_NAME}:${IMAGE_TAG}

                      buildah rmi ${DOCKER_REGISTRY}/${PROJECT_NAME}/${IMAGE_NAME}:${IMAGE_TAG}
                  '''
                  }
              }
          }
        }
       ... 
    }
    ...
}
© 版权声明
博主的文章没有高度、深度和广度，只是凑字数。利用读书、参考、引用、抄袭、复制和粘贴等多种方式打造成自己的纯镀 24k 文章！如若有侵权，请联系博主删除。

☆ END ☆
网络技术
喜欢就点个赞吧
点赞0 分享
相关推荐
图片正在生成中，请稍后...