Golang-Mix
首页 编程笔记

drone 的流水线操作配置文件

pyweeX 发布于 07-05
编程笔记
pyweeX

配了两天关于 drone 的流水线文件，这个支持回滚操作，暂时不算完美。

这里面有几个十分关键的点要注意，一个是超时重试。我在正常的构建中，分的流程比较多，常常会出现超时的情况，而事情已经做到一半了，所以有时候因为超时中断导致原先的服务已断开，新的起不来。

通过这几行代码加入超时

  1. ...
  2.     retry:
  3.       limit: 2
  4.       delay: 25s
  5. ...

需要特别注意，如果是将 secret 传入到基于 ssh 连接的服务器中时，变量不会生效。要操作这三步才会生效：

  1. # 第一步，在environment中声明
  2. environment:
  3.     PROJECT_API_NAME:
  4.         from_secret: PROJECT_API_NAME
  6. # 第二步，在 envs 中将这个变量加进去
  7. envs: [PROJECT_API_NAME]
  9. # 第三步，在scripts中调用时，写法为
  10. script:
  11.     - echo $${PROJECT_API_NAME}

做到上述配置之后，最终执行时，drone自带的控制面板会以 ** 替代变量的值

整个pipeline可通过定义 trigger 来决定何时触发部署，也可以将 tigger 放在某一个 step 中。其中要特别注意 event，它可以使用 include 和 exclude 来配置包含和排除。其中的 custom 表示手动触发的情况。

  1. trigger:
  2.   branch:
  3.     - master
  4.     - develop
  5.   event:
  6.     include:
  7.       - custom
  8.       - merge_request
  9.       - push

完整例子

  1. kind: pipeline
  2. type: docker
  3. name: service-deploy
  5. steps:
  6.   - name: CI
  7.     image: docker:git
  8.     environment:
  9.       PROJECT_API_NAME:
  10.         from_secret: PROJECT_API_NAME
  11.       PROJECT_RPC_NAME:
  12.         from_secret: PROJECT_RPC_NAME
  13.       DOMAIN_PREFIX:
  14.         from_secret: DOMAIN_PREFIX
  15.     commands:
  16.       - export DOCKER_TAG=prod_${DRONE_COMMIT:0:10}
  17.       - docker build -f api/Dockerfile -t $PROJECT_API_NAME:latest .
  18.       - docker tag $PROJECT_API_NAME:latest harbor.xxx.com/kkkk/$PROJECT_API_NAME:${DRONE_COMMIT:0:10}
  19.       - docker login harbor.xxx.com -u admin -p xxx
  20.       - docker push harbor.xxx.com/kkkk/$PROJECT_API_NAME:${DRONE_COMMIT:0:10}
  21.       - echo "API构建完成, 你可以通过 docker pull harbor.xxx.com/kkkk/$PROJECT_API_NAME:${DRONE_COMMIT:0:10} 获取镜像"
  22.       - |
  23.         if [ -d "rpc" ]; then
  24.           docker build -f rpc/Dockerfile -t $PROJECT_RPC_NAME:latest .
  25.           docker tag $PROJECT_RPC_NAME:latest harbor.xxx.com/kkkk/$PROJECT_RPC_NAME:${DRONE_COMMIT:0:10}
  26.           docker push harbor.xxx.com/kkkk/$PROJECT_RPC_NAME:${DRONE_COMMIT:0:10}
  27.           echo "RPC构建完成, 你可以通过 docker pull harbor.xxx.com/kkkk/$PROJECT_RPC_NAME:${DRONE_COMMIT:0:10} 获取镜像"
  28.         fi
  29.     volumes:
  30.       - name: docker-sock
  31.         path: /var/run/docker.sock
  32.     when:
  33.       branch:
  34.         - develop
  35.         - master
  36.     retry:
  37.       limit: 2
  38.       delay: 10m
  40.   - name: stage-publish
  41.     image: appleboy/drone-ssh
  42.     environment:
  43.       STAGE_DOMAIN_PREFIX:
  44.         from_secret: STAGE_DOMAIN_PREFIX
  45.       PROJECT_API_NAME:
  46.         from_secret: PROJECT_API_NAME
  47.       PROJECT_RPC_NAME:
  48.         from_secret: PROJECT_RPC_NAME
  49.     settings:
  50.       host:
  51.         from_secret: STAGE_HOST_IP
  52.       username:
  53.         from_secret: STAGE_HOST_NAME
  54.       password:
  55.         from_secret: STAGE_HOST_PWD
  56.       port: 22
  57.       command_timeout: 360s
  58.       envs: [PROJECT_API_NAME,STAGE_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  59.       script:
  60.         - cd /home/kkkk/app_project/${DRONE_REPO_NAME}
  61.         - cat ./docker-compose.yml
  62.         #- git pull https://userName:userPwd@gitea.xxx.com/path/${DRONE_REPO_NAME}.git
  63.         #- docker login harbor.xxx.com -u admin -p xxx
  64.         - git pull origin
  65.         - git branch
  66.         - git fetch origin
  67.         - git checkout develop
  68.         - docker-compose down
  69.         - export DOMAIN_PREFIX=$${STAGE_DOMAIN_PREFIX}
  70.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  71.         - export ENV=""
  72.         - docker images | grep $${PROJECT_RPC_NAME} | awk '{print $3}' | xargs docker rmi -f
  73.         - docker images | grep $${PROJECT_API_NAME} | awk '{print $3}' | xargs docker rmi -f
  74.         - docker-compose up -d
  75.         #- echo "CI构建完成 ${DRONE_COMMIT_MESSAGE} ${DRONE_COMMIT:0:10}"
  76.     when:
  77.       branch:
  78.         - develop
  80.   - name: prod-publish
  81.     image: appleboy/drone-ssh
  82.     environment:
  83.       PROD_DOMAIN_PREFIX:
  84.         from_secret: PROD_DOMAIN_PREFIX
  85.       PROJECT_API_NAME:
  86.         from_secret: PROJECT_API_NAME
  87.       PROJECT_RPC_NAME:
  88.         from_secret: PROJECT_RPC_NAME
  89.     settings:
  90.       host:
  91.         from_secret: PROD_HOST_IP
  92.       username:
  93.         from_secret: PROD_HOST_NAME
  94.       password: 
  95.         from_secret: PROD_HOST_PWD
  96.       port: 22
  97.       command_timeout: 360s
  98.       envs: [PROJECT_API_NAME,PROD_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  99.       script:
  100.         - bash
  101.         - cd /kkkk/app_project/${DRONE_REPO_NAME}
  102.         - git pull https://userName:userPwd@gitea.xxx.com/path/${DRONE_REPO_NAME}.git
  103.         - docker login harbor.xxx.com -u admin -p xxx
  104.         - git pull origin
  105.         - git branch
  106.         - git fetch origin
  107.         - git checkout master
  108.         - export ENV="-prod"
  109.         - export DOMAIN_PREFIX=$${PROD_DOMAIN_PREFIX}
  110.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  111.         - docker-compose down
  112.         - docker images | grep $${PROJECT_API_NAME} | awk '{print $3}' | xargs docker rmi -f
  113.         - docker images | grep $${PROJECT_RPC_NAME} | awk '{print $3}' | xargs docker rmi -f
  114.         - docker-compose up -d
  115.         #- echo "CI构建完成 ${DRONE_COMMIT_MESSAGE} ${DRONE_COMMIT:0:10}"
  116.     when:
  117.       branch:
  118.         - master
  119.     retry:
  120.       limit: 2
  121.       delay: 360s
  123.   - name: notify
  124.     image: drillster/drone-email
  125.     settings:
  126.       host: 
  127.       username: 
  128.       password: 
  129.       port: 465
  130.       from: 
  131.     when:
  132.       status:
  133.         - failure
  135. volumes:
  136.   - name: docker-sock
  137.     host:
  138.       path: /var/run/docker.sock
  140. trigger:
  141.   branch:
  142.     - develop
  143.     - master
  144.   event:
  145.     include:
  146.       - push
  147.       - custom
  148.       - merge_request
  150. ---
  151. kind: pipeline
  152. type: docker
  153. name: promote/rollback
  155. steps:
  156.   - name: stage-rollback
  157.     image: appleboy/drone-ssh
  158.     environment:
  159.       PROJECT_API_NAME:
  160.         from_secret: PROJECT_API_NAME
  161.       PROJECT_RPC_NAME:
  162.         from_secret: PROJECT_RPC_NAME
  163.       DOMAIN_PREFIX:
  164.         from_secret: DOMAIN_PREFIX
  165.     settings:
  166.       host:
  167.         from_secret: STAGE_HOST_IP
  168.       username:
  169.         from_secret: STAGE_HOST_NAME
  170.       password:
  171.         from_secret: STAGE_HOST_PWD
  172.       port: 22
  173.       command_timeout: 300s
  174.       envs: [PROJECT_API_NAME,STAGE_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  175.       script:
  176.         - cd /home/kkkk/app_project/${DRONE_REPO_NAME}
  177.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  178.         - export DOMAIN_PREFIX=$${STAGE_DOMAIN_PREFIX}
  179.         - export ENV=""
  180.         - docker-compose down
  181.         - docker-compose up -d
  182.         - echo "版本已经更新到 ${DRONE_COMMIT:0:10}, ${DRONE_COMMIT_MESSAGE}"
  183.     when:
  184.       branch:
  185.         - develop
  187.   - name: prod-rollback
  188.     image: appleboy/drone-ssh
  189.     environment:
  190.       PROJECT_API_NAME:
  191.         from_secret: PROJECT_API_NAME
  192.       PROJECT_RPC_NAME:
  193.         from_secret: PROJECT_RPC_NAME
  194.       DOMAIN_PREFIX:
  195.         from_secret: DOMAIN_PREFIX
  196.     settings:
  197.       host:
  198.         from_secret: PROD_HOST_IP
  199.       username:
  200.         from_secret: PROD_HOST_NAME
  201.       password:
  202.         from_secret: PROD_HOST_PWD
  203.       port: 22
  204.       command_timeout: 300s
  205.       envs: [PROJECT_API_NAME,STAGE_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  206.       script:
  207.         - cd /kkkk/app_project/${DRONE_REPO_NAME}
  208.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  209.         - export DOMAIN_PREFIX=$${STAGE_DOMAIN_PREFIX}
  210.         - export ENV="-prod"
  211.         - docker-compose down
  212.         - docker-compose up -d
  213.         - echo "版本已经更新到 ${DRONE_COMMIT:0:10}, ${DRONE_COMMIT_MESSAGE}"
  214.     when:
  215.       branch:
  216.         - master
  218. volumes:
  219.   - name: docker-sock
  220.     host:
  221.       path: /var/run/docker.sock
  223. trigger:
  224.   branch:
  225.     - develop
  226.     - master
  227.   event:
  228.     - rollback
  229.     - promote

构建时整个流程链路

  1. 从 git 拉取代码
  2. docker build 打镜像
  3. docker push 推送到私有镜像
  4. ssh 登录到远程主机拉取代码
  5. docker compose 运行 此过程通常会拉取私有镜像中的新镜像

回滚时整个流程链路
由于在创建镜像的时候将git的commit放在了镜像后面作为tag，所以回滚时直接指定 git commit 值进行回滚，在docker-compose中引用的镜像格式为 image:gitcommit 所以很方便。

后来我发现这种方式太慢了，而且很不稳定，有时候会卡住完全不动，所以改成了直接在宿主机构建，这时候需要做以下修改

  1. kind: pipeline
  2. # type 使用 exec
  3. type: exec
  4. name: CI-building

第二种方式，type 使用 exec，此时则需要安装一个扩展，否则这个机制使用不了

  1. curl -L https://github.com/drone-runners/drone-runner-exec/releases/latest/download/drone_runner_exec_linux_amd64.tar.gz | tar zx
  2. sudo install -t /usr/local/bin drone-runner-exec
  4. vim /etc/drone-runner-exec/config

将这些变量保存到config中：

DRONE_RPC_PROTO=http
DRONE_RPC_HOST=drone.yoururl.com
DRONE_RPC_SECRET=之前部署时的自定义secret

完成以上操作后，安装并启动它

  1. drone-runner-exec service install
  2. drone-runner-exec service start
  3. # drone-runner-exec service stop

这样就可以直接在 Drone 执行自定以的bash命令了。

下有完整的配置文件

  1. kind: pipeline
  2. type: exec
  3. name: CI-building
  5. environment:
  6.   GOOS: linux
  7.   GOARCH: amd64
  9. steps:
  10.   - name: CI
  11.     # 这里一定要在变量中配置go环境，否则 go mod tidy 这类命令如果有下载依赖包，每次构建都不会缓存
  12.     environment:
  13.       GOPATH: /root/go
  14.       PROJECT_API_NAME:
  15.         from_secret: PROJECT_API_NAME
  16.       PROJECT_RPC_NAME:
  17.         from_secret: PROJECT_RPC_NAME
  18.       DOMAIN_PREFIX:
  19.         from_secret: DOMAIN_PREFIX
  20.     commands:
  21.       - pwd
  22.       - cd /home/xxx/app_project/${DRONE_REPO_NAME}
  23.       - git switch ${DRONE_COMMIT_BRANCH} 
  24.       - git branch --set-upstream-to=origin/${DRONE_COMMIT_BRANCH} ${DRONE_COMMIT_BRANCH}
  25.       - git pull
  26.       - export PATH=$PATH:/usr/local/go/bin
  27.       - go env
  28.       - go version
  29.       - go build -ldflags="-s -w" -o ./app api/*.go
  30.       - docker build -f api/Dockerfile -t $${PROJECT_API_NAME}:latest .
  31.       - docker tag $${PROJECT_API_NAME}:latest harbor.xxx.com/xxx/$${PROJECT_API_NAME}:${DRONE_COMMIT:0:10}
  32.       - docker login harbor.xxx.com -u admin -p xxx
  33.       - docker push harbor.xxx.com/xxx/$${PROJECT_API_NAME}:${DRONE_COMMIT:0:10}
  34.       - echo "API构建完成, 你可以通过 docker pull harbor.xxx.com/xxx/$PROJECT_API_NAME:${DRONE_COMMIT:0:10} 获取镜像"
  35.       - |
  36.         if [ -d "rpc" ]; then
  37.           go build -ldflags="-s -w" -o ./rpc_app rpc/*.go
  38.           docker build -f rpc/Dockerfile -t $${PROJECT_RPC_NAME}:latest .
  39.           docker tag $${PROJECT_RPC_NAME}:latest harbor.xxx.com/xxx/$${PROJECT_RPC_NAME}:${DRONE_COMMIT:0:10}
  40.           docker push harbor.xxx.com/xxx/$${PROJECT_RPC_NAME}:${DRONE_COMMIT:0:10}
  41.           echo "RPC构建完成, 你可以通过 docker pull harbor.xxx.com/xxx/$${PROJECT_RPC_NAME}:${DRONE_COMMIT:0:10} 获取镜像"
  42.         fi
  44.   - name: stage-publish
  45.     environment:
  46.       PROJECT_API_NAME:
  47.         from_secret: PROJECT_API_NAME
  48.       PROJECT_RPC_NAME:
  49.         from_secret: PROJECT_RPC_NAME
  50.       DOMAIN_PREFIX:
  51.         from_secret: DOMAIN_PREFIX
  52.     when:
  53.       branch:
  54.         - develop
  55.     commands:
  56.       - echo ">>>>>>>>>>>>>>>>>>>>"
  57.       - cd /home/xxx/app_project/${DRONE_REPO_NAME}
  58.       - docker-compose down
  59.       - export DOMAIN_PREFIX=$${STAGE_DOMAIN_PREFIX}
  60.       - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  61.       - export ENV=""
  62.       - |
  63.         if [ -d "rpc" ]; then
  64.           echo $PROJECT_RPC_NAME 12345
  65.           docker tag $PROJECT_RPC_NAME:latest harbor.xxx.com/xxx/$PROJECT_RPC_NAME:${DRONE_COMMIT:0:10}
  66.           docker push harbor.xxx.com/xxx/$PROJECT_RPC_NAME:${DRONE_COMMIT:0:10}
  67.           docker images | grep $${PROJECT_RPC_NAME} | awk '{print $3}' | xargs docker rmi -f
  68.         fi
  69.       - docker images | grep $${PROJECT_API_NAME} | awk '{print $3}' | xargs docker rmi -f
  70.       - docker-compose up -d
  72. volumes:
  73.   - name: docker-sock
  74.     host:
  75.       path: /var/run/docker.sock
  77. trigger:
  78.   branch:
  79.     - develop
  80.     - master
  81.   event:
  82.     include:
  83.       - push
  84.       - custom
  85.       - merge_request
  87. ---
  88. kind: pipeline
  89. type: docker
  90. name: CI-Master-publish
  92. depends_on:
  93.   - CI-building
  95. steps:
  96.   - name: prod-publish
  97.     image: appleboy/drone-ssh
  98.     environment:
  99.       PROD_DOMAIN_PREFIX:
  100.         from_secret: PROD_DOMAIN_PREFIX
  101.       PROJECT_API_NAME:
  102.         from_secret: PROJECT_API_NAME
  103.       PROJECT_RPC_NAME:
  104.         from_secret: PROJECT_RPC_NAME
  105.     settings:
  106.       host:
  107.         from_secret: PROD_HOST_IP
  108.       username:
  109.         from_secret: PROD_HOST_NAME
  110.       password: 
  111.         from_secret: PROD_HOST_PWD
  112.       port: 22
  113.       command_timeout: 360s
  114.       envs: [PROJECT_API_NAME,PROD_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  115.       script:
  116.         - bash
  117.         - cd /xxx/app_project/${DRONE_REPO_NAME}
  118.         - git pull https://userName:userPwd@gitea.xxx.com/xxx/${DRONE_REPO_NAME}.git
  119.         - docker login harbor.xxx.com -u admin -p xxx
  120.         - git pull origin
  121.         - git branch
  122.         - git fetch origin
  123.         - git checkout master
  124.         - export ENV="-prod"
  125.         - export DOMAIN_PREFIX=$${PROD_DOMAIN_PREFIX}
  126.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  127.         - docker-compose down
  128.         - docker images | grep $${PROJECT_API_NAME} | awk '{print $3}' | xargs docker rmi -f
  129.         - docker images | grep $${PROJECT_RPC_NAME} | awk '{print $3}' | xargs docker rmi -f
  130.         - docker-compose up -d
  131.     retry:
  132.       limit: 2
  133.       delay: 360s
  135. volumes:
  136.   - name: docker-sock
  137.     host:
  138.       path: /var/run/docker.sock
  140. trigger:
  141.   branch:
  142.     - master
  143.   event:
  144.     include:
  145.       - push
  146.       - custom
  147.       - merge_request
  149. ---
  150. kind: pipeline
  151. type: docker
  152. name: promote/rollback
  154. steps:
  155.   - name: stage-rollback
  156.     image: appleboy/drone-ssh
  157.     environment:
  158.       PROJECT_API_NAME:
  159.         from_secret: PROJECT_API_NAME
  160.       PROJECT_RPC_NAME:
  161.         from_secret: PROJECT_RPC_NAME
  162.       DOMAIN_PREFIX:
  163.         from_secret: DOMAIN_PREFIX
  164.     settings:
  165.       host:
  166.         from_secret: STAGE_HOST_IP
  167.       username:
  168.         from_secret: STAGE_HOST_NAME
  169.       password:
  170.         from_secret: STAGE_HOST_PWD
  171.       port: 22
  172.       command_timeout: 300s
  173.       envs: [PROJECT_API_NAME,STAGE_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  174.       script:
  175.         - cd /home/xxx/app_project/${DRONE_REPO_NAME}
  176.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  177.         - export DOMAIN_PREFIX=$${STAGE_DOMAIN_PREFIX}
  178.         - export ENV=""
  179.         - docker-compose down
  180.         - docker-compose up -d
  181.         - echo "版本已经更新到 ${DRONE_COMMIT:0:10}, ${DRONE_COMMIT_MESSAGE}"
  182.     when:
  183.       branch:
  184.         - develop
  186.   - name: prod-rollback
  187.     image: appleboy/drone-ssh
  188.     environment:
  189.       PROJECT_API_NAME:
  190.         from_secret: PROJECT_API_NAME
  191.       PROJECT_RPC_NAME:
  192.         from_secret: PROJECT_RPC_NAME
  193.       DOMAIN_PREFIX:
  194.         from_secret: DOMAIN_PREFIX
  195.     settings:
  196.       host:
  197.         from_secret: PROD_HOST_IP
  198.       username:
  199.         from_secret: PROD_HOST_NAME
  200.       password:
  201.         from_secret: PROD_HOST_PWD
  202.       port: 22
  203.       command_timeout: 300s
  204.       envs: [PROJECT_API_NAME,STAGE_DOMAIN_PREFIX,PROJECT_RPC_NAME]
  205.       script:
  206.         - cd /xxx/app_project/${DRONE_REPO_NAME}
  207.         - export GIT_COMMIT=${DRONE_COMMIT:0:10}
  208.         - export DOMAIN_PREFIX=$${STAGE_DOMAIN_PREFIX}
  209.         - export ENV="-prod"
  210.         - docker-compose down
  211.         - docker-compose up -d
  212.         - echo "版本已经更新到 ${DRONE_COMMIT:0:10}, ${DRONE_COMMIT_MESSAGE}"
  213.     when:
  214.       branch:
  215.         - master
  217. volumes:
  218.   - name: docker-sock
  219.     host:
  220.       path: /var/run/docker.sock
  222. trigger:
  223.   branch:
  224.     - develop
  225.     - master
  226.   event:
  227.     - rollback
  228.     - promote
drone devops

声明: 因编程语言版本更新较快，当前文章所涉及的语法或某些特性相关的信息并不一定完全适用于您当前所使用的版本，请仔细甄别。文章内容仅作为学习和参考，若有错误，欢迎指正。

讨论 支持 Markdown 语法 点击演示
回复
评论预览框

开发者

专题·造轮子

利用Go语言编写一个搜索引擎

编程笔记·热门
Docker容器间网络互通

Docker容器间网络互通

MySQL面试题，亿级数据库分表方案

MySQL面试题，亿级数据库分表方案

相关文章
开发者·注册登录
  • 获取验证码
  • 取消