Kill a forked process in a docker container make the container exit on Jenkins docker slave plugin -
i using docker build , testing. during testing, test cases launch separate process(mongod process) , after test cases finish kill process. during whole test cases running time, mongod
process generated , killed many times. problem have when mongod process killed, docker container exits code 137. docker file below:
from ubuntu run sed -i -e 's/us.archive.ubuntu.com/archive.ubuntu.com/g' /etc/apt/sources.list && apt-get update && apt-get install -y curl run apt-get update && apt-get install -y --no-install-recommends \ openssh-client openssh-server rsync vim \ dh-autoreconf libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev \ && curl -ol https://github.com/git/git/archive/v2.13.1.tar.gz \ && tar -zxf v2.13.1.tar.gz && cd git-2.13.1 && make configure && ./configure --prefix=/usr && make && make install && cd .. \ && apt-get remove -y dh-autoreconf libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev \ && rm -rf v2.13.1.tar.gz git-2.13.1 \ && rm -rf /var/lib/apt/lists/* run echo "unsafe-perm = true" > /root/.npmrc \ && echo "australia/melbourne" > /etc/timezone # && dpkg-reconfigure -f noninteractive tzdata run curl -sl https://deb.nodesource.com/setup_7.x | bash - && apt-get install -y nodejs && npm install -g yarn run apt-get update && apt-get install -y --no-install-recommends \ build-essential \ python \ python-pip python-dev \ && rm -rf /var/lib/apt/lists/* \ && yarn global add node-gyp && pip install --upgrade pip && pip install setuptools run apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0c49f3730359a14518585931bc711f9ba15703c6 \ && echo "deb http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.4 main" | tee /etc/apt/sources.list.d/mongodb-org-3.4.list run apt-get update && apt-get install -y --no-install-recommends \ mongodb-org \ python-pip python-dev \ && pip install mtools pymongo numpy \ && rm -rf /var/lib/apt/lists/* user root run adduser jenkins run chmod 777 /home/jenkins add ./.aws /home/jenkins/.aws user jenkins
i use below command launch mongod
process:
mongod --dbpath /home/jenkins/.jenkins/data/db --logpath /home/jenkins/.jenkins/data/mongod.log --port 27017 --logappend --fork
there no problem me run project test cases locally. think relates docker. docker running in coreos
. version is:
$ docker --version docker version 17.05.0-ce, build 89658be
this container launched jenkins docker slave
plugin. after investigation, works fine if launch container manually docker run -it -d $name bash
not work if container launched jenkins docker slave
. wonder launch command may cause issue. how can fix issue without changing docker launcher command?
below docker inspect
output:
[ { "id": "81ffad4b15115077dda23c221eb8053915f41cdf0e75de4417d23f81df12f87f", "created": "2017-07-28t01:08:40.598084613z", "path": "/trampoline", "args": [ "wait" ], "state": { "status": "running", "running": true, "paused": false, "restarting": false, "oomkilled": false, "dead": false, "pid": 14152, "exitcode": 0, "error": "", "startedat": "2017-07-28t01:08:41.044051569z", "finishedat": "0001-01-01t00:00:00z" }, "image": "sha256:7d1e60ecf32337654f46c54b05e3db11f724f12c4f879ccd42c9566c59edc6b2", "resolvconfpath": "/var/lib/docker/containers/097e51587d94fa0282ee51f6b84c006a6a410aadb2891f24b25caf41a1d14f39/resolv.conf", "hostnamepath": "/var/lib/docker/containers/097e51587d94fa0282ee51f6b84c006a6a410aadb2891f24b25caf41a1d14f39/hostname", "hostspath": "/var/lib/docker/containers/097e51587d94fa0282ee51f6b84c006a6a410aadb2891f24b25caf41a1d14f39/hosts", "logpath": "/var/lib/docker/containers/81ffad4b15115077dda23c221eb8053915f41cdf0e75de4417d23f81df12f87f/81ffad4b15115077dda23c221eb8053915f41cdf0e75de4417d23f81df12f87f-json.log", "name": "/heuristic_ritchie", "restartcount": 0, "driver": "overlay2", "mountlabel": "system_u:object_r:svirt_lxc_file_t:s0:c896,c995", "processlabel": "system_u:system_r:svirt_lxc_net_t:s0:c896,c995", "apparmorprofile": "", "execids": [ "a827e4666391224dc9b3e7f94ea3a7615fb5215c82de6a481289fa125bf9495f" ], "hostconfig": { "binds": null, "containeridfile": "", "logconfig": { "type": "json-file", "config": {} }, "networkmode": "container:097e51587d94fa0282ee51f6b84c006a6a410aadb2891f24b25caf41a1d14f39", "portbindings": {}, "restartpolicy": { "name": "no", "maximumretrycount": 0 }, "autoremove": false, "volumedriver": "", "volumesfrom": [ "097e51587d94fa0282ee51f6b84c006a6a410aadb2891f24b25caf41a1d14f39" ], "capadd": null, "capdrop": null, "dns": [], "dnsoptions": [], "dnssearch": [], "extrahosts": null, "groupadd": null, "ipcmode": "container:097e51587d94fa0282ee51f6b84c006a6a410aadb2891f24b25caf41a1d14f39", "cgroup": "", "links": null, "oomscoreadj": 0, "pidmode": "", "privileged": false, "publishallports": false, "readonlyrootfs": false, "securityopt": [ "label=user:system_u", "label=role:system_r", "label=type:svirt_lxc_net_t", "label=level:s0:c896,c995" ], "utsmode": "", "usernsmode": "", "shmsize": 67108864, "runtime": "runc", "consolesize": [ 0, 0 ], "isolation": "", "cpushares": 0, "memory": 4294967296, "nanocpus": 0, "cgroupparent": "", "blkioweight": 0, "blkioweightdevice": null, "blkiodevicereadbps": null, "blkiodevicewritebps": null, "blkiodevicereadiops": null, "blkiodevicewriteiops": null, "cpuperiod": 0, "cpuquota": 0, "cpurealtimeperiod": 0, "cpurealtimeruntime": 0, "cpusetcpus": "", "cpusetmems": "", "devices": [], "devicecgrouprules": null, "diskquota": 0, "kernelmemory": 0, "memoryreservation": 0, "memoryswap": 8589934592, "memoryswappiness": -1, "oomkilldisable": false, "pidslimit": 0, "ulimits": null, "cpucount": 0, "cpupercent": 0, "iomaximumiops": 0, "iomaximumbandwidth": 0 }, "graphdriver": { "data": { "lowerdir": "/var/lib/docker/overlay2/bf2b1c63a0862bcc09c3a126e41b6658b60178ade943a8844780a24b01cf2f7c-init/diff:/var/lib/docker/overlay2/be89c1495150690fd934ba56411d144f7de43d06d268fe4b2d0e12552bab6b31/diff:/var/lib/docker/overlay2/d222b270a29034ff81603a80776dd05a259369dcb721515bb2ad08039e9327f1/diff:/var/lib/docker/overlay2/0f1c10959b14c67f2776cd5696a8281f514a42505ec2e0aab759f8543f0e6eed/diff:/var/lib/docker/overlay2/adfec4e3f9b8b1252520f8a56ee27d4cf5fb3f3a01dae344c9932f5bda7d732f/diff:/var/lib/docker/overlay2/e350f5d2bcc4252a5f067eb72b5ad65c6c620178347373736e1308c35e0a17d3/diff:/var/lib/docker/overlay2/2ed71b40ecf304222f071b0ccfc14f10dbd7566de01d16ff80169dea80dd72d4/diff:/var/lib/docker/overlay2/1b20e86eb201191ce6fd4d904f1bfd95fe3423bd6d083cd763c7995a518c9e23/diff:/var/lib/docker/overlay2/35c6c2859ec66d741643c7f9ff47e8513504e7560ea63369f671b305a66edaa1/diff:/var/lib/docker/overlay2/50e277363a9f73f105e6a60edb47c257b04f8383e4575e0cb67e6d628148c305/diff:/var/lib/docker/overlay2/3163f8c2fc0c94623b3390178b5d9de9938d2e9399df0f0341384f8d0fc75761/diff:/var/lib/docker/overlay2/1dc179342c7786f88e98fd68f72dcb187d8524b203ce6122158cf68d94df3a3a/diff:/var/lib/docker/overlay2/f3f749adb1ea12915a0bc1b6258682cbf7571288656325152784b02039b56fd1/diff:/var/lib/docker/overlay2/deb0c7c423d5f685db5213536810c71fae89fee553d830f6a7cfba3b29853097/diff", "mergeddir": "/var/lib/docker/overlay2/bf2b1c63a0862bcc09c3a126e41b6658b60178ade943a8844780a24b01cf2f7c/merged", "upperdir": "/var/lib/docker/overlay2/bf2b1c63a0862bcc09c3a126e41b6658b60178ade943a8844780a24b01cf2f7c/diff", "workdir": "/var/lib/docker/overlay2/bf2b1c63a0862bcc09c3a126e41b6658b60178ade943a8844780a24b01cf2f7c/work" }, "name": "overlay2" }, "mounts": [ { "name": "46705245e5a0a71c14665c843a34ee349f19b4cc0efeed655e9b3575b0b83a3d", "source": "/var/lib/docker/volumes/46705245e5a0a71c14665c843a34ee349f19b4cc0efeed655e9b3575b0b83a3d/_data", "destination": "/home/jenkins/.jenkins", "driver": "local", "mode": "", "rw": true, "propagation": "" } ], "config": { "hostname": "097e51587d94", "domainname": "", "user": "10000:10000", "attachstdin": false, "attachstdout": true, "attachstderr": true, "tty": false, "openstdin": false, "stdinonce": false, "env": [ "tmpdir=/home/jenkins/.jenkins/.tmp", "path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "npm_config_loglevel=info", "node_version=7.10.0", "yarn_version=0.24.4" ], "cmd": [ "/trampoline", "wait" ], "image": "jenkins-mongo-3.4", "volumes": null, "workingdir": "/home/jenkins/.jenkins", "entrypoint": null, "onbuild": null, "labels": { "maintainer": "zhaoyi0113@gmail.com" } }, "networksettings": { "bridge": "", "sandboxid": "", "hairpinmode": false, "linklocalipv6address": "", "linklocalipv6prefixlen": 0, "ports": {}, "sandboxkey": "", "secondaryipaddresses": null, "secondaryipv6addresses": null, "endpointid": "", "gateway": "", "globalipv6address": "", "globalipv6prefixlen": 0, "ipaddress": "", "ipprefixlen": 0, "ipv6gateway": "", "macaddress": "", "networks": {} } } ]
instead of using mongod process pid1, use supervisor instead.
so start mongodb using supervisor ( if needed @ , test not start ).
this way, test can start / stop process using:
supervisorctl stop mongod
, start again using supervisorctl start mongod
stopping mongod service not exit container, since supervisord pid1
if want test start using binary , not need pid tracking , connivence, can use busyscript pid , start mongod process in container using tests.
a busyscript looks this
#!/bin/sh #set -x pid=0 # sigterm-handler term_handler() { if [ $pid -ne 0 ]; kill -sigterm "$pid" wait "$pid" fi exit 143; # 128 + 15 -- sigterm } # setup handlers # on callback, kill last background process, `tail -f /dev/null` , execute specified handler trap 'kill ${!}; term_handler' sigterm echo "started dw php code" # wait forever while true tail -f /dev/null & wait ${!} done
alternatively, lot more effort dockish way be, use containers. 1 test-runner host-mounted docker socket. 1 run test , start mongod contaienr , kill @ like, test kill /rm mongod containers instead of process - , start container again.
Comments
Post a Comment