Docker build from Dockerfile with more memory -

how docker build dockerfile more memory?

this different question allow more memory when docker build dockerfile

when installing software natively, there enough memory build , install marian tool

but when building docker image using dockerfile https://github.com/marian-nmt/marian/blob/master/scripts/docker/dockerfile.cpu , fails multiple memory exhausted errors

virtual memory exhausted: cannot allocate memory 

[out]:

step : run cmake $marianpath && make -j  ---> running in 4867d166d17a -- c compiler identification gnu 5.4.0 -- cxx compiler identification gnu 5.4.0 -- check working c compiler: /usr/bin/cc -- check working c compiler: /usr/bin/cc -- works -- detecting c compiler abi info -- detecting c compiler abi info - done -- detecting c compile features -- detecting c compile features - done -- check working cxx compiler: /usr/bin/c++ -- check working cxx compiler: /usr/bin/c++ -- works -- detecting cxx compiler abi info -- detecting cxx compiler abi info - done -- detecting cxx compile features -- detecting cxx compile features - done cuda_toolkit_root_dir not found or specified -- cannot find cuda libraries. compiling without them. -- not find cuda (missing:  cuda_toolkit_root_dir cuda_nvcc_executable cuda_include_dirs cuda_cudart_library)  -- looking pthread.h -- looking pthread.h - found -- looking pthread_create -- looking pthread_create - not found -- looking pthread_create in pthreads -- looking pthread_create in pthreads - not found -- looking pthread_create in pthread -- looking pthread_create in pthread - found -- found threads: true   -- boost version: 1.58.0 -- found following boost libraries: --   system --   filesystem --   program_options --   timer --   iostreams --   python --   thread --   chrono --   regex --   date_time --   atomic -- found python -- found pythonlibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.12", minimum required "2.7")  -- found zlib: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")  -- found git: /usr/bin/git (found version "2.7.4")  -- git version: 5abc774 -- found sparsehash: /usr/include   -- configuring done -- generating done -- build files have been written to: /marian/build scanning dependencies of target fast_align scanning dependencies of target extract_lex scanning dependencies of target libcnpy scanning dependencies of target atools scanning dependencies of target libyaml-cpp-amun [  2%] building cxx object src/amun/3rd_party/extract_lex/cmakefiles/extract_lex.dir/exception.cpp.o [  2%] building cxx object src/amun/3rd_party/extract_lex/cmakefiles/extract_lex.dir/utils.cpp.o [  3%] building cxx object src/amun/3rd_party/fast_align/cmakefiles/fast_align.dir/src/ttables.cc.o [  6%] building cxx object src/amun/3rd_party/fast_align/cmakefiles/fast_align.dir/src/fast_align.cc.o [  7%] building cxx object src/amun/3rd_party/extract_lex/cmakefiles/extract_lex.dir/extract-lex-main.cpp.o [ 10%] building cxx object src/amun/3rd_party/cmakefiles/libcnpy.dir/cnpy/cnpy.cpp.o [ 10%] building cxx object src/amun/3rd_party/fast_align/cmakefiles/atools.dir/src/atools.cc.o [ 10%] building cxx object src/amun/3rd_party/fast_align/cmakefiles/atools.dir/src/alignment_io.cc.o [ 11%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/regex_yaml.cpp.o [ 12%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/scanner.cpp.o [ 14%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/convert.cpp.o [ 15%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/stream.cpp.o [ 16%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/memory.cpp.o [ 17%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/node.cpp.o [ 19%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/directives.cpp.o [ 20%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/null.cpp.o [ 21%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/emitfromevents.cpp.o [ 23%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/singledocparser.cpp.o [ 24%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/emitterstate.cpp.o [ 25%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/scantag.cpp.o [ 26%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/simplekey.cpp.o [ 28%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/binary.cpp.o [ 29%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/parser.cpp.o [ 30%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/nodeevents.cpp.o [ 32%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/emit.cpp.o [ 33%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/exp.cpp.o [ 35%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/nodebuilder.cpp.o [ 38%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/scanscalar.cpp.o [ 39%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/emitter.cpp.o [ 41%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/tag.cpp.o [ 42%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/scantoken.cpp.o [ 43%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/ostream_wrapper.cpp.o scanning dependencies of target libcommon [ 37%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/node_data.cpp.o [ 44%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/contrib/graphbuilder.cpp.o [ 46%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/emitterutils.cpp.o [ 34%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/contrib/graphbuilderadapter.cpp.o [ 47%] building cxx object src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/parse.cpp.o [ 48%] building cxx object src/amun/cmakefiles/libcommon.dir/common/scorer.cpp.o [ 50%] building cxx object src/amun/cmakefiles/libcommon.dir/common/git_version.cpp.o [ 51%] building cxx object src/amun/cmakefiles/libcommon.dir/common/base_matrix.cpp.o [ 52%] building cxx object src/amun/cmakefiles/libcommon.dir/common/sentence.cpp.o [ 53%] building cxx object src/amun/cmakefiles/libcommon.dir/common/config.cpp.o [ 55%] building cxx object src/amun/cmakefiles/libcommon.dir/common/utils.cpp.o [ 56%] building cxx object src/amun/cmakefiles/libcommon.dir/common/vocab.cpp.o [ 65%] building cxx object src/amun/cmakefiles/libcommon.dir/common/filter.cpp.o [ 66%] building cxx object src/amun/cmakefiles/libcommon.dir/common/god.cpp.o [ 67%] building cxx object src/amun/cmakefiles/libcommon.dir/common/exception.cpp.o [ 69%] building cxx object src/amun/cmakefiles/libcommon.dir/common/search.cpp.o [ 69%] building cxx object src/amun/cmakefiles/libcommon.dir/common/hypothesis.cpp.o [ 70%] building cxx object src/amun/cmakefiles/libcommon.dir/common/translation_task.cpp.o [ 70%] building cxx object src/amun/cmakefiles/libcommon.dir/common/printer.cpp.o [ 70%] building cxx object src/amun/cmakefiles/libcommon.dir/common/sentences.cpp.o [ 70%] building cxx object src/amun/cmakefiles/libcommon.dir/common/history.cpp.o [ 70%] building cxx object src/amun/cmakefiles/libcommon.dir/common/logging.cpp.o [ 70%] building cxx object src/amun/cmakefiles/libcommon.dir/common/processor/bpe.cpp.o [ 71%] building cxx object src/amun/cmakefiles/libcommon.dir/common/output_collector.cpp.o [ 73%] building cxx object src/amun/cmakefiles/libcommon.dir/common/types.cpp.o [ 74%] building cxx object src/amun/cmakefiles/libcommon.dir/common/loader.cpp.o virtual memory exhausted: cannot allocate memory virtual memory exhausted: cannot allocate memory virtual memory exhausted: cannot allocate memory 

and leads to:

src/amun/cmakefiles/libcommon.dir/build.make:254: recipe target 'src/amun/cmakefiles/libcommon.dir/common/loader.cpp.o' failed virtual memory exhausted: cannot allocate memory make[2]: *** [src/amun/cmakefiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o] error 1 src/amun/cmakefiles/cpumode.dir/build.make:110: recipe target 'src/amun/cmakefiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o' failed [ 79%] built target libcnpy virtual memory exhausted: cannot allocate memory src/amun/cmakefiles/libcommon.dir/build.make:326: recipe target 'src/amun/cmakefiles/libcommon.dir/common/printer.cpp.o' failed make[2]: *** [src/amun/cmakefiles/libcommon.dir/common/printer.cpp.o] error 1 cmakefiles/makefile2:340: recipe target 'src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/all' failed make[1]: *** [src/amun/3rd_party/yaml-cpp/cmakefiles/libyaml-cpp-amun.dir/all] error 2 cmakefiles/makefile2:182: recipe target 'src/amun/cmakefiles/libcommon.dir/all' failed make[1]: *** [src/amun/cmakefiles/libcommon.dir/all] error 2 make[1]: *** [src/amun/cmakefiles/cpumode.dir/all] error 2 cmakefiles/makefile2:110: recipe target 'src/amun/cmakefiles/cpumode.dir/all' failed make: *** [all] error 2 

looking @ docker build --help, there options control memory usage:

$ docker build --help  usage:  docker build [options] path | url | -  build image dockerfile  options:       --build-arg list             set build-time variables (default [])       --cache-from stringslice     images consider cache sources       --cgroup-parent string       optional parent cgroup container       --compress                   compress build context using gzip       --cpu-period int             limit cpu cfs (completely fair scheduler) period       --cpu-quota int              limit cpu cfs (completely fair scheduler) quota   -c, --cpu-shares int             cpu shares (relative weight)       --cpuset-cpus string         cpus in allow execution (0-3, 0,1)       --cpuset-mems string         mems in allow execution (0-3, 0,1)       --disable-content-trust      skip image verification (default true)   -f, --file string                name of dockerfile (default 'path/dockerfile')       --force-rm                   remove intermediate containers       --help                       print usage       --isolation string           container isolation technology       --label list                 set metadata image (default [])   -m, --memory string              memory limit       --memory-swap string         swap limit equal memory plus swap: '-1' enable unlimited swap       --network string             set networking mode run instructions during build (default "default")       --no-cache                   not use cache when building image       --pull                       attempt pull newer version of image   -q, --quiet                      suppress build output , print image id on success       --rm                         remove intermediate containers after successful build (default true)       --security-opt stringslice   security options       --shm-size string            size of /dev/shm, default value 64mb   -t, --tag list                   name , optionally tag in 'name:tag' format (default [])       --ulimit ulimit              ulimit options (default []) 

but couldn't figure out correct syntax of put -m option -_-|||

it isn't before dockerfile:

# before docker file. $ docker build -m 4g dockerfile.cpu -t ibot-cpu .  "docker build" requires 1 argument(s). see 'docker build --help'.  usage:  docker build [options] path | url | -  build image dockerfile 

it isn't after dockerfile before -t

# before -t $ docker build dockerfile.cpu -m 4g -t ibot-cpu .  "docker build" requires 1 argument(s). see 'docker build --help'.  usage:  docker build [options] path | url | -  build image dockerfile 

it isn't after -t before local path

# before local path $ docker build dockerfile.cpu -t ibot-cpu -m 4g .  "docker build" requires 1 argument(s). see 'docker build --help'.  usage:  docker build [options] path | url | -  build image dockerfile 

it isn't after local path @ end too...

# @ end... $ docker build dockerfile.cpu -t ibot-cpu . -m 4g  "docker build" requires 1 argument(s). see 'docker build --help'.  usage:  docker build [options] path | url | -  build image dockerfile 

how docker build dockerfile more memory?

my docker version:

docker version client:  version:      17.03.1-ce  api version:  1.27  go version:   go1.7.5  git commit:   c6d412e  built:        tue mar 28 00:40:02 2017  os/arch:      darwin/amd64  server:  version:      17.04.0-ce  api version:  1.28 (minimum version 1.12)  go version:   go1.7.5  git commit:   4845c56  built:        wed apr  5 18:45:47 2017  os/arch:      linux/amd64  experimental: false 

it not order. dockerfile must specified -f

docker build -f dockerfile.cpu -t ibot-cpu -m 4g . 

as can see in help.

however, take account default docker not limit container memory. can take hole free memory.

as can see on osx, runs docker on linux vm. configure max memory clicking whale icon in task bar. default 2g.

for further information please see other answer: how assign more memory docker container