docker
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
docker [2019/11/28 18:40] – [Proposed solution] mimbert | docker [2023/09/28 17:24] (current) – [Proposed solution] cmorin | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ===== Docker ===== | ||
+ | |||
Since November 2019, a new way to conduct experiments is implemented in CorteXlab. | Since November 2019, a new way to conduct experiments is implemented in CorteXlab. | ||
- | ====== Why ====== | + | ==== Why ==== |
The legacy way of running experiments is to run one (or more) commands on each node of the experiment. These commands are run from the minus task. This has some drawbacks: | The legacy way of running experiments is to run one (or more) commands on each node of the experiment. These commands are run from the minus task. This has some drawbacks: | ||
Line 17: | Line 19: | ||
* All the results are gathered with the task directory, compressed, and sent back to airlock. This means that the results may include huge unneeded things, such as experiment code, input datasets, etc. | * All the results are gathered with the task directory, compressed, and sent back to airlock. This means that the results may include huge unneeded things, such as experiment code, input datasets, etc. | ||
- | ====== Proposed solution | + | ==== Proposed solution ==== |
The proposed solution is to use [[https:// | The proposed solution is to use [[https:// | ||
- | * [[https:// | + | * [[https:// |
* Preparing an image is a much more convenient process than preparing a task, when it comes to complex software bundles such as TensorFlow, OpenBTS, etc. One just needs to install dependencies and build as if it was a real machine, there is no need to tweak or hack build steps, it just works directly. | * Preparing an image is a much more convenient process than preparing a task, when it comes to complex software bundles such as TensorFlow, OpenBTS, etc. One just needs to install dependencies and build as if it was a real machine, there is no need to tweak or hack build steps, it just works directly. | ||
* Image preparation can be an interactive process, or it can be automated with a [[https:// | * Image preparation can be an interactive process, or it can be automated with a [[https:// | ||
Line 26: | Line 28: | ||
* When running a task, images are instanciated to [[https:// | * When running a task, images are instanciated to [[https:// | ||
* The experiment results are structured differently. For each node, there is one directory per container, containing the stdout/ | * The experiment results are structured differently. For each node, there is one directory per container, containing the stdout/ | ||
- | | + | * The homes of the users are NFS mounted on the nodes, and the home of the user is mounted in the containers |
docker.1574962805.txt.gz · Last modified: 2019/11/28 18:40 by mimbert