From 50dbe6bdcd0964f266507e1eb577f74f7770c734 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:00:11 +0100 Subject: [PATCH 01/40] Update singularity/Singularity_tp.md --- singularity/Singularity_tp.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/singularity/Singularity_tp.md b/singularity/Singularity_tp.md index 6c19fdf..c272148 100644 --- a/singularity/Singularity_tp.md +++ b/singularity/Singularity_tp.md @@ -5,15 +5,17 @@ <<< Author : +* Jacques Lagnel <jacques.lagnel@inrae.fr>, +based on: * Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, * Rémy Dernat <remy.dernat@umontpellier.fr>, * Martin Souchal <souchal@apc.in2p3.fr> All the files mentioned in this recipe are available in the following git repository : -https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git + [source, bash] -git clone https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git +git clone You can build an HTML version : [source, bash] @@ -21,12 +23,7 @@ asciidoctor -d book -a toc singularity_tp_4h.adoc This recipe describes how to build an Ubuntu image using Singularity, with the latest Singularity 2 version (2.6). You can download Singularity and find some docs here: https://www.sylabs.io/docs/ -We are going to install singularity in an Ubuntu VM on the Amazon cloud (AWS), then we are gonna build containers and run them on an HPC cluster (meso@LR). - -* User: singu -* Password: singularity - -Everyone has access to an empty ready to use Ubuntu VM. Please refer to the etherpad to know your VM IP address. +We are going to install singularity on your local linux machine (debian based distribution) or in an Ubuntu/debian VM (virtualbob), then we are gonna build containers and run them on an HPC cluster genotoul. HELP: -- GitLab From 006fb1fafcf2270a80111fa00eeec2b75afc1b33 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:01:27 +0100 Subject: [PATCH 02/40] Update Singularity_tp.md -- GitLab From dde51fbdf5c3bfe9aa5d82fd02a2456411b43102 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:02:34 +0100 Subject: [PATCH 03/40] Update Singularity_tp.md --- singularity/{Singularity_tp.md => Singularity_tp.adoc} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename singularity/{Singularity_tp.md => Singularity_tp.adoc} (100%) diff --git a/singularity/Singularity_tp.md b/singularity/Singularity_tp.adoc similarity index 100% rename from singularity/Singularity_tp.md rename to singularity/Singularity_tp.adoc -- GitLab From 12d76b2b614ad775af74e4ed92ffc9e94fedac3b Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:05:08 +0100 Subject: [PATCH 04/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index c272148..3bea74b 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -6,7 +6,7 @@ Author : * Jacques Lagnel <jacques.lagnel@inrae.fr>, -based on: += based on: * Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, * Rémy Dernat <remy.dernat@umontpellier.fr>, * Martin Souchal <souchal@apc.in2p3.fr> @@ -15,7 +15,7 @@ All the files mentioned in this recipe are available in the following git reposi [source, bash] -git clone +git clone https://forgemia.inra.fr/inter_cati_omics/reproductibility.git You can build an HTML version : [source, bash] -- GitLab From 1348c2d6b9702b1480dde5c7a22ca0f756c7cb9f Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:09:05 +0100 Subject: [PATCH 05/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 3bea74b..09477d6 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -3,7 +3,8 @@ :icons: font <<< -Author : +.Some Authors +[circle] * Jacques Lagnel <jacques.lagnel@inrae.fr>, = based on: -- GitLab From ed1f68aa258b5652795419fe859afa186d4d2dd6 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:10:33 +0100 Subject: [PATCH 06/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 09477d6..e1e8749 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -3,11 +3,12 @@ :icons: font <<< +[%hardbreaks] .Some Authors [circle] -* Jacques Lagnel <jacques.lagnel@inrae.fr>, -= based on: +- Jacques Lagnel <jacques.lagnel@inrae.fr>, +based on: * Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, * Rémy Dernat <remy.dernat@umontpellier.fr>, * Martin Souchal <souchal@apc.in2p3.fr> -- GitLab From 1adf4e9feee961f1084ec1a1b47af167dbba79f1 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:11:14 +0100 Subject: [PATCH 07/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index e1e8749..d510de1 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -7,8 +7,9 @@ .Some Authors [circle] -- Jacques Lagnel <jacques.lagnel@inrae.fr>, -based on: +- Jacques Lagnel <jacques.lagnel@inrae.fr> + +From : * Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, * Rémy Dernat <remy.dernat@umontpellier.fr>, * Martin Souchal <souchal@apc.in2p3.fr> -- GitLab From a684d2dd04d1c975f30623b526663d9fbda8e22e Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:12:08 +0100 Subject: [PATCH 08/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index d510de1..9fd5be0 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -4,15 +4,13 @@ <<< [%hardbreaks] -.Some Authors +. Authors [circle] - - Jacques Lagnel <jacques.lagnel@inrae.fr> - -From : -* Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, -* Rémy Dernat <remy.dernat@umontpellier.fr>, -* Martin Souchal <souchal@apc.in2p3.fr> + From : + Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, + Rémy Dernat <remy.dernat@umontpellier.fr>, + Martin Souchal <souchal@apc.in2p3.fr> All the files mentioned in this recipe are available in the following git repository : -- GitLab From eaf6a7fcbd191f7438162f151659d237dd1826d1 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:13:13 +0100 Subject: [PATCH 09/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 9fd5be0..5ab7526 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -8,9 +8,9 @@ [circle] - Jacques Lagnel <jacques.lagnel@inrae.fr> From : - Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, - Rémy Dernat <remy.dernat@umontpellier.fr>, - Martin Souchal <souchal@apc.in2p3.fr> +- Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, +- Rémy Dernat <remy.dernat@umontpellier.fr>, +- Martin Souchal <souchal@apc.in2p3.fr> All the files mentioned in this recipe are available in the following git repository : -- GitLab From d510cd44338c4f7ef97c03255a34a34a658f4f2b Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:13:40 +0100 Subject: [PATCH 10/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 5ab7526..eec73a4 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -7,7 +7,8 @@ . Authors [circle] - Jacques Lagnel <jacques.lagnel@inrae.fr> - From : +[%hardbreaks] +From : - Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, - Rémy Dernat <remy.dernat@umontpellier.fr>, - Martin Souchal <souchal@apc.in2p3.fr> -- GitLab From 57aefa42cb2fa971cea3779f4d91602de6239dee Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:14:38 +0100 Subject: [PATCH 11/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index eec73a4..67154c5 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -7,10 +7,13 @@ . Authors [circle] - Jacques Lagnel <jacques.lagnel@inrae.fr> -[%hardbreaks] + From : + - Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, + - Rémy Dernat <remy.dernat@umontpellier.fr>, + - Martin Souchal <souchal@apc.in2p3.fr> All the files mentioned in this recipe are available in the following git repository : -- GitLab From 33e6835ef244fb4524fc14028bcc00123349aae4 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:14:53 +0100 Subject: [PATCH 12/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 67154c5..f8b1496 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -3,7 +3,7 @@ :icons: font <<< -[%hardbreaks] + . Authors [circle] - Jacques Lagnel <jacques.lagnel@inrae.fr> -- GitLab From 83daab99e5fda314a315e2ebe49ea88f6906386e Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:15:19 +0100 Subject: [PATCH 13/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index f8b1496..f822e0d 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -10,11 +10,9 @@ From : -- Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, - -- Rémy Dernat <remy.dernat@umontpellier.fr>, - -- Martin Souchal <souchal@apc.in2p3.fr> +* Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, +* Rémy Dernat <remy.dernat@umontpellier.fr>, +* Martin Souchal <souchal@apc.in2p3.fr> All the files mentioned in this recipe are available in the following git repository : -- GitLab From 576716fd8e0182672dba21a341d71aaa58791e76 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:15:44 +0100 Subject: [PATCH 14/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index f822e0d..1aca7cb 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -3,13 +3,11 @@ :icons: font <<< - . Authors [circle] -- Jacques Lagnel <jacques.lagnel@inrae.fr> +* Jacques Lagnel <jacques.lagnel@inrae.fr> From : - * Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, * Rémy Dernat <remy.dernat@umontpellier.fr>, * Martin Souchal <souchal@apc.in2p3.fr> -- GitLab From d3951100a822f21a92eb72c51cb163c967135a40 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:16:05 +0100 Subject: [PATCH 15/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 1aca7cb..dd0c388 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -8,9 +8,9 @@ * Jacques Lagnel <jacques.lagnel@inrae.fr> From : -* Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, -* Rémy Dernat <remy.dernat@umontpellier.fr>, -* Martin Souchal <souchal@apc.in2p3.fr> + Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, + Rémy Dernat <remy.dernat@umontpellier.fr>, + Martin Souchal <souchal@apc.in2p3.fr> All the files mentioned in this recipe are available in the following git repository : -- GitLab From b3677e9e43c7efb76993079f9b9c32d12b0e532b Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:20:06 +0100 Subject: [PATCH 16/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index dd0c388..8c97479 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -7,7 +7,7 @@ [circle] * Jacques Lagnel <jacques.lagnel@inrae.fr> -From : +Based on : https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, Rémy Dernat <remy.dernat@umontpellier.fr>, Martin Souchal <souchal@apc.in2p3.fr> -- GitLab From a61bb00a4c7077e11c3d89069a3ddb996efdf2fd Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:20:54 +0100 Subject: [PATCH 17/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 8c97479..89750b0 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -8,9 +8,8 @@ * Jacques Lagnel <jacques.lagnel@inrae.fr> Based on : https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git - Alexandre Dehne Garcia <alexandre.dehne-garcia@inra.fr>, - Rémy Dernat <remy.dernat@umontpellier.fr>, - Martin Souchal <souchal@apc.in2p3.fr> + + Alexandre Dehne Garcia, Rémy Dernat, Martin Souchal All the files mentioned in this recipe are available in the following git repository : -- GitLab From cda21ddaeaa05b3a9668d68f81042458208b53fa Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:21:13 +0100 Subject: [PATCH 18/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 1 - 1 file changed, 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 89750b0..772bd42 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -8,7 +8,6 @@ * Jacques Lagnel <jacques.lagnel@inrae.fr> Based on : https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git - Alexandre Dehne Garcia, Rémy Dernat, Martin Souchal All the files mentioned in this recipe are available in the following git repository : -- GitLab From b70377a23ec86a6db2fd8ece76fdbe8a47306973 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:21:38 +0100 Subject: [PATCH 19/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 772bd42..ce66549 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -3,12 +3,12 @@ :icons: font <<< -. Authors +. [circle] * Jacques Lagnel <jacques.lagnel@inrae.fr> Based on : https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git - Alexandre Dehne Garcia, Rémy Dernat, Martin Souchal + (Alexandre Dehne Garcia, Rémy Dernat, Martin Souchal) All the files mentioned in this recipe are available in the following git repository : -- GitLab From a188ca6cfb33ff05e364311c420aa96d06b62e6b Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:21:53 +0100 Subject: [PATCH 20/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index ce66549..831e829 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -3,8 +3,6 @@ :icons: font <<< -. -[circle] * Jacques Lagnel <jacques.lagnel@inrae.fr> Based on : https://gitlab.in2p3.fr/alexandre.dehne-garcia/TP_singularity_EcoleConteneursProd.git -- GitLab From 8ba091fde4847eeefaa25d0752d6434b01a3f954 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:23:41 +0100 Subject: [PATCH 21/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 831e829..2850291 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -16,9 +16,9 @@ git clone https://forgemia.inra.fr/inter_cati_omics/reproductibility.git You can build an HTML version : [source, bash] -asciidoctor -d book -a toc singularity_tp_4h.adoc +asciidoctor -d book -a toc Singularity_tp.adoc -This recipe describes how to build an Ubuntu image using Singularity, with the latest Singularity 2 version (2.6). You can download Singularity and find some docs here: https://www.sylabs.io/docs/ +This recipe describes how to build an Ubuntu image using Singularity, with the Singularity 3 version (3.5.3). You can download Singularity and find some docs here: https://www.sylabs.io/docs/ We are going to install singularity on your local linux machine (debian based distribution) or in an Ubuntu/debian VM (virtualbob), then we are gonna build containers and run them on an HPC cluster genotoul. -- GitLab From 7f66ff17a9eec9612e508a644f92011fdf198bdc Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:24:46 +0100 Subject: [PATCH 22/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 2850291..d83889a 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -24,7 +24,7 @@ We are going to install singularity on your local linux machine (debian based di HELP: -* http://singularity.lbl.gov/faq +* https://sylabs.io/guides/3.5/user-guide/ * https://groups.google.com/a/lbl.gov/forum/#!forum/singularity * For the admins : https://singularity-container.slack.com/ -- GitLab From 5fd9e4e508bcf267f748119b953b04b6da57d076 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:27:51 +0100 Subject: [PATCH 23/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index d83889a..c59cece 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -26,7 +26,6 @@ HELP: * https://sylabs.io/guides/3.5/user-guide/ * https://groups.google.com/a/lbl.gov/forum/#!forum/singularity -* For the admins : https://singularity-container.slack.com/ = Introduction @@ -59,19 +58,19 @@ Singularity is a container solution created by necessity for scientific and appl * Interact with a container === Singularity installation +[source, bash] +sudo apt-get update && sudo apt-get install -y \ + build-essential \ + libssl-dev \ + uuid-dev \ + libgpgme11-dev \ + squashfs-tools \ + libseccomp-dev \ + wget \ + pkg-config \ + git \ + cryptsetup -Log in your Amazon VM and enter following commands : -[source, bash] -sudo apt-get update --fix-missing -sudo apt-get install -y squashfs-tools automake autoconf libtool python git make libarchive13 libarchive-dev -git clone https://github.com/singularityware/singularity.git -cd singularity -git fetch --all -git checkout 2.6.0 -./autogen.sh -./configure --prefix=/usr/local -make -sudo make install You can self test your install to be sure everything is ok : [source, bash] -- GitLab From 39b8e1bfb4efdbbe1b3694cd4f70c9f6bbb01964 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:29:03 +0100 Subject: [PATCH 24/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index c59cece..b1c1c36 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -57,7 +57,7 @@ Singularity is a container solution created by necessity for scientific and appl * Build images from scratch * Interact with a container -=== Singularity installation +=== Singularity installation (debian based distribution) [source, bash] sudo apt-get update && sudo apt-get install -y \ build-essential \ @@ -71,6 +71,18 @@ sudo apt-get update && sudo apt-get install -y \ git \ cryptsetup +Install Go +Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. +[source, bash] +# Replace the values as needed +export VERSION=1.13.7 OS=linux ARCH=amd64 +# Downloads the required Go package +wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz +# Extracts the archive +tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz +# Deletes the tar file +rm go$VERSION.$OS-$ARCH.tar.gz +echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && source ~/.bashrc You can self test your install to be sure everything is ok : [source, bash] -- GitLab From cf72d238ba149983cc71431f496f64cc8087d354 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:29:40 +0100 Subject: [PATCH 25/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index b1c1c36..8f46a0f 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -73,6 +73,7 @@ sudo apt-get update && sudo apt-get install -y \ Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. + [source, bash] # Replace the values as needed export VERSION=1.13.7 OS=linux ARCH=amd64 @@ -84,6 +85,7 @@ tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz rm go$VERSION.$OS-$ARCH.tar.gz echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && source ~/.bashrc + You can self test your install to be sure everything is ok : [source, bash] singularity selftest -- GitLab From ccca4d5619e91f16e16dacb8df05bc78a9f78002 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:31:11 +0100 Subject: [PATCH 26/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 8f46a0f..f2b5255 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -58,6 +58,7 @@ Singularity is a container solution created by necessity for scientific and appl * Interact with a container === Singularity installation (debian based distribution) +Prepare your system [source, bash] sudo apt-get update && sudo apt-get install -y \ build-essential \ @@ -71,6 +72,7 @@ sudo apt-get update && sudo apt-get install -y \ git \ cryptsetup + Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. -- GitLab From d443d3ce659f71b6353b812c976983e2403006d3 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:31:47 +0100 Subject: [PATCH 27/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index f2b5255..065d4d3 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -77,13 +77,13 @@ Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. [source, bash] -# Replace the values as needed +\# Replace the values as needed export VERSION=1.13.7 OS=linux ARCH=amd64 -# Downloads the required Go package +\# Downloads the required Go package wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz -# Extracts the archive +\# Extracts the archive tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz -# Deletes the tar file +\# Deletes the tar file rm go$VERSION.$OS-$ARCH.tar.gz echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && source ~/.bashrc -- GitLab From 02c615415d173c13742bf287a2d7a7fde833b81a Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:32:56 +0100 Subject: [PATCH 28/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 065d4d3..6790e11 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -77,7 +77,7 @@ Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. [source, bash] -\# Replace the values as needed +## Replace the values as needed export VERSION=1.13.7 OS=linux ARCH=amd64 \# Downloads the required Go package wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz -- GitLab From 80a7baba9cb8f876be7805efe5544e6a1c4ff7e4 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:33:15 +0100 Subject: [PATCH 29/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 6790e11..edf6b3e 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -77,7 +77,7 @@ Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. [source, bash] -## Replace the values as needed +\\# Replace the values as needed export VERSION=1.13.7 OS=linux ARCH=amd64 \# Downloads the required Go package wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz -- GitLab From 6d5067d085ff18461d630e736c17e62f256660a0 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:34:32 +0100 Subject: [PATCH 30/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index edf6b3e..b5057e9 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -77,7 +77,8 @@ Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. [source, bash] -\\# Replace the values as needed +---- +# Replace the values as needed export VERSION=1.13.7 OS=linux ARCH=amd64 \# Downloads the required Go package wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz @@ -86,7 +87,7 @@ tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz \# Deletes the tar file rm go$VERSION.$OS-$ARCH.tar.gz echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && source ~/.bashrc - +---- You can self test your install to be sure everything is ok : [source, bash] -- GitLab From 5052c23e99dcae4084a34e56e7f32e21af51f56a Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:35:06 +0100 Subject: [PATCH 31/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index b5057e9..d8e0b54 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -60,6 +60,7 @@ Singularity is a container solution created by necessity for scientific and appl === Singularity installation (debian based distribution) Prepare your system [source, bash] +---- sudo apt-get update && sudo apt-get install -y \ build-essential \ libssl-dev \ @@ -71,7 +72,7 @@ sudo apt-get update && sudo apt-get install -y \ pkg-config \ git \ cryptsetup - +---- Install Go Singularity v3 and above is written primarily in Go, so you will need Go installed to compile it from source. @@ -80,11 +81,11 @@ Singularity v3 and above is written primarily in Go, so you will need Go install ---- # Replace the values as needed export VERSION=1.13.7 OS=linux ARCH=amd64 -\# Downloads the required Go package +# Downloads the required Go package wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz -\# Extracts the archive +# Extracts the archive tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz -\# Deletes the tar file +# Deletes the tar file rm go$VERSION.$OS-$ARCH.tar.gz echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && source ~/.bashrc ---- -- GitLab From e456298babc404b3a7329b5999f7bb898e1b0a7d Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:36:16 +0100 Subject: [PATCH 32/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index d8e0b54..37172e1 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -90,6 +90,21 @@ rm go$VERSION.$OS-$ARCH.tar.gz echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && source ~/.bashrc ---- +Download Singularity from a release +[source, bash] +---- + # adjust this as necessary +export VERSION=3.5.2 +wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz +tar -xzf singularity-${VERSION}.tar.gz +cd singularity +#to avoid error with libseccomp +PKG_CONFIG_PATH=/usr/lib/x86_64-linux-gnu/pkgconfig ./mconfig +PKG_CONFIG_PATH=/usr/lib/x86_64-linux-gnu/pkgconfig make -C builddir +sudo PKG_CONFIG_PATH=/usr/lib/x86_64-linux-gnu/pkgconfig make -C builddir install +singularity help +---- + You can self test your install to be sure everything is ok : [source, bash] singularity selftest -- GitLab From 08ea08cb250e95061a6650ce5f9ddfd9107265ed Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:39:04 +0100 Subject: [PATCH 33/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 101 ++++++++++++++++++-------------- 1 file changed, 57 insertions(+), 44 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 37172e1..797b164 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -105,54 +105,67 @@ sudo PKG_CONFIG_PATH=/usr/lib/x86_64-linux-gnu/pkgconfig make -C builddir instal singularity help ---- -You can self test your install to be sure everything is ok : -[source, bash] -singularity selftest - The --help option gives an overview of Singularity options and subcommands as follows: +[source, bash] ---- singularity help - USAGE: singularity [global options...] <command> [command options...] ... - - GLOBAL OPTIONS: - -d|--debug Print debugging information - -h|--help Display usage summary - -s|--silent Only print errors - -q|--quiet Suppress all normal output - --version Show application version - -v|--verbose Increase verbosity +1 - -x|--sh-debug Print shell wrapper debugging information - - GENERAL COMMANDS: - help Show additional help for a command or container - selftest Run some self tests for singularity install - - CONTAINER USAGE COMMANDS: - exec Execute a command within the container - run Launch a runscript within the container - shell Run a Bourne shell within the container - test Launch a test script within the container - - CONTAINER MANAGEMENT COMMANDS: - apps List available apps within a container - bootstrap *Deprecated* use build instead - build Build a new Singularity container - check Perform container lint checks - inspect Display container's metadata - mount Mount a Singularity container image - pull Pull a Singularity/Docker container to $PWD - - COMMAND GROUPS: - image Container image command group - instance Persistent instance command group - - - CONTAINER USAGE OPTIONS: - see singularity help <command> - - For any additional help or support visit the Singularity - website: https://www.sylabs.io/ +---- +---- +Linux container platform optimized for High Performance Computing (HPC) and +Enterprise Performance Computing (EPC) + +Usage: + singularity [global options...] + +Description: + Singularity containers provide an application virtualization layer enabling + mobility of compute via both application and environment portability. With + Singularity one is capable of building a root file system that runs on any + other Linux system where Singularity is installed. + +Options: + -d, --debug print debugging information (highest verbosity) + -h, --help help for singularity + --nocolor print without color output (default False) + -q, --quiet suppress normal output + -s, --silent only print errors + -v, --verbose print additional information + --version version for singularity + +Available Commands: + build Build a Singularity image + cache Manage the local cache + capability Manage Linux capabilities for users and groups + config Manage various singularity configuration (root user only) + delete Deletes requested image from the library + exec Run a command within a container + help Help about any command + inspect Show metadata for an image + instance Manage containers running as services + key Manage OpenPGP keys + oci Manage OCI containers + plugin Manage Singularity plugins + pull Pull an image from a URI + push Upload image to the provided URI + remote Manage singularity remote endpoints + run Run the user-defined default command within a container + run-help Show the user-defined help for an image + search Search a Container Library for images + shell Run a shell within a container + sif siftool is a program for Singularity Image Format (SIF) file manipulation + sign Attach a cryptographic signature to an image + test Run the user-defined tests within a container + verify Verify cryptographic signatures attached to an image + version Show the version for Singularity + +Examples: + $ singularity help <command> [<subcommand>] + $ singularity help build + $ singularity help instance start + + +For additional help or support, please visit https://www.sylabs.io/docs/ ---- -- GitLab From aafdd6e457c07d5afc9ab33966d9f3ea75742926 Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:39:43 +0100 Subject: [PATCH 34/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 797b164..0656c87 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -106,10 +106,10 @@ singularity help ---- -The --help option gives an overview of Singularity options and subcommands as follows: +The help option gives an overview of Singularity options and subcommands as follows: [source, bash] ---- -singularity help +singularity --help ---- ---- Linux container platform optimized for High Performance Computing (HPC) and -- GitLab From ce14898a77d85e006acf8d2754cf952f7df0905a Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:49:54 +0100 Subject: [PATCH 35/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 0656c87..e81a882 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -183,7 +183,7 @@ singularity pull shub://GodloveD/lolcow Then run it : [source, bash] -singularity run GodloveD-lolcow-master-latest.simg +singularity run lolcow_latest.sif Or in one line : [source, bash] @@ -195,7 +195,7 @@ You can also use pull with the docker:// URI to reference Docker images served f singularity pull docker://godlovedc/lolcow # with the default name [source, bash] -singularity pull --name funny.simg docker://godlovedc/lolcow # with custom name +singularity pull --name funny.sif docker://godlovedc/lolcow # with custom name NOTE: With this method, you have one compressed read-only squashfs file system suitable for production, but you can also use writable ext3 file system suitable for interactive development ( --writable option ) or writable (ch)root directory called a sandbox for interactive development ( --sandbox option) @@ -211,6 +211,7 @@ The header is at the top of the file, and tells Singularity the base Operating S Depending on the value assigned to Bootstrap, other keywords may also be valid in the header. Create a text file with the recipe above. Call it train.def. +[source] ---- BootStrap: docker From: ubuntu:16.04 @@ -249,7 +250,7 @@ echo passed arguments are "$*" Build it (reminder: you have to be root!): [source, bash] -sudo singularity build train.simg train.def +sudo singularity build train.sif train.def Then run it : [source, bash] -- GitLab From ca568394829101b271e31759a96ec68a4e816e0e Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 12:50:20 +0100 Subject: [PATCH 36/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index e81a882..aaa9a51 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -211,7 +211,7 @@ The header is at the top of the file, and tells Singularity the base Operating S Depending on the value assigned to Bootstrap, other keywords may also be valid in the header. Create a text file with the recipe above. Call it train.def. -[source] +[source,python] ---- BootStrap: docker From: ubuntu:16.04 -- GitLab From 6e4a75a51426c07cfdd80479862c7070270bf6fe Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 13:05:39 +0100 Subject: [PATCH 37/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 49 +++++++++++++++++---------------- 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index aaa9a51..5aa2049 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -210,7 +210,7 @@ The header is at the top of the file, and tells Singularity the base Operating S Depending on the value assigned to Bootstrap, other keywords may also be valid in the header. -Create a text file with the recipe above. Call it train.def. +Create a text file with the recipe above. Call it Singularity.Steam_Locomotive. [source,python] ---- BootStrap: docker @@ -218,24 +218,25 @@ From: ubuntu:16.04 %help My first train container -Run the sl train +Run the sl Steam Locomotive %setup mkdir ${SINGULARITY_ROOTFS}/data %post apt-get -y update +#install sl :"Steam Locomotive" package apt -y install sl %help Help me. The train is stuck in this container. - %labels -Maintainer dehneg -Updater Rémy Dernat <remy.dernat@umontpellier.fr> +Maintainer author1 +Updater author2 ContainerVersion v1.5 Software sl +Default runscript: sl %environment export LC_ALL=en_US.utf8 @@ -250,37 +251,37 @@ echo passed arguments are "$*" Build it (reminder: you have to be root!): [source, bash] -sudo singularity build train.sif train.def +sudo singularity build Steam_Locomotive.sif Singularity.Steam_Locomotive Then run it : [source, bash] -singularity run train.simg arg1 bb arg3 -./train.simg a1 BB a3 +singularity run Steam_Locomotive.sif arg1 bb arg3 +./Steam_Locomotive.sif a1 BB a3 Once you have an image, you can interact with it in several ways. The shell command allows you to spawn a new shell within your container and interact with it as though it were a small virtual machine : [source, bash] -singularity shell train.simg +singularity shell Steam_Locomotive.sif which sl exit You can start the train in the container, or just run commands : [source, bash] -singularity exec train.simg /usr/games/sl -singularity exec train.simg echo toto -singularity exec train.simg cat /.singularity.d/labels.json +singularity exec Steam_Locomotive.sif /usr/games/sl +singularity exec Steam_Locomotive.sif echo toto +singularity exec Steam_Locomotive.sif cat /.singularity.d/labels.json If you're lost, you can call for help : [source, bash] -singularity help train.simg +singularity run-help Steam_Locomotive.sif You can also bind host directory inside your container : [source, bash] -singularity shell --bind /usr:/mnt/usrHost train.simg +singularity shell --bind /usr:/mnt/usrHost Steam_Locomotive.sif -With help from the official doc : https://www.sylabs.io/guides/2.6/user-guide/container_recipes.html , edit train recipe to import a soulTrain.txt file inside your container, and display it after the train (without using bing option). +With help from the official doc : https://sylabs.io/guides/3.5/user-guide/definition_files.html, edit Steam_Locomotive recipe to import a soulTrain.txt file inside your container, and display it after the train (without using bing option). Before rebuilding your container, delete your first image. Then use the command `build`. @@ -436,17 +437,17 @@ From: ubuntu:16.04 ---- [source, bash] -sudo singularity build --sandbox min.img min.def +sudo singularity build --sandbox min.sif min.def The resulting directory operates just like a container in an image file. You are permitted to make changes and write files within the directory, but those changes will not persist when you are finished using the container. To make your changes persistent, use the --writable flag when you invoke your container. Let's try to add wget command in our container : [source, bash] -sudo singularity shell --writable min.img +sudo singularity shell --writable min.sif Singularity: Invoking an interactive shell within container... -Singularity min.img:~> apt-get update -Singularity min.img:~> apt-get install wget build-essential +Singularity min.sif:~> apt-get update +Singularity min.sif:~> apt-get install wget build-essential To exit the container, just type "exit". @@ -493,7 +494,7 @@ mpicc /mpi-ping.c -o ping To compile mpi-ping.c, you just have to execute the runscript : [source, bash] -singularity run openmpi.simg +singularity run openmpi.sif In your folder, you should have a "ping" executable. In that case, you have compiled your file outside the container. You can also compile it inside, to bring the compiled file with your container. @@ -504,7 +505,7 @@ mpirun ping If you have a server with an mpi installed ypu can test it outside your container (note that you have to use the mpirun from the host, because mpi use the hardware infrastructure to dispatch program) : [source, bash] -mpirun singularity exec mpi.simg ./ping +mpirun singularity exec mpi.sif ./ping WARNING: The OpenMPI versions inside and outside the container can different but must be compatible to work together. @@ -529,7 +530,7 @@ singularity run shub://GodloveD/lolcow Let's try to submit a job on the cluster : [source, bash] singularity pull shub://GodloveD/lolcow -srun -p singularity ./GodloveD-lolcow-master.simg +srun -p singularity ./GodloveD-lolcow-master.sif You can now upload your OpenMPI container and run it on the cluster. @@ -542,7 +543,7 @@ module load openmpi You can submit in interactive mode with the following command : [source, bash] -srun -p singularity mpirun singularity exec mpi.simg ./a.out +srun -p singularity mpirun singularity exec mpi.sif ./a.out Or in batch mode, with the following submission script : @@ -557,7 +558,7 @@ _mpi.sh_ #SBATCH --partition=singularity #SBATCH --time=01:00:00 echo « Running on: $SLURM_NODELIST » -mpirun singularity exec mpi.simg ./a.out +mpirun singularity exec mpi.sif ./a.out ---- Use sbatch to run it : -- GitLab From ca902e9b598322b3e65963299556a6340da8d09a Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 13:10:26 +0100 Subject: [PATCH 38/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 5aa2049..1ed7a1c 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -197,7 +197,15 @@ singularity pull docker://godlovedc/lolcow # with the default name [source, bash] singularity pull --name funny.sif docker://godlovedc/lolcow # with custom name -NOTE: With this method, you have one compressed read-only squashfs file system suitable for production, but you can also use writable ext3 file system suitable for interactive development ( --writable option ) or writable (ch)root directory called a sandbox for interactive development ( --sandbox option) +NOTE: +---- +With this method, you have one compressed read-only squashfs file system suitable for production, but you can also use writable ext4 file system suitable for interactive development: + Build a base sandbox from DockerHub, make changes to it, then build sif + [source, bash] + singularity build --sandbox /tmp/debian docker://debian:latest + singularity exec --writable /tmp/debian apt-get install python + singularity build /tmp/debian2.sif /tmp/debian +---- A Singularity Recipe is the driver of a custom build, and the starting point for designing any custom container. It includes specifics about installation software, environment variables, files to add, and container metadata. You can even write a help section, or define modular components. -- GitLab From d87eabbf3f5986ddcfa26a6a4d949a255565fa3e Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 13:11:19 +0100 Subject: [PATCH 39/40] Update Singularity_tp.adoc --- singularity/Singularity_tp.adoc | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/singularity/Singularity_tp.adoc b/singularity/Singularity_tp.adoc index 1ed7a1c..77f90f6 100644 --- a/singularity/Singularity_tp.adoc +++ b/singularity/Singularity_tp.adoc @@ -197,11 +197,9 @@ singularity pull docker://godlovedc/lolcow # with the default name [source, bash] singularity pull --name funny.sif docker://godlovedc/lolcow # with custom name -NOTE: +NOTE: With this method, you have one compressed read-only squashfs file system suitable for production, but you can also use writable ext4 file system suitable for interactive development: ---- -With this method, you have one compressed read-only squashfs file system suitable for production, but you can also use writable ext4 file system suitable for interactive development: Build a base sandbox from DockerHub, make changes to it, then build sif - [source, bash] singularity build --sandbox /tmp/debian docker://debian:latest singularity exec --writable /tmp/debian apt-get install python singularity build /tmp/debian2.sif /tmp/debian -- GitLab From c1ff7ff2e2ec216eb9fbda9be520fabafaca7d0e Mon Sep 17 00:00:00 2001 From: Jacques Lagnel <jacques.lagnel@inra.fr> Date: Wed, 28 Oct 2020 14:23:39 +0100 Subject: [PATCH 40/40] Update Singularity_tp.adoc -- GitLab