Differences

This shows you the differences between two versions of the page.

--- embedding_oot_modules_or_custom_libraries_binaries_in_minus_scenario [2016/04/12 14:49] – [Packaging executable binaries or libraries in a minus task scenario] mimbert
+++ embedding_oot_modules_or_custom_libraries_binaries_in_minus_scenario [2016/04/21 17:06] (current) – mimbert
@@ Line 16: / Line 16: @@
 There is basically no way to avoid this issue: users need to target the same gnuradio version than the one on the cortexlab nodes.
-Actually this is not only about gnuradio, but about the whole package of software used to run the radio on the nodes. We refer to this package as the **cortexlab toolchain**. It is made of:
+Actually this is not only about gnuradio, but about the whole package of software used to run the radio on the nodes. We refer to this package as the **cortexlab toolchain**. See [[Building a toolchain]]
-* [[http://gnuradio.org/|gnuradio]]
-* some gnuradio additional modules / blocks, such as blocks which interface with the radio hardware ([[http://files.ettus.com/manual/|uhd]] for usrp and similar blocks to interface with picosdr nodes)
-* some gnuradio additional modules adding convenient tools, which we decided to incude, such as [[http://git.osmocom.org/gr-iqbal|gr-iqbal]].
-* software for supporting the picosdr: [[http://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/design-tools.html|xilinx ise]], [[https://reference.digilentinc.com/digilent_adept_2#adept_2_for_linux|digilent hs2]] (the jtag used on the cortexlab nodes) and [[http://www.nutaq.com/support/latest-utca-edition|adp]] from nutaq.
 There is currently three possibilities for working with the same version of toolchain than the one on the cortexlab nodes:
 * use the cortexlab toolchain which is installed on the airlock frontend. It is the exact same toolchain as on the nodes, the only difference is that there is no hardware (usrp, picosdr, jtag) connected to the frontend.
-* build a cortexlab toolchain on a user's workstation. We provide the [[https://github.com/CorteXlab/cxlb-build-toolchain|cxlb-build-toolchain]] tool, which is the tool that we use to build the toolchain on the nodes. This tool is limited to 64 bit linux and is tested only under debian. For each toolchain built with this tool, a //bundle// file is written in the toolchain directory, which tracks all exact versions of all packages included in the toolchain. The bundle file for the current cortexlab toolchain can be found on airlock in ''/cortexlab/toolchains/current/share/cxlb-build-toolchain/build.bundle''. This file can be used as input to ''cxlb-build-toolchain'' to be sure to build the exact same toolchain. ''cxlb-build-toolchain'' also provides options to select which parts of the toolchain to build. For example, a user willing to experiment only on usrp won't need to include the picosdr specific stuff.
+* build a cortexlab toolchain on a user's workstation. We provide the [[https://github.com/CorteXlab/cxlb-build-toolchain|cxlb-build-toolchain]] tool, which is the tool that we use to build the toolchain on the nodes. See [[Building a toolchain]].
 * build the same version of gnuradio than on airlock, without using our ''cxlb-build-toolchain'' script. In this case, users are on their own. In the majority of situation this should cause no issue but there may be some slight incompatibilities with the toolchain on the nodes.
-Besides, the main advantage of the [[https://github.com/CorteXlab/cxlb-build-toolchain|cxlb-build-toolchain]] is that it takes care to install everything in a given directory, as regular user. It then generates and installs two scripts, one that has to be run by root, to configure the dynamic loader and udev to use the toolchain, the other that has to be sourced by the user which will use the toolchain, to configure the environment. Thus:
-* as it runs as a regular user, there is zero risk to damage anything in the system, you are sure that anything it does is restricted to the rights of the user running the script. The only thing that has to be run as root is a single script generated at the end, which is easy to review and audit.
-* thus,you are sure that it does not conflict with anything in the regular directory tree structure of your system. In particular, it cannot conflit with the debian package system.
-* everything from the toolchain is kept isolated in the toolchain install directory, instead of being scattered everywhere.
-* it is possible to have more than one toolchain installed in parallel. You only need to re-run the root script to switch to a different toolchain, and reboot (or possibly only restart udev? that needs to be checked), and re-source the environment configuration script.
 ===== Packaging executable binaries or libraries in a minus task scenario =====
@@ Line 47: / Line 35: @@
 The location on the cortexlab nodes where the task will be unpacked and executed is **''/home/cxlbuser/tasks/task''**
-Thus, the compilation / stage installation of a custom binary in a minus task scenario has the following scheme:
+Thus, the compilation / stage installation of a custom binary in a minus task scenario has the following scheme, assuming that the task is assembled in directory ''/cortexlab/homes/<userlogin>/tasks/mytask'' and that the temporary staging directory for compiled binaries is ''/cortexlab/homes/<userlogin>/tasks/mytask/tmp''
-= For autotools project =
+(in the following, we assume that we compile directly on ''airlock'')
-= For cmake project =
+== For autotools project ==
+  $ ./configure --prefix=/home/cxlbuser/tasks/task
+  $ make
+  $ make DESTDIR=/cortexlab/homes/<userlogin>/tasks/mytask/tmp install
+== For cmake project ==
+  $ cmake -DCMAKE_INSTALL_PREFIX=/home/cxlbuser/tasks/task
+  $ make
+  $ make DESTDIR=/cortexlab/homes/<userlogin>/tasks/mytask/tmp install
+Then, in both cases, we will find in ''/cortexlab/homes/<userlogin>/tasks/mytask/tmp'' the complete directory hierarchy ''/cortexlab/homes/<userlogin>/tasks/mytask/tmp/home/cxlbuser/tasks/task'' and directories such as ''usr'', ''share'', ''include'', etc. below. Now we simply have to move this up to the toplevel of the task:
+  $ mv /cortexlab/homes/<userlogin>/tasks/mytask/tmp/home/cxlbuser/tasks/task/* /cortexlab/homes/<userlogin>/tasks/mytask/
+  $ rm -rf /cortexlab/homes/<userlogin>/tasks/mytask/tmp
+Then we can assemble our task:
+  $ minus task create /cortexlab/homes/<userlogin>/tasks/mytask
+If we compile on a workstation instead of ''airlock'', the steps are identical except that the task directory has be be transfered to ''airlock'' before running ''minus task create'' on ''airlock''
 ==== gnuradio Out Of Tree (OOT) modules ====
+A gnuradio OOT module is just a special case of custom binary, as described in the previous section.
 The gnuradio build system is designed to allow [[https://gnuradio.org/redmine/projects/gnuradio/wiki/OutOfTreeModules|building OOT gnuradio modules]] easily. For this, users need to use ''gr_modtool'' to create their module.
@@ Line 58: / Line 69: @@
 ''gr_modtool'' will generate a cmake project which will automatically detect gnuradio, if running cmake from airlock (using the cortexlab toolchain instance on airlock), or from a user's workstation, with a gnuradio built with [[https://github.com/CorteXlab/cxlb-build-toolchain|cxlb-build-toolchain]] and a properly configured environment.
-==== Custom binaries / libraries ====
+Then, as described in the previous section, run cmake and install this way:
+  $ cmake -DCMAKE_INSTALL_PREFIX=/home/cxlbuser/tasks/task
+  $ make
+  $ make DESTDIR=/cortexlab/homes/<userlogin>/tasks/mytask/tmp install
+  $ mv /cortexlab/homes/<userlogin>/tasks/mytask/tmp/home/cxlbuser/tasks/task/* /cortexlab/homes/<userlogin>/tasks/mytask/
+  $ rm -rf /cortexlab/homes/<userlogin>/tasks/mytask/tmp
+==== multiple chained dependencies ====
+The most complex situation is when you need to embed into a minus task a binary (be it a gnuradio OOT module, an executable or a library) which depends itself from other binaries (libraries, most of the time) which need to be also embedded in the minus task. These dependency chains may be longer (eg. OOT module //A// depends on library //B//, which depends on library //C// and //D//, and library //C// depends on //D// also, as well as on library //E//).
+In this situation, you need to compile everything in reverse order starting from the end of the chain (the libraries which have no custom dependencies. In the previous example, the order would be //E//, //D//, //C//, //B//, //A//). The challenge is to be able to tell following libraries (in the order of compilation) where to find previous ones includes (for compilation) and libs (for linking)
+FIXME explain how to, when compiling //D//, telling it where to find //E//