Setting up the build system
-----------------------------

A case that is not handled very well by Makefiles is when you need to link to numerous external libraries. Let's assume, you have downloaded VTK yourself to a custom directory and have built its libraries. In order to link to the library you need to tell the compiler where the library file and header files are located. An exemplary compile command could look like this:

.. code-block:: bash
  
  g++ main.cpp -lvtk -L/software/vtk/lib -I/software/vtk/include -o test

The `-l<library>` flag specifies to search for and link to the `libvtk.so` library. The `-L<path>` option specifies the directory where `libvtk.so` is located. The `-I<path>` option specifies the `include` directory where all header files that come with the library are located. They can be included in the C++ code, e.g. as `#include <vtkXMLImageDataWriter.h>`.

In general, if the library was installed by the system, the `-L` and `-I` flags are not necessary as the locations will be found automatically. We need them only if the installation location is custom as in this case or the environment is special as on a supercomputer.

The above command is obviously not portable when the path is different on every computer. Also, it gets tedious for a high number of libraries. The solution is to not write Makefiles on your own, but to use `cmake` to do this task. `cmake` is another linux tool, as mentioned it creates Makefiles that then can be executed with `make`.

Example using Cmake
^^^^^^^^^^^^^^^^^^^^^^
In the following example we will setup `cmake` to build a program that outputs files that can be visualized using ParaView. This will serve as basis for the code for the first submission.

Creating the source
~~~~~~~~~~~~~~~~~~~~~~

At first, we take a look at the directory structure. Inside your preferred workspace directory, create two folders: 

* `src` where we'll put all source code and header files
* `build` where the executable will be created, as well as all program input and output and the byproducts of the build process.

In `src` create a subdirectory `output_writer`. In this directory, save the files `write_paraview_output.h` and `write_paraview_output.cpp`.

The header file, `write_paraview_output.h`, has the following content:

.. code-block:: c++
    
  //! Write a file out/output_<fileNo>.vti to be visualized in ParaView.
  //! It contains 10x10 nodes with an artifical pressure field.
  //! This method is only for demonstration purpose and does nothing useful.
  //! However, we will provide similar files, e.g. "output_writer_paraview.h", to be used in the submission code.
  void writeParaviewOutput(int fileNo);

The source file, `write_paraview_output.cpp` should have the following content:

.. code-block:: c++
  :linenos:
  
  #include "output_writer/write_paraview_output.h"

  #include <vtkSmartPointer.h>
  #include <vtkImageData.h>
  #include <vtkXMLImageDataWriter.h>
  #include <vtkDoubleArray.h>
  #include <vtkPointData.h>

  #include <cstdlib>
  #include <iostream>

  void writeParaviewOutput(int fileNo)
  {
    // create "out" subdirectory if it does not yet exist
    int returnValue = system("mkdir -p out");
    if (returnValue != 0)
      std::cout << "Could not create subdirectory \"out\"." << std::endl;
    
    // Create a vtkWriter
    vtkSmartPointer<vtkXMLImageDataWriter> vtkWriter = vtkSmartPointer<vtkXMLImageDataWriter>::New();
    
    // Assemble the filename
    std::stringstream fileName;
    fileName << "out/output_" << std::setw(4) << setfill('0') << fileNo << "." << vtkWriter->GetDefaultFileExtension();
    
    std::cout << "Write file \"" << fileName.str() << "\"." << std::endl;
    
    // assign the new file name to the output vtkWriter
    vtkWriter->SetFileName(fileName.str().c_str());
    
    // initialize data set that will be output to the file
    vtkSmartPointer<vtkImageData> dataSet = vtkSmartPointer<vtkImageData>::New();
    dataSet->SetOrigin(0, 0, 0);

    // set spacing of mesh
    const double dx = 1;
    const double dy = 1;
    const double dz = 1;
    dataSet->SetSpacing(dx, dy, dz);

    // set number of points in each dimension, 1 cell in z direction
    dataSet->SetDimensions(10, 10, 1);
    
    // add pressure field variable
    // ---------------------------
    vtkSmartPointer<vtkDoubleArray> arrayPressure = vtkDoubleArray::New();

    // the pressure is a scalar which means the number of components is 1
    arrayPressure->SetNumberOfComponents(1);

    // Set the number of pressure values and allocate memory for it. We already know the number, it has to be the same as there are nodes in the mesh.
    arrayPressure->SetNumberOfTuples(dataSet->GetNumberOfPoints());
    
    arrayPressure->SetName("pressure");

    // loop over the nodes of the mesh and assign an artifical value that changes with fileNo
    for (int j = 0; j < 10; j++)
    {
      for (int i = 0; i < 10; i++)
      {
        int index = j*10 + i;
        arrayPressure->SetValue(index, i+j-fileNo*(i-j));
      }
    }

    // Add the field variable to the data set
    dataSet->GetPointData()->AddArray(arrayPressure);
    
    // Remove unused memory
    dataSet->Squeeze();
    
    // Write the data
    vtkWriter->SetInputData(dataSet);
    
    //vtkWriter->SetDataModeToAscii();     // comment this in to get ascii text files: those can be checked in an editor
    vtkWriter->SetDataModeToBinary();      // set file mode to binary files: smaller file sizes

    // finally write out the data
    vtkWriter->Write();
  }

Note in line 1, how the header file is included. In our setup, the path will always be relative to the `src` directory. In lines 3-7, headers of the library VTK are included. 
This means, we have to properly tell the compiler to use the VTK library.
Skim through the rest of the code to see approximately what will happen.

Next, we need a `main.cpp` file. We put it directly in the `src` folder, having the following content:

.. code-block:: c++
  :linenos:
  
  #include "output_writer/write_paraview_output.h"

  #include <iostream>
  #include <cstdlib>

  int main(int argc, char *argv[])
  {
    // write 5 output files
    for (int i = 0; i < 5; i++)
    {
      writeParaviewOutput(i);
    }
    
    return EXIT_SUCCESS;
  }

Creating CMakeLists.txt
~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to compile and link everything, we create a file to specify for `cmake` what we want to have.
Create the file `CMakeLists.txt` in the `src` directory. Insert the lines that will be presented in the following blocks (with an empty line between the blocks, for readability):

.. code-block:: c++
  :lineno-start: 1
  
  cmake_minimum_required(VERSION 3.8)

  # Define the project name.
  project(numsim)

Here, we require that at least the given version of cmake is used. You can check your installed version of cmake by running

.. code-block:: bash

  cmake --version
  
Then, line 4 defines the name of our project. Next, we specify that we want to build an executable program, as opposed to a library.

All our source files (`*.cpp`) have to be listed. 
The macro ${PROJECT_NAME} is equal 
to `numsim` in our case and will be the name of the binary to build.

.. code-block:: cmake
  :lineno-start: 6
  
  # Specify the name of the executable (${PROJECT_NAME} which is equal to what was set in the project() command).
  # Also specify the source files.
  add_executable(${PROJECT_NAME}
     main.cpp
     output_writer/write_paraview_output.cpp
  )

Next, we specify the `src` directory as base directory for our `#include "file.h"` directives in the code:

.. code-block:: cmake
  :lineno-start: 13
  
  # Add the project directory to include directories, to be able to include all project header files from anywhere
  target_include_directories(${PROJECT_NAME} PUBLIC ${PROJECT_SOURCE_DIR})

The next section handles the inclusion of the external library `VTK`. The simple command in line 17 searches for VTK on the system and sets cmake variables some of which are printed in lines 20-22. Lines 25-28 add the required include directories and linker commands if VTK was found.

.. code-block:: cmake
  :lineno-start: 16
  
  # Search for the external package "VTK"
  find_package(VTK)

  # Output various cmake variables for demonstration purpose
  message("If VTK was found on the system: VTK_FOUND: ${VTK_FOUND}")
  message("The directory of VTK: VTK_DIR: ${VTK_DIR}")
  message("The include directory of VTK: VTK_INCLUDE_DIRS: ${VTK_INCLUDE_DIRS}")

  # If an installation of vtk was found on the system
  if (VTK_FOUND)
    include_directories(${VTK_INCLUDE_DIRS})               # add the include directory where the header files are for the compiler
    target_link_libraries(${PROJECT_NAME} ${VTK_LIBRARIES}) # add the libraries for the linker
  endif(VTK_FOUND)

By default, the executable `numsim` will be created in the directory `build/src` because the sources are also located in a `src` subdirectory. But we'd like to have it directly in the `build` directory. To copy `build/src/numsim` to `build/numsim`, we define an `install` rule with the following lines. It will be executed if we run `make install`.

.. code-block:: cmake
  :lineno-start: 30
  
  # install numsim executable in build directory
  install(TARGETS ${PROJECT_NAME} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/../build)
  
The `CMakeLists.txt` file closes with various other options:

.. code-block:: cmake
  :lineno-start: 33

  # Add additonial compile options to enable more warnings
  add_compile_options(-Wall -Wextra)

  # Set the version of the C++ standard to use, we use C++14, published in 2014
  set(CMAKE_CXX_STANDARD 14)
  set(CMAKE_CXX_STANDARD_REQUIRED ON)
  
  message("CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}")
  
Now, move up one directory to your workspace in which `src` and `build` are located. Create a second `CMakeLists.txt` file here, with only the following two lines:

.. code-block:: cmake
  :lineno-start: 1

  cmake_minimum_required(VERSION 3.8)
  add_subdirectory(src)

Line 2 references the `CMakeLists.txt` file in `src`. If you later want to add, e.g. a `documentation` directory, where a website with documentation from your code is automatically build (using `doxygen <http://www.doxygen.nl/>`_), simply add a `add_subdirectory(documentation)` statement and provide another `CMakeLists.txt` in `documentation` with the details.

Now, we're done with the configuration of cmake in the `src` directory. For comparison, install the tool `tree`

.. code-block:: bash

  sudo apt install tree
  
and run it from the workspace directory. The displayed directory tree should look like this:

.. code-block:: bash

  $ tree 
  .
  ├── build
  ├── CMakeLists.txt
  └── src
      ├── CMakeLists.txt
      ├── main.cpp
      └── output_writer
          ├── write_paraview_output.cpp
          └── write_paraview_output.h

  3 directories, 5 files


Building
~~~~~~~~~~~~~~~
As already mentioned, building consists of invoking `cmake` followed by `make`. In the `build` directory, run

.. code-block:: bash

  cmake ..

The two dots `..` specify the parent directory where the `CMakeLists.txt` file is located.

CMake will look for VTK and generate the `Makefile` on success. If there is an error finding VTK, maybe it is not yet installed? (`sudo apt install libvtk7-dev libvtk7.1`) If it still doesn't work, you can download and install VTK on your own, paste the following into a terminal and wait for completion:

.. code-block:: bash

  mkdir -p ~/software && cd ~/software            # create software directory in home
  wget https://www.vtk.org/files/release/8.2/VTK-8.2.0.tar.gz  # download source archive
  tar xf VTK-8.2.0.tar.gz && cd VTK-8.2.0         # extract files and enter directory
  mkdir build && cd build                         # create build directory
  cmake .. && make                                # build
  sudo make install                               # install (needs password)

If cmake succeeds, it should say `-- Build files have been written to: ...` at the end. Now execute `make`:

.. code-block:: bash
  
  $ make
  Scanning dependencies of target numsim
  [ 33%] Building CXX object src/CMakeFiles/numsim.dir/main.cpp.o
  [ 66%] Building CXX object src/CMakeFiles/numsim.dir/output_writer/write_paraview_output.cpp.o
  [100%] Linking CXX executable numsim
  [100%] Built target numsim
  Install the project...
  -- Install configuration: ""
  -- Installing: /store/wiss/2019/12_numsim/exercises/exercise1/resources/cmake_test/src/../build/numsim
  -- Set runtime path of "/store/wiss/2019/12_numsim/exercises/exercise1/resources/cmake_test/src/../build/numsim" to ""
  

As you can see, the two steps of building and linking were performed separately, contrary to what we had in the previous examples. If you inspect the `build` directory now with the command `tree`, you'll see that `cmake` produced a lot of temporary files. Among these we find our executable: `src/numsim`. It was also copied from the `src` subdirectory to `./numsim` as we requested. Now run it, to see the program in action:

.. code-block:: bash

  $ ./numsim 
  Write file "out/output_0000.vti".
  Write file "out/output_0001.vti".
  Write file "out/output_0002.vti".
  Write file "out/output_0003.vti".
  Write file "out/output_0004.vti".

You can always delete and recreate the `build` folder. No important information will be lost. When running `cmake ..` and `make install` again the executable will be build again. 

The `cmake` command is necessary only once as long as nothing changed in the `CMakeLists.txt` files. After you have made changes to the C++ source files, run `make install` again. Only the compilation units where something has changed will be recompiled. This speeds up the building process if you have a lot of source files and want to rebuild after changing only a small number of files. Now, try running `make install` another time. If nothing has changed, no compilation will be done and cmake should something with "Up-to-date".

CMake itself stores its various settings in cmake variables. The current values are stored in `build/CMakeCache.txt`. For example, the variable `CMAKE_BUILD_TYPE` controls if the `debug` or `release` target should be build. You can change values of cmake variables by running `cmake -D<variable>=<value> .`. To configure the release target, run (in the build directory):

.. code-block:: bash

  cmake -DCMAKE_BUILD_TYPE=Release .
  
This time, cmake can be given the current directory "." instead of "..", because the cached variables exist already. However, ".." would have the same effect.

If you run `make install` again, everything will be recompiled, this time in `release` mode. To get back to `debug` mode, execute `cmake -DCMAKE_BUILD_TYPE=Debug .` You can clean the previous build with `make clean`.

It is also interesting to see the actual compile commands of GCC that we were saved from writing. After `make clean`, execute

.. code-block:: bash

  make VERBOSE=1

Note how many shared objects (`*.so`) from vtk get linked into the program.

To summarize, we used `cmake` to generate a `Makefile` that we executed with `make`. This executed the `g++` compile commands.

Viewing the result
~~~~~~~~~~~~~~~~~~~~~
After executing the built binary `./numsim`, a directory `out` has been created with five output files. Now, visualize them with `paraview` as shown in :numref:`paraview1` - :numref:`paraview4`.


.. _paraview1:
.. figure:: images/paraview1.png
  :width: 100%
  
  Click on the yellow folder in the top left to open files.

.. _paraview2:
.. figure:: images/paraview2.png
  :width: 80% 
  :align: center
  
  Navigate to the "out" directory inside "build" and select the group of files.

.. _paraview3:
.. figure:: images/paraview3.png
  :width: 100%
  
  Change the representation "Slice" to "Surface With Edges" to get an interpolated image.

.. _paraview4:
.. figure:: images/paraview4.png
  :width: 100%

  Press `Play` or the neighbouring green buttons to navigate through all files.

Conclusion
^^^^^^^^^^^^^
Now, you have some experience using `cmake`. It is used by many open-source project (e.g. VTK). Most of them provide more options that can be set besides `CMAKE_BUILD_TYPE`. If the `cmake -D<variable>=<value>` mechanism is too inconvenient, you can try the graphical interfaces `ccmake` (works in the terminal) or `cmake-gui` (opens the graphical window in :numref:`cmake-gui2`).

.. _cmake-gui2:
.. figure:: images/cmake-gui2.png
  :width: 60%
  :align: center
  
  Cmake-gui after filling in source and build directory and pressing "Configure". Click "Generate" next to get the Makefile.

For the submission of exercise 1, keep the presented directory structure and work with `cmake` because only then the program can be automatically compiled and executed on the cluster during the submission process.