Alpine3D  Alpine3D-3.2.0
Running a simulation

Introduction

Simulation workflow

When running a simulation, it is important to keep in mind that the model is organized as several modules that interract together. It is possible to configure some parameters for the various modules and to enable/disable modules. Some modules can be used outside of Alpine3D (like MeteoIO that is used in various applications or libSnowpack that is used by the standalone Snowpack model) .

Simulation workflow

Installing Alpine3D

Please follow the instructions given on the forge in order to download Alpine3D (from svn, from source or from a binary package) and its dependencies (Snowpack and MeteoIO, knowing that binary packages might already contain all the required dependencies). If you've downloaded a binary package, there is nothing special to do, just install it on your system.

When installing Alpine3D from sources, please keep in mind that Alpine3D's compilation process is exactly the same as Snowpack's or MeteoIO's (all are based on cmake). After a successful compilation, it is necessary to install Alpine3D (the same must have been done for Snowpack and MeteoIO). There are two options:

  • system-wide install. In this case, you need to have administrator permissions. Simply set CMAKE_INSTALL_PREFIX (in cmake) to a system path (by default, it is /usr/local) and install by typing 'make install' in a terminal opened at Alpine3D's source root folder.
  • user install: if you do not have administrator permissions, you can still install Alpine3D into a directory where you have write permissions. Simply set CMAKE_INSTALL_PREFIX (in cmake) to a directory where you can write (it is highly recommended to set it to '${HOME}/usr'), make sure this directory exists and is writable and then install by typing 'make install' in a terminal opened at Alpine3D's source root folder. Make sure that CMAKE_INSTALL_PREFIX/bin is in your PATH and that CMAKE_INSTALL_PREFIX/lib is recognized as a library path (variables PATH and LD_LIBRARY_PATH for Linux, PATH and DYLD_FALLBACK_LIBRARY_PATH for osX, see Getting Started on the forge).

After Alpine3D has been installed, you can check that it works by opening a terminal and typing "alpine3d". Alpine3D should be found and display its help message.

Simulation directories setup

After you installed a binary package or compiled and installed Alpine3D (on most compute clusters, you will need to install in your own home directory, see installing Alpine3D above), you can run your first simulation. We highly recommend that you use the following structure: first, create a directory for your simulation, for example "Stillberg". Then, create the following sub-directories:

  • input, to put all the input data required by the simulation
    • input/meteo, for the meteorological input data
    • input/surface-grids, for the dem, land cover and optional catchments grids
    • input/snowfiles, for the input .sno files
  • output, where Alpine3D will write the results -output/grids, where Alpine3D will write the gridded results
    • output/snowfiles, where Alpine3D will write its status files (sno files)
  • setup, to put the configuration of the simulation and its start scripts

Edit the "run.sh" script to set it up with the proper start and end dates, the modules that you want to enable and the proper configuration for a sequential or parallel run.

Simple sequential simulation

This is the simplest way of running an Alpine3D simulation: it runs on one core on one computer. The drawback is that if the simulation is quite large, it might require a very long time to run or even not have enough memory (RAM) to run once there is snow in the simulated domain. In order to run a sequential simulation, set "PARALLEL=N" in the run.sh script. Then run the following command in a terminal (this can be a remote terminal such as ssh) on the computer where the simulation should run:

nohup ./run.sh &

nohup means that if you close the terminal the simulation will keep going; & means that the terminal prompt can acept other commands after you've submitted this one. In order to monitor what is going on with your simulation, simply run something such as (-f means that it keeps updating with the new content in this file. Replace it with something such as -500 to show the last 500 lines of this file):

tail -f stdouterr.log

If you need to terminate the simulation, first find out its Process ID (PID) by doing

ps ux

Then kill the process

kill {PID}

Parallel simulations

When a simulated domain gets bigger, the computational requirements (memory and runtime) get bigger. In order to reduce these requirements, it is possible to run the simulation in parallel accross multiple cores or nodes.

Multi-cores simulation

This is the easiest way to run a parallel simulation because it does not require any specific software, only a compiler that supports OpenMP (see also its wikipedia page). Such compilers are for example gcc, clang, etc. The limitations on the memory still remain (ie a simulation requiring lots of memory will still only have access to the local computer's memory) but the run time will be roughtly divided by the number of available cores that are given to the simulation. In order to run such a simulation, please compile Alpine3D with the OpenMP option set to ON in cmake. Then in the simulation's run.sh file, set "PARALLEL=OPENMP" as well as the number of cores you want to us as "NCORES=". Then run the simulation as laid out in the previous section.

Multi-nodes simulation

This is the most powerful way of running a simulation: the load is distributed among distinct computing nodes, therefore reducing the amount of memory that must be available on each node. For very large simulations, this might be the only way to proceed. This is achieved by relying on MPI to exchange data between the nodes and distribute the workload. In order to run such a simulation, please compile Alpine3D with the MPI option set to ON in cmake. Then in the simulation's run.sh file, set "PARALLEL=MPI" as well as the number of processors/cores you want to us as "NPROC=" and a machine file. This machine file contains the list of machines to use for the simulation as well as how many processors/cores to use. For example, such as file could be:

192.168.0.11:2
192.168.0.5
192.168.0.4:3
192.168.2.25

Then run the simulation as laid out in the previous section.

Scaling of Alpine3D with the number of processors when using MPI (Gemstock_1m, 1 year simulation)
Note
Please make sure that the environment variables $TEMPDIR, $TEMP and $TMP (if defined) don't point to a shared drive (such as an NFS mounted home directory), otherwise MPI might run very slowly.

Sun/Oracle Grid Engine

If your computing infrastructure relies on Sun/Oracle Grid Engine (SGE) (for example on a computing cluster), you need to do things differently. First, the job management directives/options must be provided, either on the command line or in the run.sh script. These lines rely on special comments, starting with "#" followed by "$":

#$ -m aes
#$ -N {simulation name}
#$ -S /bin/bash
#$ -cwd
#$ -j y
#$ -pe smp {number of cores to use}

The last line specifies the computing profile that should be used. Since the job manager will allocate the ressources, there is no need to provide either NCORES or NPROC. The machine file (for MPI) is also not used. Then, submit the computing job to the system: "qsub ./run.sh". This should return almost immediately with a message providing the allocated job number. This job number is useful to delete the job "qdel {job_number}" or querry its status "qstat {job_number}" (or "qstat" to see for all jobs).

If the job submission fails with an error message such as unknown command, please check that there is no extra "#$" in the script. This happens frequently when commenting out some part of the script and is mis-interpreted by SGE. In such a case, simply add an extra "#" in front of the comment.

Note
At WSL, the computing profiles are either smp for shared memory runs such as OpenMP or orte for load distributed among nodes such as MPI

Data strategies

Depending on the meteorological data availability, the meteorological data quality as well as convenience (if the raw data is difficult to access, for example), there are different data strategies that can be used. This is illustrated in the figure below.

Meteorological data strategies

The simplest case is 1: the meteorological forcings are directly read and processed by Alpine3D. Since it embbeds MeteoIO, it can perform the whole preprocessing on the fly. In order to have a look at the pre-processed data (or to spare a lengthy data extraction), it is possible to preprocess the data first, then dump them to files and run Alpine3D from these intermediate file (case 2). Since Alpine3D anyway embbeds MeteoIO, it would be possible to add some pre-processing to be done on the fly within Alpine3D (such as resampling to the sampling rates that Alpine3D needs). This is actually highly recommended in order to guarantee that Alpine3D gets the data it needs (ie proper sampling rates as well as basic checks on the ranges, etc).

In some cases, it might be interesting to first run the data through Snowpack in order to produce some meteorological parameter from other parameters, such as the ISWR from RSWR (since Snowpack computes the snow albedo, the generate data would be much better than assuming a fixed albedo) or to generate precipitation from the measured snow height. This is shown in case 3.

Case 4 shows a strategy that could be used to prepare an Alpine3D simulation: it consists of simple Snowpack simulations performed at spatially interpolated locations, using MeteoIO's virtual stations concept. This is useful to (relatively) quickly validate the spatially interpolated fields with some measured data (specially if there are some meteorological data that could be compared to the spatially interpolated meteorological fields).

Coupling with Snowpack

Alpine3D needs spatially interpolated forcings for each grid points. Unfortunately, it limits the choice of forcing parameters: for some parameters (such as HS or RSWR), there are no reliable interpolation methods. One way to make use of the existing measurements that could not be easily interpolated is to run a Snowpack simulation at the stations that provided these measurements, then use alternate, computed parameters (such as PSUM or ISWR) as inputs to Alpine3D.

This process is made easier by writing Snowpack's outputs in the smet format and making sure all the necessary parameters are written out. This means that Snowpack should be configured along the following lines (only using one slope):

[Output]
TS_WRITE = TRUE
TS_FORMAT = SMET
TS_DAYS_BETWEEN = 0.04166667 ;so we get hourly values
OUT_CANOPY = FALSE
OUT_HAZ = FALSE
OUT_SOILEB = FALSE
OUT_HEAT = FALSE
OUT_T = FALSE
OUT_STAB = FALSE
OUT_LW = TRUE
OUT_SW = TRUE
OUT_MASS = TRUE
OUT_METEO = TRUE
AVGSUM_TIME_SERIES = TRUE
CUMSUM_MASS = FALSE
PRECIP_RATES = FALSE

Then the output smet files produced by Snowpack can be directly used as Alpine3D inputs, performing some on-the-fly renaming and conversions (here, from split precipitation to precipitation/phase):

[Input]
METEO = SMET
METEOPATH = ../input/meteo
STATION1 = WFJ2
PSUM_S::MOVE = MS_Snow
PSUM_L::MOVE = MS_Rain
HS::MOVE = HS_meas ;so we can still compare measured vs modelled snow height
TSG::MOVE = T_bottom ;so we can compare the ground temperatures
TSS::MOVE = TSS_meas ;so we can compare the surface temperatures
WFJ2::KEEP = TA TSS TSG RH ISWR ILWR HS VW DW PSUM_S PSUM_L PSUM PSUM_PH ;so we do not keep all kind of unnecessary parameters
PSUM_PH::create = PRECSPLITTING
PSUM_PH::PRECSPLITTING::type = THRESH
PSUM_PH::PRECSPLITTING::snow = 274.35
PSUM::create = PRECSPLITTING
PSUM::PRECSPLITTING::type = THRESH
PSUM::PRECSPLITTING::snow = 274.35
[SNOWPACK]
ENFORCE_MEASURED_SNOW_HEIGHTS = FALSE

Of course, other stations can also be part of the meteo input and their inputs should remain unaffected (assuming they don't use parameter names such as MS_Snow, MS_Rain or HS_meas and assuming that their parameters are not rejected by the KEEP command). As a side note, the last three parameter renamings (with MOVE) must be set when using Snowpack's outputs as inputs for a new Snowpack simulation.