StreamFlow

Running-StreamFlow

You are looking at an old revision of the page Running-StreamFlow. This revision was created by Aurelien Gallice.

Table of Content

This page explains how to run a StreamFlow simulation

Introduction

StreamFlow is an extension to the spatially-distributed snow model Alpine3D which allows the user to perform hydrological simulations. Both tools run independently, but Alpine3D needs to be configured in such a way that it outputs a set of files which are expected by StreamFlow as input data (see below). In case you do not know how to run an Alpine3D simulation, please have a look at the Alpine3D documentation.

Once the Alpine3D simulation is run, you can either copy the output data to a new folder or directly perform the StreamFlow simulation in the same folder as Alpine3D (StreamFlow was designed so that no conflict occurs). Similarly to the other tools of the Alpine3D suite, the behavior of StreamFlow is controlled by a configuration file. You can actually use the same one as for Alpine3D, simply adding the keys which are required by StreamFlow (see below).

Additionally to the output data generated by Alpine3D, StreamFlow requires specific files in order to run properly. The latter include the grid delineating the subwatersheds as well as two data files specifying the location and structure of the stream network. All these files can be generated by TauDEM, a tool that was designed to extract various hydrological variables from digital elevation models (see below).

Before being actually run, StreamFlow may need to be calibrated in case it is applied over a new catchment. As for any other hydrological model, this step may be quite computationally intensive since at least a thousand runs are typically required to obtain good parameter estimates. A few guidelines helping you efficiently calibrating the model can be found below.

Once calibrated, the model can eventually be run. The configuration file allows you to control many aspects of the simulation, such as the algorithm which is used to advect water along the stream network, whether stream temperature should be computed or not, or even the rate at which data should be written in the output files.

In summary, the usual procedure for running a StreamFlow simulation is the following:

  1. Configure and run Alpine3D so that it produces the data required by StreamFlow
  2. Create the simulation folder for the StreamFlow simulation, or simply re-use the Alpine3D one
  3. Create the configuration file for the StreamFlow simulation, or simply modify the Alpine3D one by adding the missing keys
  4. Use TauDEM to generate the additional files required by StreamFlow, and copy these files in the simulation folder of StreamFlow
  5. Calibrate StreamFlow
  6. Run the StreamFlow simulation by calling the executable on the command line with the proper options

Each one of these six steps is detailed in a dedicated section below.

Configuration of Alpine3D

Minimum requirements

At the very minimum, StreamFlow requires from Alpine3D the amount of water infiltrating into each subwatershed at every time step. Two possibilities are available for this:

  • In case you already have the grid delineating the subwatersheds (see section about TauDEM below), you can configure Alpine3D so that it directly computes the total amount of water infiltrating into each subwatershed at each time step. The infiltration values will then be written out in SMET files, one for each subwatershed. To this end, you need to copy the grid defining the subwatersheds in folder input\surface-grids of the Alpine3D simulation directory. Please make sure that this grid is in a format which is readable by Alpine3D (TauDEM generates a TIFF grid which needs to be converted; the list of Alpine3D compatible formats can be found in the MeteoIO documentation). You then need to add the following keys in section [INPUT] of the Alpine3D configuration file, with [format] to be replaced with the grid format (e.g. ARC), [folder] with the (absolute or relative) path to the folder in which the grid is located (e.g. ./input/surface-grids), and [file_name] with the grid name (without the path):

        GRID2D              = [format]
        GRID2DPATH          = [folder]
        CATCHMENT           = [file_name]
        CATCHMENT_NUMBERING = TAUDEM
        

    Additionally, the following line has to be added in section [OUTPUT] of the Alpine3D configuration file, where [folder] stands for the (absolute or relative) path to the folder in which the SMET files containing the subwatershed infiltration rates will be generated by Alpine3D:

        CATCHMENTS_PATH = [folder]
        

  • If you do not have the grid delineating the subwatersheds yet, you need to configure Alpine3D so that it outputs the grids containing the water percolation rate at the bottom of each soil column. The computation of the total water inflow into each subwatershed will then be performed by StreamFlow itself based on these grids. The writing of the percolation rate grids is triggered by adding the following keys in section [OUTPUT] of the Alpine3D configuration file, with [format] and [folder] to be replaced with the format and the (relative or absolute) path to the folder in which the grids should be written, respectively:

        WRITE_RUNOFF_GRIDS = TRUE
        RUNOFF_GRID2D      = [format]
        RUNOFF_GRID2DPATH  = [folder]
        

    It is recommended that you use the binary NETCDF format for the grids, since it is associated with reduced reading and writing overheads (see section about the NETCDF plugin in the MeteoIO documentation).

Additional requirements

The above configuration of Alpine3D is sufficient in case you intend to compute only discharge and water depth in the stream network. The computation of stream temperature requires additional output from Alpine3D:

  1. Depending on the configuration of StreamFlow, the computation of the temperature of water exfiltrating out of the subwatersheds into the stream network may require soil or air temperature averaged over each subwatershed. It should be mentioned that the value of soil temperature which is expected by StreamFlow corresponds to the average temperature between the soil surface and a given depth (see below). Two cases might occur:
    • In case you have the grid delineating the subwatersheds (see previous section), you can simply proceed as explained in the first bullet point of the above section and add the following lines in section [INPUT] of the Alpine3D configuration file:

          RUNOFF_FILES_EXTRA_DATA = TA TSOIL
          SOIL_TEMPERATURE_DEPTH  = [depth]
          

      [depth] should be replaced with the depth (in meters) down to which soil temperature should be averaged. The first line tells Alpine3D to add the average air and soil temperature to the SMET files which are created for each subwatershed and contain the water infiltration rates.
    • If you have not generated the grid delineating the subwatersheds yet, you should simply follow the steps indicated in point 2 below. In this case, StreamFlow will itself average soil temperature over each subwatershed based on the corresponding grids generated by Alpine3D. It will also average air temperature based on the same meteorological input data as used by Alpine3D.
  2. Additionally, StreamFlow expects soil temperature grids in order to compute the diffusive thermal exchange between the water and the stream bed in each stream cell. Similarly to point 1 above, soil temperature should be understood here as a vertical profile average between the soil surface and a given depth. In order for Alpine3D to output soil temperature grids, the following keys should be added in the Alpine3D configuration file:

        [INPUT]
        SOIL_TEMPERATURE_DEPTH = [depth]
    
        [OUTPUT]
        GRID2D                 = [format]
        GRID2DPATH             = [folder]
        GRIDS_WRITE            = TRUE
        GRIDS_START            = 0
        GRIDS_DAYS_BETWEEN     = 0.125
        

    In the above lines, [depth] should be replaced with the depth (in meters) down to which the soil temperature profile must be averaged. Similarly, [format] and [folder] should be replaced by the format and folder in which the soil temperature grids should be written out. It is highly recommended to use the NETCDF format in order to reduce the size and generation time of the output grids. Key GRIDS_DAYS_BETWEEN controls the time interval (in days) at which the output grids are generated. It is usually sufficient to have only one grid produced every 3 hours (= 0.125 days) since soil temperature does not evolve very rapidly in time.

    Adding the above keys in the Alpine3D configuration file will actually not only trigger the writing of grids of soil temperature, but also grids of soil surface temperature, snow height, snow water equivalent, air temperature as well as incoming short wave and long wave radiations. The production of all these grids may be extremely time consuming. In case only the soil temperature grids are of interest to you, you can deactivate the generation of the other grids by commenting the corresponding lines in function SnowpackInterface::writeOutput() in file trunk/alpine3d/SnowpackInterface.cc.

Simulation folder

Although not strictly required, it is recommended that you create a new folder for each StreamFlow simulation with the same structure as required for Alpine3D. Alternatively, you can simply use the same folder as the one in which the Alpine3D simulation was run. In both cases, it is advised that the main folder has a structure similar to the following:

    |
    |--- bin/
    |
    |--- input/
    |     |
    |     |--- measurements/
    |     |--- meteo/
    |     |--- grids/
    |     |--- runoff/
    |     |--- stream-network/
    |
    |--- output/
    |     |
    |     |--- calibration/
    |     |--- runoff/
    |
    |--- setup/

  • The bin/ folder is intended to contain a copy of the StreamFlow executable. Please mind that the executable depends on the MeteoIO and StreamFlow libraries, which have to be on your system path (see Installing-StreamFlow for details on how to add them to your system path).
  • The input/ folder is expected to regroup all the input data.
    • The measurements/ subfolder should contain observations, typically discharge and temperature time series, to be used when calibrating the model. These observations can be in any of the formats supported by MeteoIO for files containing meteorological time series (see list of MeteoIO plugins).
    • The meteo/ subfolder should regoup the meteorological observations to be interpolated by StreamFlow over the stream cells. This folder should actually contain the exact same files as in the corresponding Alpine3D simulation folder. Note that it is only required in case you intend to compute stream temperature (see Additional requirements).
    • In subfolder grids/, you should place the digital elevation model which was used for the Alpine3D simulation as well as the grid delineating the subwatersheds which was produced by TauDEM (see Generation of the stream network with TauDEM). This subfolder can optionally contain the grids of water exfiltration rate at the bottom of each soil column as well as the soil temperature grids generated by Alpine3D (see Minimum requirements and Additional requirements), if required.
    • Subfolder runoff/ is expected to group the SMET files generated by Alpine3D which contain the time series of the water infiltration rate into each subwatershed. In case you use the same root folder as for the Alpine3D simulation, you can actually let these files in the subfolder were Alpine3D has produced them (e.g. output/runoff), but be sure to specify the correct path to them under key CATCHMENTS_PATH in section [INPUT] of the StreamFlow configuration file (see Configuration file). As explained in Minimum requirements, this subfolder is only required in case the SMET files are available.
    • The stream-network/ subfolder is intended to group the two files defining the stream network structure which are generated by TauDEM (see Generation of the stream network with TauDEM).
  • The output/ folder will contain the output data generated by StreamFlow
    • The calibration/ subfolder is intended to receive the file containing the best model parameters when running StreamFlow in calibration mode.
    • The runoff/ subfolder will contain the time series of discharge and water depth (and optionally temperature) for each reach of the stream network.
  • In folder setup/ you can place the configuration file for the StreamFlow simulation.

Configuration file

The configuration file of StreamFlow follows the same structure and syntax as the one of Snowpack or Alpine3D. In particular, each section is separated from the preceding one by a header contained within square brackets, e.g. [INPUT]. Each controllable program option or parameter is defined by a unique keyword. In order to set the value of a given option or parameter, the user should write its associated keyword in the proper section of the configuration file, followed by an equal sign (=) and the desired option value. For example, the simulation time step can be set to 1 minute by writing TIME_STEP = 1 in section [HYDROMODEL] of the StreamFlow configuration file. Most options have default values and are therefore not required to be present in the configuration file. Comments can be inserted anywhere in the file, they should start either with ; or # and extend until the end of the line.

As a general rule, it is recommended that you complete the configuration file of the Alpine3D simulation with the keys required by StreamFlow, since many parts are common to both programs. On top of the sections associated with MeteoIO ([GENERAL], [INPUT], [OUTPUT], [FILTERS], [INTERPOLATIONS1D] and [INTERPOLATIONS2D]), three new sections are required for a StreamFlow simulation. Section [INPUTFROMA3D] specifies the location and format of the data generated by Alpine3D which are expected by StreamFlow as input. Section [HYDROMODEL] defines which algorithms should be used in the model and specifies the values of the algorithm parameters. Section [CALIBRATION] defines both the optimization technique and the objective function to be used when calibrating the model. Below is presented the minimum additional set of keys required to run a StreamFlow simulation on top of the keys used for a typical Alpine3D simulation. A complete description of all available keys can be found in the documentation of StreamFlow.

  • The simulation time step is controlled by key TIME_STEP in section [HYDROMODEL] of the configuration file, where it should be indicated in minutes. Note that some modules of StreamFlow require the time step not to exceed a certain threshold, they will automatically adapt the time step to their needs. As such, the value that you indicate under key TIME_STEP should rather be understood as a maximum allowed value rather than a fixed one.
  • StreamFlow has been designed so as to be as modular as possible regarding the following hydrological aspects:
    1. The so-called transfer function, used to compute the lateral inflow discharge into the stream network from the surrounding hillslopes. The choice of the transfer function is controlled in section [HYDROMODEL] of the configuration file by key SUBWATERSHED_CLASS, which is optional since only one transfer function (TWO_VERTICAL_LINEAR_RESERVOIRS) has been implemented in StreamFlow to date. This function corresponds to the vertical superposition of two linear reservoirs representing the fast and slow responses of the subwatershed (see documentation of StreamFlow for further details). Other options may become available in the future, though.
    2. The algorithm used to compute the temperature of water exfiltrating out of the hillslopes into the stream network. StreamFlow currently offers four different options for this algorithm, which are all described into detail in the documentation. Depending on his choice, the user should set the value of configuration key SUBWATERSHED_TEMPERATURE_MODEL in section [HYDROMODEL] to one of the following options:
      • NONE in case the temperature of the exfiltration water should not be computed. This is the default option in case key SUBWATERSHED_TEMPERATURE_MODEL is not present in the configuration file.
      • HSPF if the temperature of exfiltrating water should be computed according to the algorithm implemented in the hydrological model HSPF (Hydrological Simulation Program Fortran). This option requires the specification of additional keys in the configuration file.
      • ENERGY_BALANCE in order to rely on the algorithm described in Comola et al., 2015, "Thermodynamics in the hydrologic response: Travel time formulation and application to Alpine catchments," Water Resources Research, 51(3): 1671-1687. This option requires the specification of additional keys in the configuration file.
      • SOIL_TEMPERATURE so as to set the temperature of exfiltrating water equal to the soil temperature right next to the stream.
    3. The algorithm used to advect water within the stream network. The selection between the three possible algorithms is performed by setting the value of key STREAM_CLASS to one of the following (see documentation for more details on each option):
      • INSTANTANEOUS_ROUTING, if water should be instantaneously routed through the stream network at each time step. This option treats each stream reach as a lumped entity, i.e. it computes a single value of discharge and water depth per stream reach. It requires the specification of the model used to compute water depth in the configuration file (see documentation). This is the default option in case key STREAM_CLASS is not specified in the configuration file.
      • DIRECT_ROUTING, which corresponds to the same algorithm as INSTANTANEOUS_ROUTING but for discretized stream reaches. It splits each reach into cells, for each of which a value of discharge and water depth is computed.
      • MUSKINGUM_CUNGE in order to rely on the Muskingum-Cunge algorithm. This option requires the specification of key MANNING_COEFF in the configuration file.
    4. The algorithm used to compute water temperature within the stream network. The user can select between the two available options for this algorithm by setting the value of key STREAM_TEMPERATURE_MODEL to one of the following:
      • NONE in case stream temperature should not be computed. This is the default option.
      • ENERGY_BALANCE in order to compute stream temperature according to the energy-balance equation. This option requires the specification of additional keys in the configuration file and can only be chosen if SUBWATERSHED_TEMPERATURE_MODEL is not set to NONE. Please refer to the documentation of StreamFlow for more details on this method.
  • In case stream temperature is to be computed, StreamFlow requires the values of given meteorological variables (air temperature, wind velocity, relative humidity, and incoming short and long wave radiations) over each stream cell. These values are spatially interpolated by StreamFlow based on the same station measurements or meteo grids than in Alpine3D (see Alpine3D documentation). As such, the same keys are required by StreamFlow that are also expected by Alpine3D to specify the location and format of the meteorological input data. For example, the interpolation of meteorological station measurements stored under the SMET format and located in folder ./input/meteo can be specified using the following keys:

        [INPUT]
        METEO     = SMET
        METEOPATH = ./input/meteo
        STATION1  = WFJ
        STATION2  = DAV
        STATION3  = SBG
        

    In case of meteorological data interpolation, it is often necessary to specify additional keys in sections [FILTERS], [INTERPOLATIONS1D] and [INTERPOLATIONS2D] of the configuration file as described in the MeteoIO documentation. These keys are used to remove outliers, fill measurement gaps and specify the algorithm to be used in order to spatially interpolate the data.
  • As mentioned in the section about Alpine3D above, StreamFlow requires as input data the water infiltration rate over each subwatershed at each time step. In case you use the same configuration file and simulation folder as for the Alpine3D simulation, you do not need to add any particular key: StreamFlow will automatically find the required files. Otherwise, two cases are possible:
    • In case you configured Alpine3D so that it computed the infiltration rates over the subwatersheds and wrote them in SMET files, you need to indicate the folder in which those SMET files are located in section [INPUTFROMA3D]:

          [INPUTFROMA3D]
          CATCHMENTS_PATH = ./input/runoff   # folder location may be different
          

    • In case Alpine3D produced grids of water percolation rate at the bottom of each soil column, you have to indicate the format and folder (or file) in which those grids are in section [INPUTFROMA3D]:

          [INPUTFROMA3D]
          GRID2D     = ARC             # format may be different
          GRID2DPATH = ./input/grids   # folder location may be different
          

  • StreamFlow requires the specification of additional keys in section [INPUT] as compared to Alpine3D:
    • CATCHMENTS_PATH
    • TAUDEM_TREE_FILE

In summary, a StreamFlow configuration file should at least have the following keys, on top of those contained in the Alpine3D configuration file:

    [HydroModel]
    TIME_STEP = 60
    SUBWATERSHED_CLASS             = TWO_VERTICAL_LINEAR_RESERVOIRS
    SUBWATERSHED_TEMPERATURE_MODEL = HSPF
    STREAM_CLASS                   = INSTANTANEOUS_ROUTING
    STREAM_TEMPERATURE_MODEL       = ENERGY_BALANCE

    [Input]
    TAUDEM_TREE_FILE         = ./input/stream-network/dischmatree.dat
    TAUDEM_COORDS_FILE       = ./input/stream-network/dischmacoord.dat
    STREAM_MEASUREMENTS      = SMET
    STREAM_MEASUREMENTS_PATH = ./input/measurements
    MEASUREMENT_POINT1       = outlet_point.smet

    [InputFromA3D]
    CATCHMENTS_PATH = ./input/runoff

    [Output]
    WATERSHED::WRITE      = TRUE
    STREAM::WRITE         = TRUE
    STREAM::PATH          = ./output/runoff
    WATERSHED::WRITE_STEP = 0.041667    # 1 hour
    STREAM::WRITE_STEP    = 0.041667    # 1 hour
    CALIBRATION           = ./output/calibration/best_params.smet

    [Calibration]
    NUM_ITERATIONS         = 100
    NUM_PARAM_SETS_TO_KEEP = 10
    OBJECTIVE_FUNCTION1           = NSE
    OBJECTIVE_FUNCTION1::variable = temperature
    OBJECTIVE_FUNCTION1::weight   = -1
    OBJECTIVE_FUNCTION1::points   = ALL

Generation of the stream network with TauDEM

Calibration

Calling the executable