This page explains how to run a StreamFlow simulation
IMPORTANT : READ FIRST
METEOIO rev 2072 breaks the grid reading in Streamflow, so for now do not use meteoio newer than revision 2071
StreamFlow is under heavy development, the information present here might be wrong (especially the keys needed in the configuration file for StreamFlow). Pleas refers to the provided documented examples in trunk/doc/examples/. Please contact us for any question.
StreamFlow is an extension to the spatially-distributed snow model Alpine3D which allows the user to perform hydrological simulations. Both tools run independently, but Alpine3D needs to be configured in such a way that it outputs a set of files which are expected by StreamFlow as input data (see below). In case you do not know how to run an Alpine3D simulation, please have a look at the Alpine3D documentation.
Once the Alpine3D simulation is run, you can either copy the output data to a new folder or directly perform the StreamFlow simulation in the same folder as Alpine3D (StreamFlow was designed so that no conflict occurs). Similarly to the other tools of the Alpine3D suite, the behavior of StreamFlow is controlled by a configuration file. You can actually use the same one as for Alpine3D, simply adding the keys which are required by StreamFlow (see below).
Additionally to the output data generated by Alpine3D, StreamFlow requires specific files in order to run properly. The latter include the grid delineating the subwatersheds as well as two data files specifying the location and structure of the stream network. All these files can be generated by TauDEM, a tool that was designed to extract various hydrological variables from digital elevation models (see below).
Before being actually run, StreamFlow may need to be calibrated in case it is applied over a new catchment. As for any other hydrological model, this step may be quite computationally intensive since at least a thousand runs are typically required to obtain good parameter estimates. A few guidelines helping you efficiently calibrating the model can be found below.
Once calibrated, the model can eventually be run. The configuration file allows you to control many aspects of the simulation, such as the algorithm which is used to advect water along the stream network, whether stream temperature should be computed or not, or even the rate at which data should be written in the output files.
In summary, the usual procedure for running a StreamFlow simulation is the following:
- Configure and run Alpine3D so that it produces the data required by StreamFlow
- Create the simulation folder for the StreamFlow simulation, or simply re-use the Alpine3D one
- Create the configuration file for the StreamFlow simulation, or simply modify the Alpine3D one by adding the missing keys
- Use TauDEM to generate the additional files required by StreamFlow, and copy these files in the simulation folder of StreamFlow
- Calibrate StreamFlow
- Run the StreamFlow simulation by calling the executable on the command line with the proper options
Each one of these six steps is detailed in a dedicated section below.
Configuration of Alpine3D
At the very minimum, StreamFlow requires from Alpine3D the amount of water infiltrating into each subwatershed at every time step. Two possibilities are available for this:
In case you already have the grid delineating the subwatersheds (see section about TauDEM below), you can configure Alpine3D so that it directly computes the total amount of water infiltrating into each subwatershed at each time step. The infiltration values will then be written out in SMET files, one for each subwatershed. To this end, you need to copy the grid defining the subwatersheds in folder
input\surface-gridsof the Alpine3D simulation directory. Please make sure that this grid is in a format which is readable by Alpine3D (TauDEM generates a TIFF grid which needs to be converted; the list of Alpine3D compatible formats can be found in the MeteoIO documentation). You then need to add the following keys in section
[INPUT]of the Alpine3D configuration file, with
[format]to be replaced with the grid format (e.g.
[folder]with the (absolute or relative) path to the folder in which the grid is located (e.g.
[file_name]with the grid name (without the path):
Additionally, the following line has to be added in section
GRID2D = [format] GRID2DPATH = [folder] CATCHMENT = [file_name] CATCHMENT_NUMBERING = TAUDEM
[OUTPUT]of the Alpine3D configuration file, where
[folder]stands for the (absolute or relative) path to the folder in which the SMET files containing the subwatershed infiltration rates will be generated by Alpine3D:
CATCHMENTS_PATH = [folder]
If you do not have the grid delineating the subwatersheds yet, you need to configure Alpine3D so that it outputs the grids containing the water percolation rate at the bottom of each soil column. The computation of the total water inflow into each subwatershed will then be performed by StreamFlow itself based on these grids. The writing of the percolation rate grids is triggered by adding the following keys in section
[OUTPUT]of the Alpine3D configuration file, with
[folder]to be replaced with the format and the (relative or absolute) path to the folder in which the grids should be written, respectively:
It is recommended that you use the binary
WRITE_RUNOFF_GRIDS = TRUE RUNOFF_GRID2D = [format] RUNOFF_GRID2DPATH = [folder]
NETCDFformat for the grids, since it is associated with reduced reading and writing overheads (see section about the NETCDF plugin in the MeteoIO documentation).
The above configuration of Alpine3D is sufficient in case you intend to compute only discharge and water depth in the stream network. The computation of stream temperature requires additional output from Alpine3D:
Depending on the configuration of StreamFlow, the computation of the temperature of water exfiltrating out of the subwatersheds into the stream network may require soil or air temperature averaged over each subwatershed. It should be mentioned that the value of soil temperature which is expected by StreamFlow corresponds to the average temperature between the soil surface and a given depth (see below). Two cases might occur:
In case you have the grid delineating the subwatersheds (see previous section), you can simply proceed as explained in the first bullet point of the above section and add the following lines in section
[INPUT]of the Alpine3D configuration file:
RUNOFF_FILES_EXTRA_DATA = TA TSOIL SOIL_TEMPERATURE_DEPTH = [depth]
[depth]should be replaced with the depth (in meters) down to which soil temperature should be averaged. The first line tells Alpine3D to add the average air and soil temperature to the SMET files which are created for each subwatershed and contain the water infiltration rates.
- If you have not generated the grid delineating the subwatersheds yet, you should simply follow the steps indicated in point 2 below. In this case, StreamFlow will itself average soil temperature over each subwatershed based on the corresponding grids generated by Alpine3D. It will also average air temperature based on the same meteorological input data as used by Alpine3D.
- In case you have the grid delineating the subwatersheds (see previous section), you can simply proceed as explained in the first bullet point of the above section and add the following lines in section
Additionally, StreamFlow expects the values of various meteorological variables as well as the values of depth-averaged soil temperature over each grid cell belonging to the stream network. Again, two possibilities are offered to obtain those values from Alpine3D:
The simplest but less elegant solution consists in defining each grid cell containing a portion of the stream network as a point of interest before running the Alpine3D simulation. A small python tool,
add_stream_poi.py(located in subolder
tools/of the StreamFlow main installation directory) was specifically designed to perform this task. Defining all the stream cells as points of interest will result in Alpine3D creating an output SMET file for each cell of the stream network. In order to force Alpine3D to add depth-averaged soil temperature to the list of variables written out for each cell in its corresponding SMET file, the following lines should be added in section
[OUTPUT]of the Alpine3D configuration file:
In the above lines,
EXPERIMENT = [experiment] SOIL_TEMPERATURE_DEPTH = [depth]
[experiment]can be replaced with any arbitrary name describing the simulation, such as e.g.
dischmain case the model is run in the Dischma catchement. In addition,
[depth]should be replaced with the depth down to which soil temperature should be averaged (typically 2 meters).
The other solution consists in obtaining the values of depth-averaged soil temperature separately from those of the meteorological variables. The latter can be obtained by interpolating the same meteorological station measurements as in Alpine3D over each cell of the stream network. No particular change in the configuration of Alpine3D is required for this. Regarding the values of depth-averaged soil temperature, they can be obtained as grids from Alpine3D. To this end, the following keys should be added in the Alpine3D configuration file:
In the above lines,
[INPUT] SOIL_TEMPERATURE_DEPTH = [depth] [OUTPUT] GRID2D = [format] GRID2DPATH = [folder] GRIDS_WRITE = TRUE GRIDS_START = 0 GRIDS_DAYS_BETWEEN = 0.125 GRIDS_PARAMETERS = TSOIL
[depth]should be replaced with the depth (in meters) down to which the soil temperature profile must be averaged. Similarly,
[folder]should be replaced by the format and folder in which the soil temperature grids should be written out. Key
GRIDS_DAYS_BETWEENcontrols the time interval (in days) at which the output grids are generated. It is usually sufficient to have only one grid produced every 3 hours (= 0.125 days) since soil temperature does not evolve very rapidly in time.
- The simplest but less elegant solution consists in defining each grid cell containing a portion of the stream network as a point of interest before running the Alpine3D simulation. A small python tool,
Although not strictly required, it is recommended that you create a new folder for each StreamFlow simulation with the same structure as required for Alpine3D. Alternatively, you can simply use the same folder as the one in which the Alpine3D simulation was run. In both cases, it is advised that the main folder has a structure similar to the following:
| |--- bin/ | |--- input/ | | | |--- measurements/ | |--- meteo/ | |--- grids/ | |--- runoff/ | |--- stream-network/ | |--- output/ | | | |--- calibration/ | |--- runoff/ | |--- setup/
bin/folder is intended to contain a copy of the StreamFlow executable. Please mind that the executable depends on the MeteoIO and StreamFlow libraries, which have to be on your system path (see Installing-StreamFlow for details on how to add them to your system path).
input/folder is expected to regroup all the input data.
measurements/subfolder should contain observations, typically discharge and temperature time series, to be used when calibrating the model. These observations can be in any of the formats supported by MeteoIO for files containing meteorological time series (see list of MeteoIO plugins).
meteo/subfolder should regoup the meteorological observations to be interpolated by StreamFlow over the stream cells. This folder should actually contain the exact same files as in the corresponding Alpine3D simulation folder. Note that it is only required in case you intend to compute stream temperature (see Additional requirements).
- In subfolder
grids/, you should place the digital elevation model which was used for the Alpine3D simulation as well as the grid delineating the subwatersheds which was produced by TauDEM (see Generation of the stream network with TauDEM). This subfolder can optionally contain the grids of water exfiltration rate at the bottom of each soil column as well as the soil temperature grids generated by Alpine3D (see Minimum requirements and Additional requirements), if required.
runoff/is expected to group the SMET files generated by Alpine3D which contain the time series of the water infiltration rate into each subwatershed. In case you use the same root folder as for the Alpine3D simulation, you can actually let these files in the subfolder were Alpine3D has produced them (e.g.
output/runoff), but be sure to specify the correct path to them under key
[INPUT]of the StreamFlow configuration file (see Configuration file). As explained in Minimum requirements, this subfolder is only required in case the SMET files are available.
stream-network/subfolder is intended to group the two files defining the stream network structure which are generated by TauDEM (see Generation of the stream network with TauDEM).
output/folder will contain the output data generated by StreamFlow
calibration/subfolder is intended to receive the file containing the best model parameters when running StreamFlow in calibration mode.
runoff/subfolder will contain the time series of discharge and water depth (and optionally temperature) for each reach of the stream network.
- In folder
setup/you can place the configuration file for the StreamFlow simulation.
The configuration file of StreamFlow follows the same structure and syntax as the one of Snowpack or Alpine3D. In particular, each section is separated from the preceding one by a header contained within square brackets, e.g.
[INPUT]. Each controllable program option or parameter is defined by a unique keyword. In order to set the value of a given option or parameter, the user should write its associated keyword in the proper section of the configuration file, followed by an equal sign (
=) and the desired option value. For example, the simulation time step can be set to 1 minute by writing
TIME_STEP = 1 in section
[HYDROMODEL] of the StreamFlow configuration file. Most options have default values and are therefore not required to be present in the configuration file. Comments can be inserted anywhere in the file, they should start either with
# and extend until the end of the line.
As a general rule, it is recommended that you complete the configuration file of the Alpine3D simulation with the keys required by StreamFlow, since many parts are common to both programs. On top of the sections associated with MeteoIO (
[INTERPOLATIONS2D]), three new sections are required for a StreamFlow simulation. Section
[INPUTFROMA3D] specifies the location and format of the data generated by Alpine3D which are expected by StreamFlow as input. Section
[HYDROMODEL] defines which algorithms should be used in the model and specifies the values of the algorithm parameters. Section
[CALIBRATION] defines both the optimization technique and the objective function to be used when calibrating the model. Below is presented the minimum additional set of keys required to run a StreamFlow simulation on top of the keys used for a typical Alpine3D simulation. A complete description of all available keys can be found in the documentation of StreamFlow.
The keys in section
[HYDROMODEL] define the model structure and specify the parameter values.
The simulation time step is controlled by key
TIME_STEPand should be indicated in minutes:
Note that some modules of StreamFlow require the time step not to exceed a certain threshold: they will automatically adapt the time step to their needs. As such, the value that you indicate under key
[HYDROMODEL] TIME_STEP = 60 # in minutes
TIME_STEPshould be understood as a maximum allowed value rather than a fixed one.
StreamFlow has been designed so as to be as modular as possible regarding the following hydrological aspects:
The so-called transfer function, used to compute the lateral inflow discharge into the stream network from the surrounding hillslopes. The choice of the transfer function is controlled by key
This key is actually optional since only one transfer function (
[HYDROMODEL] SUBWATERSHED_CLASS = TWO_VERTICAL_LINEAR_RESERVOIRS
TWO_VERTICAL_LINEAR_RESERVOIRS) has been implemented in StreamFlow to date. This function corresponds to the vertical superposition of two linear reservoirs representing the fast and slow responses of the subwatershed (see documentation of StreamFlow for further details). Other options may become available in the future, though.
The algorithm used to compute the temperature of water exfiltrating out of the hillslopes into the stream network. StreamFlow currently offers four different options for this algorithm, which are all described into detail in the documentation. In the configuration file, the user should set the value of key
SUBWATERSHED_TEMPERATURE_MODELaccording to his choice:
[HYDROMODEL] SUBWATERSHED_TEMPERATURE_MODEL = [model]
[model]can be any of the following:
NONEin case the temperature of the exfiltration water should not be computed. This is the default option in case key
SUBWATERSHED_TEMPERATURE_MODELis not present in the configuration file.
HSPFif the temperature of exfiltrating water should be computed according to the algorithm implemented in the hydrological model HSPF (Hydrological Simulation Program Fortran). This option requires the specification of additional keys in the configuration file.
ENERGY_BALANCEin order to rely on the algorithm described in Comola et al., 2015, "Thermodynamics in the hydrologic response: Travel time formulation and application to Alpine catchments," Water Resources Research, 51(3): 1671-1687. This option requires the specification of additional keys in the configuration file.
SOIL_TEMPERATUREso as to set the temperature of exfiltrating water equal to the soil temperature right next to the stream.
The algorithm used to advect water within the stream network. The selection between the three possible algorithms is performed by setting the value of key
STREAM_CLASSto the desired value:
[HYDROMODEL] STREAM_CLASS = [model]
[model]can be any of the following (see documentation for more details on each option):
LUMPED_DIRECT_ROUTING, if water should be instantaneously routed through the stream network at each time step. This option treats each stream reach as a lumped entity, i.e. it computes a single value of discharge and water depth per stream reach. It requires the specification of the model used to compute water depth in the configuration file (see documentation). This is the default option in case key
STREAM_CLASSis not specified in the configuration file.
DISCRETIZED_DIRECT_ROUTING, which corresponds to the same algorithm as
LUMPED_DIRECT_ROUTINGbut for discretized stream reaches. It splits each reach into cells, for each of which a value of discharge and water depth is computed.
LUMPED_MUSKINGUM_CUNGEin order to rely on the Muskingum-Cunge algorithm, based on a lumped discretization of the stream reaches. This option requires the specification of key
MANNING_COEFFin the configuration file.
DISCRETIZED_MUSKINGUM_CUNGE, which is the same as
LUMPED_MUSKINGUM_CUNGEbut for discretized stream reaches
The algorithm used to compute water temperature within the stream network. The user can select between the two available options for this algorithm by setting the value of key
[HYDROMODEL] STREAM_TEMPERATURE_MODEL = [model]
[model]to be replaced with one of the following:
NONEin case stream temperature should not be computed. This is the default option.
ENERGY_BALANCEin order to compute stream temperature according to the energy-balance equation. This option requires the specification of additional keys in the configuration file and can only be chosen if
SUBWATERSHED_TEMPERATURE_MODELis not set to
NONE. Please refer to the documentation of StreamFlow for more details on this method.
- The so-called transfer function, used to compute the lateral inflow discharge into the stream network from the surrounding hillslopes. The choice of the transfer function is controlled by key
This section defines the formats and names of the input files required by StreamFlow.
StreamFlow determines the structure of the stream network based on two files generated by TauDEM (see section about TauDEM below). The location of these files should be indicated in the configuration file with dedicated keys:
[INPUT] TAUDEM_TREE_FILE = ./input/stream-network/dischmatree.dat # file defining the network structure TAUDEM_COORDS_FILE = ./input/stream-network/dischmacoord.dat # file containing the stream cell coordinates
In case you intend to calibrate StreamFlow, you have to specify the file(s) in which the discharge and/or temperature observations are located. These files can be in any of the formats supported by MeteoIO for meteorological measurements (see documentation of MeteoIO). The file format is to be indicated in the configuration file using key
The keys to be used in order to specify the location of the measurement files depend on the file format. Please refer to the specific documentation of each IO plugin in order to get the list of required keys. However, in order to avoid potential conflict with the keys used in the configuration file of Alpine3D for input meteorological data, the keys associated with the stream measurement files have to be renamed. You should simply replace the word
[INPUT] STREAM_MEASUREMENTS = SMET # can be any format supported by MeteoIO
STREAM_MEASUREMENT_in each key name. Additionally, keys
STATION#have to be renamed
MEASUREMENT_POINT#. The following table provides the equivalence between the keys to be used for input meteorological data and those associated with input stream data.
Keys for meteorological data Keys for stream data
outlet.smetcontaining discharge measurements at two gauging points along the stream network. If those two files are located in subfolder
input/measurementsof your simulation directory, you should write the following lines in your configuration file:
[INPUT] STREAM_MEASUREMENTS = SMET STREAM_MEASUREMENTS_PATH = ./input/measurements MEASUREMENT_POINT1 = midpoint.smet MEASUREMENT_POINT2 = outlet.smet
In case stream temperature is to be computed, StreamFlow requires the values of given meteorological variables (air temperature, wind velocity, relative humidity, and incoming short and long wave radiations) over each stream cell. Depending on the option that you chose when running Alpine3D (see above), these values may be spatially interpolated by StreamFlow based on the same station measurements or meteo grids than in Alpine3D (see Alpine3D documentation). As such, the same keys are required by StreamFlow that are also expected by Alpine3D to specify the location and format of the meteorological input data. For example, the interpolation of meteorological station measurements stored under the SMET format and located in folder
./input/meteocan be specified using the following keys:
In case of meteorological data interpolation, it is often necessary to specify additional keys in sections
[INPUT] METEO = SMET METEOPATH = ./input/meteo STATION1 = WFJ STATION2 = DAV STATION3 = STIL
[INTERPOLATIONS2D]of the configuration file as described in the MeteoIO documentation. These keys are used to remove outliers, fill measurement gaps and specify the algorithm to be used in order to spatially interpolate the data.
This section contains information about the files generated by Alpine3D which are required as input by StreamFlow. In case you use the same configuration file and simulation folder as for the Alpine3D simulation, you actually do not need to add any particular key here: StreamFlow will automatically find the required files. Otherwise you should follow the indications below.
In case you configured Alpine3D so that it computed the infiltration rates over the subwatersheds and wrote them in SMET files, you need to indicate the folder in which those SMET files are located:
[INPUTFROMA3D] CATCHMENTS_PATH = ./input/runoff # folder location may be different
Otherwise, if Alpine3D produced grids of water percolation rate at the bottom of each soil column, you have to indicate the format and folder (or file) in which those grids are:
In addition, you have to specify the format and location of the grid delineating the subwatersheds that is produced by TauDEM (see section about TauDEM below). Please note the following keys have to be placed in section
[INPUTFROMA3D] GRID2D = ARC # format may be different GRID2DPATH = ./input/grids # folder location may be different
[INPUTFROMA3D]) of the configuration file, in order to allow for the grid delineating the subwatersheds to have a different format than the percolation grids:
[INPUT] GRID2D = ARC # format may be different GRID2DPATH = ./input/grids # folder location may be different CATCHMENT = subwatersheds.asc # file name may be different
In order to compute stream temperature, StreamFlow expects the values of depth-averaged soil temperature over each stream cell. Depending on the configuration that you chose for Alpine3D (see above), two different sets of keys are to be added to the StreamFlow configuration file:
If you configured Alpine3D so as to generate grids of depth-averaged soil temperature, you need to specify to StreamFlow the format of those grids as well as the folder in which they are located:
Important note: the folder in which the depth-averaged soil temperature grids are located must be the same as the one containing the grids of water percolation rate (if any).
[INPUTFROMA3D] GRID2D = ARC # format may be different GRID2DPATH = ./input/grids # folder location may be different
In case you asked Alpine3D to produce a SMET file for each cell belonging to the stream network (by defining each stream cell as a point of interest), you should indicate the folder in which the SMET files are located:
[INPUTFROMA3D] STREAM_DATA_SOURCE = POI # Points Of Interest METEOPATH = [path] EXPERIMENT = [experiment]
[path]should contain the path to the folder containing the SMET files produced by Alpine3D for each point of interest, and
[experiment]should be identical to the value of key
[OUTPUT]of the Alpine3D configuration file.
- If you configured Alpine3D so as to generate grids of depth-averaged soil temperature, you need to specify to StreamFlow the format of those grids as well as the folder in which they are located:
This section specifies the nature and location of the output files which will be written by StreamFlow.
StreamFlow can produce output files for two types of elements:
The subwatersheds. One output file is created for each subwatershed in order to store the discharge and/or temperature of runoff at the subwatershed outlet. The production of these files can be activated with key
[OUTPUT] WATERSHED::WRITE = TRUE
The stream reaches, whose water depth, discharge, temperature and heat fluxes (e.g. incoming solar radiation, emitted long-wave radiation, etc.) can be written out. Only one file regrouping all the variables is created per lumped stream reach (e.g. when
STREAM_CLASS = INSTANTANEOUS_ROUTING, see above), whereas one file per stream reach and variable is created in case of discretized reaches (e.g. when
STREAM_CLASS = DIRECT_ROUTING). The production of these files is activated by setting key
[OUTPUT] STREAM::WRITE = TRUE
- The subwatersheds. One output file is created for each subwatershed in order to store the discharge and/or temperature of runoff at the subwatershed outlet. The production of these files can be activated with key
Both types of output files are produced in the same folder, which is defined using key
STREAM::PATH. The frequency at which data is written in those files is controlled by keys
STREAM::WRITE_STEP, both of which expect a value in days. Please mind that these frequencies should be specified with a sufficient enough precision, of the order of 6 digits after the comma:
[OUTPUT] STREAM::PATH = ./output/runoff WATERSHED::WRITE_STEP = 0.041667 # 1 hour STREAM::WRITE_STEP = 0.041667 # 1 hour
The list of variables printed in the output files can be controlled by the following optional keys:
WATERSHED::VARIABLES, which controls the variables printed in the files associated with the subwatersheds. This key can take one (or both) of the following values:
OUTLET_TEMPERATURE(separated by a space if both are specified). By default, both variables are printed in the output files.
[OUTPUT] WATERSHED::VARIABLES = OUTLET_DISCHARGE OUTLET_TEMPERATURE
STREAM::VARIABLES, which specifies the variables to be printed in the files associated with the stream reaches. This key can be set to any combination of the following values, separated by spaces:
NISWR(net incoming short wave radiation),
NILWR(net incoming long wave radiation),
ELWR(long wave radiation emitted by the stream),
LATENT_HF(latent heat flux),
SENSIBLE_HF(sensible heat flux),
BED_COND_HF(stream bed conductive heat flux),
BED_FRIC_HF(heat flux resulting from the friction of water on the stream bed). By default, only water depth, discharge and temperature are printed in the output files.
[OUTPUT] WATERSHED::VARIABLES = WATER_DEPTH DISCHARGE TEMPERATURE
You can restrict the set of subwatersheds or stream reaches for which an output file should be produced. By default, the states of all subwatersheds and stream reaches are written in output files. If you want to output these states only for a given number of elements, you can use the following keys:
The specified indices correspond to those attributed by TauDEM to each subwatershed and its associated stream reach. Note that these indices do not necessarily start from 0 and are generally not consecutive. For more information, please refer to the TauDEM documentation.
[OUTPUT] WATERSHED::INDICES = 0 4 15 STREAM::INDICES = 1 12 43 56
CALIBRATIONspecifies the SMET file in which the calibration module will print the best values of the model parameters. Please mind that this file will be overridden in case it already exists. For more information on the calibration procedure, see Calibration.
[OUTPUT] CALIBRATION = ./output/calibration/best_params.smet
In this section, you can configure the optimization algorithm used to calibrate StreamFlow. To date, only Monte Carlo optimization is available, but more advanced techniques might become available in the future.
NUM_ITERATIONScontrols the number of parameter sets to be randomly selected and tested by the Monte Carlo algorithm. In other words, this key controls the number of times that the model will be run. As a rule of thumb, it has been observed that about 1000 runs are sufficient to obtain good parameter estimates.
[CALIBRATION] NUM_ITERATIONS = 1000
The best parameter sets will be written in the file whose name is specified in key
[OUTPUT]. The number of sets which will be printed can be restricted using key
[CALIBRATION] NUM_PARAM_SETS_TO_KEEP = 15
The optimization algorithm ranks the tested parameter sets based on a so-called objective function. The latter corresponds to a measure of the model error, which is sought to be minimized by the optimization algorithm. As such, the best parameter set is the one associated with the lowest value of the objective function. In StreamFlow, the user is free to define the objective function of his choice based on the following error measures:
- the root mean square error (RMSE)
- the Nash-Sutcliffe efficiency (NSE), which is simply another name for the coefficient of determination (R2).
- the mean absolute error (MAE), which corresponds to the average of the absolute values of the model errors
- the bias, which is computed as the average value of the model errors
For example, assuming you have stream measurement data at two points p1 and p2 along the stream, the following expression would be a valid objective function:
4 × RMSE(discharge at p1) + 1.5 × MAE(temperature at p2) - 3 × NSE(water depth at p1)The corresponding expression in the configuration file would be:
[INPUT] STREAM_MEASUREMENTS = SMET STREAM_MEASUREMENTS_PATH = ./input/measurements MEASUREMENT_POINT1 = intermediate_point.smet MEASUREMENT_POINT2 = outlet_point.smet [CALIBRATION] OBJECTIVE_FUNCTION1 = RMSE OBJECTIVE_FUNCTION1::variable = discharge OBJECTIVE_FUNCTION1::weight = 4 OBJECTIVE_FUNCTION1::point = 1 OBJECTIVE_FUNCTION2 = MAE OBJECTIVE_FUNCTION2::variable = temperature OBJECTIVE_FUNCTION2::weight = 1.5 OBJECTIVE_FUNCTION2::point = 2 OBJECTIVE_FUNCTION3 = NSE OBJECTIVE_FUNCTION3::variable = water_depth OBJECTIVE_FUNCTION3::weight = -3 OBJECTIVE_FUNCTION3::point = 1
In general, each term of the objective function should be defined using the following keys:
[CALIBRATION] OBJECTIVE_FUNCTION# = [error measure] OBJECTIVE_FUNCTION#::variable = [variable] OBJECTIVE_FUNCTION#::weight = [weight] OBJECTIVE_FUNCTION#::point = [point id]
the hash symbol (
#) should be replaced with a number, starting from 1 and increasing iteratively for each new term of the objective function
[error measure]should be replaced with one of the following options:
[variable]should be replaced either with
[weight]can be any real number. Please mind that StreamFlow attempts at minimizing the objective function, which implies that the Nash-Sutcliffe efficiency should always be associated with a negative weight.
[point id]should be replaced with the ID number of the measurement point where the error measure should be computed. The ID number corresponds to the value replacing
#in the key
[INPUT]) which is associated with the point of interest. For example, assuming that file
point.smetcontains data measured at a certain point, and that you wrote
MEASUREMENT_POINT3 = point.smetin your configuration file, the ID number associated with the point in question will be 3.
Minimal configuration file
In summary, a StreamFlow configuration file should at least have the following keys, on top of those contained in the Alpine3D configuration file:
[HydroModel] TIME_STEP = 60 SUBWATERSHED_CLASS = TWO_VERTICAL_LINEAR_RESERVOIRS SUBWATERSHED_TEMPERATURE_MODEL = HSPF STREAM_CLASS = INSTANTANEOUS_ROUTING STREAM_TEMPERATURE_MODEL = ENERGY_BALANCE [Input] TAUDEM_TREE_FILE = ./input/stream-network/dischmatree.dat TAUDEM_COORDS_FILE = ./input/stream-network/dischmacoord.dat STREAM_MEASUREMENTS = SMET STREAM_MEASUREMENTS_PATH = ./input/measurements MEASUREMENT_POINT1 = outlet_point.smet [InputFromA3D] CATCHMENTS_PATH = ./input/runoff [Output] WATERSHED::WRITE = TRUE STREAM::WRITE = TRUE STREAM::PATH = ./output/runoff WATERSHED::WRITE_STEP = 0.041667 # 1 hour STREAM::WRITE_STEP = 0.041667 # 1 hour CALIBRATION = ./output/calibration/best_params.smet [Calibration] NUM_ITERATIONS = 1000 NUM_PARAM_SETS_TO_KEEP = 10 OBJECTIVE_FUNCTION1 = NSE OBJECTIVE_FUNCTION1::variable = temperature OBJECTIVE_FUNCTION1::weight = -1 OBJECTIVE_FUNCTION1::point = 1
Generation of the stream network with TauDEM
TauDEM (Terrain Analysis Using Digital Elevation Models) is an open-source software which was developed by David Tarboton from Utah State University. Its main purpose is to automatically extract stream networks from digital elevation models (DEMs). You can find compiled versions of the software as well as its associated documentation on the official TauDEM webpage. In particular, it is highly recommended that you go through the quick start guide in order to get more familiar with the TauDEM interface. Note that ArcGIS users can install TauDEM as an external toolbox.
StreamFlow expects three of the files generated by the TauDEM tool called Stream Reach And Watershed. This tool can unfortunately not be run out-of-the-box, since it requires extra files from other TauDEM tools. The general chain of tools to be used is the same as explained in the quick start guide:
- Pit Remove in order to fill the pits in the DEM
- D8 Flow Directions in order to compute the flow direction in each DEM cell
- D8 Contributing Area so as to compute the drainage area in each DEM cell
- Stream Definition by Threshold, in order to extract a first version of the stream network
- Move Outlets to Streams, so as to move the catchment outlet point on one of the DEM cells identified by TauDEM as belonging to the stream network
- D8 Contributing Area again, but with the catchment outlet point as additional input data
- Stream Definition by Threshold again, but with the catchment outlet point as additional input data
- Stream Reach And Watershed
The following three files generated by tool Stream Reach And Watershed are required by StreamFlow:
- The grid delineating the subwatersheds, named Output Watershed Grid in the TauDEM documentation . This grid is created in the geoTIFF format by TauDEM and needs to be converted to a format which is readable by Alpine3D. It is recommended that you place this grid in folder
- The ASCII file defining the connectivity of the stream network, referred to as Output Network Connectivity Tree in the TauDEM documentation . It is recommended that you place this file as well as the following one in folder
- The ASCII file containing the coordinates of the stream cells, referred to as Output Network Coordinates in the TauDEM documentation .
input/gridsof your simulation directory.
input/stream-networkof your simulation directory.
StreamFlow has essentially two different configurations: a lumped one and a discretized one. The lumped version is activated by setting the value of key
LUMPED_MUSKINGUM_CUNGE (see section about the configuration file above) and treats every stream reach as a single lumped entity. It provides estimates of the outlet discharge and mean water depth for every stream reach. The discretized version, on the other hand, subdivides each stream reach into finer cells, which allows for intermediate estimates of water depth and discharge within each reach. It is activated by setting the value key of
The two configurations (lumped and discretized) provide very similar estimates of discharge at the downstream end of each stream reach, but the discretized version has a much finer spatial resolution which might be of interest in some cases. This greater resolution is counterbalanced by a significant increase in the simulation time: when a lumped simulation needs about 1 to 2 minutes to complete, a discretized simulation might require up to 12 hours. As such, it is recommended to always select the lumped version when calibrating the model. Only when you have good parameter estimates can you run the discretized version.
As an additional advice, we recommend you to first de-activate the stream temperature computation. This speeds up the simulation quite significantly and allows you to efficiently calibrate the parameters associated with stream flow (channel width, Manning coefficient, etc.). Once these parameters are set, you can turn the stream temperature module on and proceed with the calibration of its associated parameters. This requires to keep the value of every stream flow parameter constant, which is achieved adding the following key in the configuration file:
[HYDROMODEL] [parameter name]::CALIBRATE = FALSE
[parameter name] should be replaced with the parameter name.
Calling the executable
In order to launch a StreamFlow simulation, you need to open a terminal and move to the
bin/ folder of your simulation directory, which should contain a copy of the StreamFlow executable. The list of options to pass to the executable can be obtained by typing