MeteoIODoc  MeteoIODoc-2.9.0
mio::FilterMaths Class Reference

## Detailed Description

A filter that evaluates formulas dependent on conditions.

Date
2019-09-01

This filter evaluates an arithmetic expression and filters the meteo parameters to the result. You can use a number of substitutions from your data and meta data.

List of ini keys

Example: Convert from degrees Rankine to Kelvin:

TA::FILTER1 = MATHS
TA::ARG1::FORMULA = meteo(TA) * 5/9

You can define a condition that must be met for the filter to take action. The `EXPRESSION1` and `COMPARE1` keywords denote two formulas and their evaluated results will be compared by the operator given with `OPERATOR1`.

Example: Allow no snow below 500 m altitude:

HS::FILTER1 = MATHS
HS::ARG1::FORMULA = 0
HS::ARG1::EXPRESSION1 = altitude
HS::ARG1::OPERATOR1 = LT
HS::ARG1::COMPARE1 = 500

Operators: The following comparison operators are available: `LT` (less than), `LE` (less than or equal), `GT` (greater than), `GE` (greater equal), `EQ` (equal), `NE` (not equal), `STRCMP` (string comparison), `STRBEG` (string comparison at the beginning of a string), `STREND` (string comparison at the end of a string), and `STRINC` (search in string).

Expressions: Arithmetic expressions are evaluated with tinyexpr (developer's repo). You can use the standard arithmetic operations, angular functions, exponentials, and some combinatorics.

Note
Here is a full list of the available functions: addition (`+`), subtraction/negation (`-`), multiplication (`*`), division (`/`), exponentiation (`^`), modulus (`%`), `abs`, `acos`, `asin`, `atan`, `atan2`, `ceil`, `cos`, `cosh`, `exp`, `floor`, `ln` (calls to log), `log` (calls to log10), `log10`, `pow`, `sin`, `sinh`, `sqrt`, `tan`, `tanh`, `fac`, `ncr`, `npr`. It also features the constants `e` and `pi`.

Substitutions: The following substitutions are available in all arithmetic expressions:

• `meteo(PARAM)` where `PARAM` is any available meteo parameter,
• `year`, `month`, `day`, `hour`, `minute`, `julian` for the measurement's date,
• `altitute`, `latitude`, `longitude`, `easting`, `northing` for the station's position,
• `angle`, `azimuth` for slope parameters,
• `stdpress`, `stdtemp`, `boltzmann` for (meteorological) constants,
• `nodata` for the nodata value (usually -999).

The following substitutions are available in string comparisons:

• `stationid`, `stationname` for station identification.

Example: Add offset to specific station:

HS::FILTER1 = MATHS
HS::ARG1::FORMULA = meteo(HS) + 0.38
HS::ARG1::EXPRESSION1 = stationid
HS::ARG1::OPERATOR1 = STRCMP
HS::ARG1::COMPARE = ISEE3

If the condition is not met, you can use a different expression to evaluate via the `FORMULA_ELSE` key.

Example: If there is no wind direction available then throw away the wind speed. Else, use a logarithmic scale:

VW::FILTER1 = MATHS
VW::ARG1::FORMULA = nodata
VW::ARG1::FORMULA_ELSE = log(meteo(VW))
VW::ARG1::EXPRESSION1 = meteo(DW)
VW::ARG1::OPERATOR1 = EQ
VW::ARG1::COMPARE1 = nodata

You can make use of a simple logic parser. You can supply as many conditions as you wish, but they are always combined with the same logical operator, namely `CONNECTIVE` can be `AND` (default) or `OR`.

Example: Allow no snow between June and August:

HS::FILTER1 = MATHS
HS::ARG1::FORMULA = 0
HS::ARG1::EXPRESSION1 = month
HS::ARG1::OPERATOR1 = GE
HS::ARG1::COMPARE1 = 6
HS::ARG1::EXPRESSION2 = month ;AND is implied
HS::ARG1::OPERATOR1 = LE
HS::ARG1::COMPARE1 = 8

Or alternatively:

HS::FILTER1 = MATHS
HS::ARG1::FORMULA = 0
HS::ARG1::CONNECTIVE = OR ;switch logical connective to OR for all
HS::ARG1::EXPRESSION1 = month
HS::ARG1::OPERATOR1 = EQ
HS::ARG1::COMPARE1 = 6
HS::ARG1::EXPRESSION2 = month
HS::ARG1::OPERATOR2 = EQ
HS::ARG1::COMPARE2 = 7
HS::ARG1::EXPRESSION3 = month
HS::ARG1::OPERATOR3 = EQ
HS::ARG1::COMPARE3 = 8

Important: Save for the `SKIP_NODATA` keyword (see below), nodata values (usually -999) are kept in the calculations and so far it is up to the user to check against this!

Example: Approximate the dew point only if air temperature and relative humidity are both available:

TD::FILTER1 = MATHS
TD::ARG1::FORMULA = meteo(TA) - ((100 - meteo(RH)*100) / 5)
TD::ARG1::FORMULA_ELSE = nodata
TD::ARG1::EXPRESSION1 = meteo(RH)
TD::ARG1::OPERATOR1 = NE
TD::ARG1::COMPARE1 = nodata
TD::ARG1::EXPRESSION2 = meteo(TA)
TD::ARG1::OPERATOR2 = NE
TD::ARG1::COMPARE2 = nodata
TD::ARG1::EXPRESSION3 = meteo(RH)
TD::ARG1::OPERATOR3 = GT
TD::ARG1::COMPARE3 = 0.5 ;formula within 1 degree only for RH > 50 %

Example: Two stations are called ISEE1 and ISEE2. Suppose their humidity sensors start reporting garbage on 1st of August 2019 and we want to make sure they are not used in any calculations:

RH::FILTER1 = MATHS
RH::ARG1::FORMULA = nodata ;throw all data away
RH::ARG1::EXPRESSION1 = stationid
RH::ARG1::OPERATOR1 = STRBEG ;compare beginning of string
RH::ARG1::COMPARE1 = ISEE ;this string is as-is, i. e. no substitutions
RH::ARG1::EXPRESSION2 = julian
RH::ARG1::OPERATOR2 = GE
RH::ARG1::COMPARE2 = 2458696.5 ;2019-08-01

With the `ASSIGN` key you can provide a parameter that the result is saved to (instead of the one the filter runs on).

Important: For technical reasons, this key only works as expected if it's the last filter that is run on the set.

Example: A copy-heavy hack to make a `SHADE` processor run on a specific station only - without even calculating the heavy sun stuff for others (we must control the filter ordering to be able to do this):

[INPUT]
ISWR_TEMP::COPY = ISWR ;CREATE would refill
[FILTERS]
ISWR_TEMP::FILTER1 = MATHS ;keep only if station name matches
ISWR_TEMP::ARG1::FORMULA = meteo(ISWR_TEMP)
ISWR_TEMP::ARG1::FORMULA_ELSE = nodata
ISWR_TEMP::ARG1::EXPRESSION1 = stationname
ISWR_TEMP::ARG1::OPERATOR1 = STRCMP
ISWR_TEMP::ARG1::COMPARE1 = Seegrube
ISWR_TEMP::FILTER2 = SHADE ;only 1 station has non-nodata values
ISWR_TEMP::FILTER3 = MATHS ;copy back if not nodata
ISWR_TEMP::ARG3::FORMULA = meteo(ISWR_TEMP)
ISWR_TEMP::ARG3::FORMULA_ELSE = meteo(ISWR)
ISWR_TEMP::ARG3::EXPRESSION1 = meteo(ISWR_TEMP)
ISWR_TEMP::ARG3::OPERATOR1 = NE
ISWR_TEMP::ARG3::COMPARE1 = nodata
ISWR_TEMP::ARG3::ASSIGN = meteo(ISWR) ;must be last filter to work!
Note
With the same method you could build a chain like `if (A or B) and C` if you must.

The last remaining key is `SKIP_NODATA` to transport nodata values unaltered from input to output. It is false by default which means that all formulas are evaluated with it (and you could catch it with `meteo(XX) EQ nodata`).

# List of ini keys

KeywordMeaningOptionalDefault Value
FORMULAArithmetic expression to evaluate for meteo parameter.no-
FORMULA_ELSEExpression if the conditions below are not met.yesempty, meaning the data point is left alone
CONNECTIVELogical connective for the conditions below.yesAND (choices: OR)
EXPRESSION#1st part of condition: Expression that will be compared.yes, if neither OPERATOR# nor COMPARE# exist.empty, meaning FORMULA will always apply
OPERATOR#2nd part of condition: Comparison operator.yes, if neither EXPRESSION# nor COMPARE# exist.empty (choices: LT, GT, EQ, ...)
COMPARE#3rd part of condition: EXPRESSION is compared against this.yes, if neither EXPRESSION# nor OPERATOR# exist.empty
ASSIGNChange parameter to store final result in (only if it's the last filter running).yesempty (choices: meteo(PARAM) or PARAM)
SKIP_NODATAAny input nodata stays untouched. Checks for nodata in COMPARE will not be called.yesfalse

`#include <FilterMaths.h>`

## Public Member Functions

FilterMaths (const std::vector< std::pair< std::string, std::string > > &vecArgs, const std::string &name)

virtual void process (const unsigned int &param, const std::vector< MeteoData > &ivec, std::vector< MeteoData > &ovec)
The filtering routine as called by the processing toolchain. More...

Public Member Functions inherited from mio::ProcessingBlock
virtual ~ProcessingBlock ()

virtual void process (Date &dateStart, Date &dateEnd)

std::string getName () const

const ProcessingPropertiesgetProperties () const

const std::string toString () const

bool skipStation (const std::string &station_id) const

bool noStationsRestrictions () const

## Additional Inherited Members

Static Public Member Functions inherited from mio::ProcessingBlock
static void readCorrections (const std::string &filter, const std::string &filename, std::vector< double > &X, std::vector< double > &Y)

static void readCorrections (const std::string &filter, const std::string &filename, std::vector< double > &X, std::vector< double > &Y1, std::vector< double > &Y2)

static std::vector< double > readCorrections (const std::string &filter, const std::string &filename, const size_t &col_idx, const char &c_type, const double &init)

static std::vector< offset_specreadCorrections (const std::string &filter, const std::string &filename, const double &TZ, const size_t &col_idx=2)

static std::map< std::string, std::vector< dates_range > > readDates (const std::string &filter, const std::string &filename, const double &TZ)

static std::set< std::string > initStationSet (const std::vector< std::pair< std::string, std::string > > &vecArgs, const std::string &keyword)

Protected Member Functions inherited from mio::ProcessingBlock
ProcessingBlock (const std::vector< std::pair< std::string, std::string > > &vecArgs, const std::string &name)
protected constructor only to be called by children More...

Static Protected Member Functions inherited from mio::ProcessingBlock
static void extract_dbl_vector (const unsigned int &param, const std::vector< MeteoData > &ivec, std::vector< double > &ovec)

static void extract_dbl_vector (const unsigned int &param, const std::vector< const MeteoData *> &ivec, std::vector< double > &ovec)

Protected Attributes inherited from mio::ProcessingBlock
const std::set< std::string > excluded_stations

const std::set< std::string > kept_stations

ProcessingProperties properties

const std::string block_name

Static Protected Attributes inherited from mio::ProcessingBlock
static const double soil_albedo = .23

static const double snow_albedo = .85

static const double snow_thresh = .1
parametrize the albedo from HS More...

## ◆ FilterMaths()

 mio::FilterMaths::FilterMaths ( const std::vector< std::pair< std::string, std::string > > & vecArgs, const std::string & name )

## ◆ process()

 void mio::FilterMaths::process ( const unsigned int & param, const std::vector< MeteoData > & ivec, std::vector< MeteoData > & ovec )
virtual

The filtering routine as called by the processing toolchain.

Parameters
 [in] param Parameter index the filter is asked to run on by the user. [in] ivec Meteo data to filter. [out] ovec Filtered meteo data. Cf. main documentation.

Implements mio::ProcessingBlock.

The documentation for this class was generated from the following files: