MeteoIODoc 20240329.18c31bd1
Processing elements developer's guide

In order to add a new filter/processing element to the already existing set of components, the developer only needs to add a class derived from either (in meteoio/meteoFilters):

  • ProcessingBlock, for filters that only look at one point at a time;
  • or WindowedFilter, for filters that work on a whole time window at a time. It provides members to read the time window definition as well as to compute the proper indices matching the user-defined time window;

Of course, each developer is free to bypass the helper methods provided in these two classes and reimplement his/her own if the provided ones don't fit with the needs of a specific processing element (but it must at least be derived from ProcessingBlock). Templates header and code files are available to get you started, look into the "meteoFilters" subdirectory of the source directory (files "template.cc" and "template.h"). It is important to understand that the processing elements operate on a "per parameter" basis. This means that an element might be executed for the parameter TA and another one for the parameter PSUM, so the algorithm only has to deal with a generic processing method based on double values.

To implement a new processing element, the following steps are necessary:

  1. Create a derived class of ProcessingBlock or WindowedFilter in the meteoFilters subdirectory of the source code. You can copy and rename the template files "template.cc" and "template.h" that are in the "meteoFilters" subdirectory. Please do not forget to rename all occurences of "TEMPLATE" in these files! Keep the process method as it is for now.
  2. Add the created implementation file to meteoFilters/CMakeLists.txt in a similar way as for the other filters
  3. Add the filter in the processing loop, in meteoFilters/ProcessingBlock.cc in the BlockFactory::getBlock() method by adding three lines similar to:
    else if (blockname == "MIN_MAX"){
    return new FilterMinMax(vecArgs, blockname);
    }
    The key (here the string "MIN_MAX") is the key that the user will put in his io.ini to select the processing block.
  4. Include the filter's header file in meteoFilters/ProcessingBlock.cc
  5. Try to compile and run your filter on a test data set (for example with the "meteo_reading" example)
  6. Then really implement your filter. Its class contains two public methods: a constructor and a "process" method and at least one private method, "parse_args" to read the arguments from a provided vector of strings.
    1. The constructor takes a vector of strings containing the element's arguments and a constant string (that contains the block name, for example for printing in an error message from which filter/block it comes). The constructor would therefore have a declaration similar to:
      FilterMax::FilterMax(const std::vector< std::pair<std::string, std::string> >& vecArgs, const std::string& name)
      FilterMax(const std::vector< std::pair< std::string, std::string > > &vecArgs, const std::string &name, const Config &cfg)
      Definition: FilterMax.cc:25
    2. The process method applies the element to the provided vector of values, for a meteo parameter pointed to by index. This index is the MeteoData parameter that this filter shall be run upon (see MeteoData for the enumeration of parameters). The constructor must set up processing.stage to mark if the filter should be applied only during the first pass (ie before the resampling), or both at the first and second pass (ie before and after resampling). Its declaration is such as:
      process(const unsigned int& index, const std::vector<MeteoData>& ivec, std::vector<MeteoData>& ovec)
    3. The private parse_args method reads the arguments from a vector of pairs of strings, with the following declaration:
      parse_args(const std::vector< std::pair<std::string, std::string> >& vecArgs)

Although you are encouraged to use the provided templates (files "template.cc" and "template.h" in the meteoFilters subdirectory), the class FilterMax can be used as an example of implementation of a basic filter that will check whether a value is greater than an argument supplied to the filter and if so changes the value either to IOUtils::nodata (normal operation) or to the maximum value supplied in the argument (soft mode of operation). An example section in the io.ini file supplied to the Config could look like this:

[Filters]
TA::filter1 = max
TA::arg1::soft = true
TA::arg1::max = 280

Which has the following interpretation: Apply filter max (max-value-filter) to the parameter TA (air temperature) in case that a value is greater than 280 degrees Kelvin change that value to 280. A more customized operation could be:

[Filters]
TA::filter1 = max
TA::arg1::soft = true
TA::arg1::max = 280
TA::arg1::max_reset = 260

Which will replace any value greater than 280 Kelvin by 260 Kelvin.

Another good example is the FilterUnheatedPSUM that uses a combination of several meteorological parameters to filter the precipitation.