Edit source

Design considerations

  • I want to reuse as many parameter structs as possible, because of DRY and to simplify/unify documentation of those.

  • Problem is that some modules will require different parameters (or only a subset) to work, e.g. AV module and all version thereof. Q: Would it be a good idea to make a parameter struct for each AV method? +: Makes it clear which parameters to be used for each method. -: Risk to violate DRY, because e.g. MDA uses a subset of parameters of MDH. Ad: In case of AV models this might be an invalid concerne, because the parameters of MDH and MDA, although one model is derived from another, have distinct meaning in their implementation. What this meaning is has to be documented!

  • IO: Use TOML and dictionaries as intermediate communication format. +: Julia has native TOML support which parses automatically to dictionaries. -: ... Q: What were the reason to prefer TOML over YAML? Real life example: gist +: In the above example YAML looks nicer, but TOML allows to remove nested levels easily with the bracket syntax. Only drawback of TOML would be that strings always have to be enclosed in double quotes, but IRC TOML.jl parses everything as strings automatically, or? Unfortunately, it does care about double quotes ... This is kind of a bummer, although, it seems to be included in the TOML specs to allow for unambigoutus parsing ... +: The YAML spec seems to have many features, but YAML.jl also allow to parse everything as an Dictionary{Symbol,Any}, e.g. through everything into the same bin. Although, the package allows to parse YAML directly to structs, I would rather leave the type conversion to the maintainer of the corresponding struct.

    Decision: Use TOML and parse everything into dictionaries.

  • DSL:

    !!! DSL is actually just syntactic icing on top of (a set of) functions. I should first come up with the functions before thinking about the syntax.

Independently of the actual file format, the parsed parameter file should always be converted into a dictionary which we then further process. +: Dictionaries only allow unique key names.

The idea is to then map those names to members of some parameter class. Those members can be also parameter structs themselves. The fields could then be documented using julia's doc syntax. Q: Ability of ignoring parameters of certain sub-structs? QQ: Is this even a thing? When would this be needed? Probably YAGNI...

Q: What to do if multiple instances of a parameter struct are needed? E.g. MDA1, MDA2, ...? PA: Build parameter structs with meaninungful and non-duplicated member names (dictionary key names must be unique) and use those names as TOML headers. TOML parsing happens separated from constructing the parameter struct. So the user then has to explicitly specify which parameter struct he wants to construct out of the resulting dictionary. The parameter struct must be fully type annotated so that one can figure out dynamically/programmatically which sub-parameter struct must be called if a certain key is found in a dictionary. -: This might introduce inconsistency between projects, e.g. one names the evolution struct evolve, the other calls it time_stepping ... Q: Is this really a problem if one documents the member names properly? PA: Separate projects are separate, should one really care about what others do? If I would not want this, then I should use the module name, because that will be unqiue. Problem with module names is that the scope of nested modules will have to be accounted for in the header naming. Provided we only use the actual module name, without module scope, then this could break uniqueness, because different modules could contain submodules that are named the same. Also: What if multiple parameter modules are needed in a module? This would not work with using the module name as a unique identifier.

  • Q: How to do parameter validation? A: This sould happen in the sub-parameter constructors.

  • Q: Could this become a separate Julia package? Q: How much does this really differ from Configurations.jl?

  • How to handle invalid keys?

They could be 1. ignored – does not encourage to keep parameter files clean, 2. ignored with a warning – does encourage to keep parameter files clean, but this can be also ignored by the user, 3. or errored out – this can break backwards compatibility, but enforces clean parameter files. Q: How relevant is backwards compatibility? A: If a proper logging policy is established that includes log info about current git version, then it is not relevant at all. If needed, one could write a converter to update parameter files on a case by case basis.

  • Add syntax for 'base' and 'special' parameter file. E.g. One prepares a file with all parameters that should be common for all runs and includes a file, or passes a second file, that may override certain parameters. +: DRY in input. Q: Is this really needed? Or isn't the standard case that we use julia anyways to generate parameter configurations and then call the programs? On a second thought this might be a overkill here, because parsing parfiles in julia should be really simple with the above strategy. The (current) idea is that in the end the parameters will be merged again into a single file, so then this feature is only concerned with convinience that can be dealt with julia alone also. Especially, since one still has to write the 'special' parfiles for each run ...

=====

New draft

abstract type AbstractParameter end

struct Parameter
  key::String
  type::DataType
  value
  default
  options
  doc::String
end

struct ParameterOption
  list::Vector{Parameter}
end

struct ParameterTable
  table::Dict{Symbol,AbstractParameter}
end

# e.g. let every configureable item implement a method
# parameters(::Type{Item}, kwargs...) that builds up a list of default parameters
# that also include docs and options
# given a dict of input parameters, we then match it against the default parameters to
# fill up all unset parameters and check if there are two many parameters set
# if needed, one should also overload sanitize(prms::Parameters{Type}) to check consistency
# to combine all parameters we add them up in a table and group them by sections.
# every section should be able to handle multiple options, which one to pick should be
# indicate by a keyword, e.g. 'type'
# Example
# ```toml
# [HRSC]
# Option = "EntropyViscosity"
# cmax = 1.0
# ce = 0.5
# [Evolution]
# algorithm = "LSERK"
# ...
# ```