In the following sections, we will describe all the elements of an MDLTL source file. To start, we will have a look at an example file that can be found in the MDL SDK API distribution at examples/mdl_sdk/distilling_target/distilling_target_plugin_rules.mdltl:
The example file contains one rule set called Make_simple_rules. This rule set starts with an import statement for MDL module math, followed by a number of rules. Rules are defining transformations: the left-hand side of a rule determines to which nodes a rule applies, and the right-hand side defines what a matching node will be replaced with.
All these elements are described in detail in the following sections.
An MDLTL file (usually a file with file name extension .mdltl) is a compilation unit. Each MDLTL file is compiled into a .cpp and a .h file, which declare and define one C++ class for each rule set in the input file. Each class is a subclass of the class mi::mdl::IRule_matcher from the MDL SDK, which enables the class to be used by the Distiller. The MDLTL compiler can process multiple MDLTL files when invoked, and will generate C++ source code files for each of them.
The syntax for patterns and expressions in MDLTL is based on the syntax of MDL: basic elements have the same representation (identifiers, strings, numbers, punctuation, comments, function calls, arithmetic expressions). The MDLTL syntax defines a few additional syntactic categories, such as extra punctuation and keywords for describing rules and their application.
A rule set consists of a name, an evaluation strategy, import statements, a list of zero or more rule definitions and an optional postcondition expression. A rule set name must be a valid identifier. Each rule set is compiled into a C++ class with a name corresponding to the rule set name. The generated C++ class inherits from the abstract mi::mdl::IRuleMatcher class.
The following example shows a simple rule set with two rules, taken from the above example file.
The evaluation strategy is either bottomup or topdown (as in the example above). The strategy defines the order of application of all the rules defined in a rule set when the rule set is applied at runtime by the Distiller. For topdown, rules are first applied to the node at the root of the material graph, then to its children, and so on. When the strategy is bottomup, matching starts at the leaves of the graph, followed by the parents, until it reaches the root.
Within a rule set, for each node, rules are applied from the first to the last in order, until the left-hand side of a rule matches the node that is currently being processed. The matched node is replaced by the right-hand side expression. If annotated with the return code repeat_rules, the whole rule set is applied again on the new node until either no rule matches or a rule without a repeat_rules annotation is matched. The annotation skip_recursion stops rule application on the current node so that child nodes will not be matched. skip_recursion is only applicable for topdown rules. See below in section Rules for more details on these return codes.
The order of rules in rule sets only defines the evaluation order of the rules within one set, not how multiple rule sets are combined. The order of applications of rule sets, the number of times they are applied and how the results of rule set applications are combined is implemented in Distiller plugins and varies depending on the Distiller target.
At the beginning, a rule set can import MDL modules with an import statement to make their exported materials known to the MDLTL compiler.
At the end of a rule set, a postcondition can be defined. Postconditions are assertions on the structure of a material after it has been processed by a rule set. A postcondition is a boolean expression containing only boolean operators and calls to the special predicate functions nonode and match. When a rule set has been fully applied to a material graph, the Distiller applies postcondition checks to verify that the Distilling results have the expected structure. If the postcondition check fails, the distilling process fails with an error code.
When applying a nonode predicate, the Distiller traverses the whole graph to ensure that the node name does not appear anywhere. A match predicate is checked by verifying that the root node matches the pattern given.
As an example, the following postcondition states that the resulting material does not contain any occurrences of the node bsdf_color_mix1, bsdf_color_mix2 and bsdf_color_mix3:
The following postcondition asserts that the resulting material matches the structure of a pattern. In this example, the postcondition states that the distilled material has both a surface and a backface with a simple diffuse_reflection_bsdf:
Each rule definition consists of
skip_recursion or repeat_rules)dead_rule declaration.The meaning of the return code is as follows:
skip_recursion: after rule application, no further matching on the current node or its sub-expressions is done (for top-down evaluation)repeat_rules: after rule application, all rules of the current rule set are matched again on the result node.A guard expression must be a boolean expression. On each rule application, after the pattern of the rule is matched, the guard expression is evaluated and if it is false, the rule is skipped. All variables used in guard expressions must be bound in the pattern or the where clause of the rule.
Here is a rule that only is applied if the w parameter is the color black:
A where clause binds variable names to expressions. This may be useful if the same expression is used multiple times in the replacement expression, guard expression or other where clauses. The following example uses two where-bound variables to make the code more readable:
Three are two kinds of debug statements:
_dbg_debug_print set to true.The following example illustrates the use of these debug statements:
When this rule is matched, the name of the rule will be remove_glossy in trace and debug output, and the values of the variables d, rv and ru will be displayed to the user during matching. The format of the output and how it is presented is dependent on the integrating application.
The dead_rule declaration marks the rule as one that is not expected to match a node. This is mainly a tool for testing and can be used by coverage tools to exclude the rule.
A pattern is a limited form of an expression, restricted to variables, function applications, type annotations, node aliases and attribute patterns.
A variable named _ (underscore) is allowed as a wildcard pattern. When such a wildcard variable is matched, it matches any expression and does not bind any variables.
A wildcard variable can appear multiple times and each occurrence matches a different expression. Other variables can only appear once in a pattern.
Variables match any expression and bind a name to the matched expression, which can be referred to in the right-hand expression of a rule, guard expressions and where clauses. Variables can be annotated with types in type annotations.
A pattern can be a node alias pattern.
An pattern can be followed by an attribute pattern set. Variables bound in attribute patterns have the same scope as variables bound in other pattern variables. Attribute names are scoped over all rules in a rule set and must have the same types in all rules of the pattern set.
All functions mentioned in a pattern must be
nvidia::distilling_support module, orOnly functions with the following result types can appear in a pattern:
materialmaterial_surfacematerial_geometrymaterial_volumematerial_emissionbsdfedfvdfhair_bsdfThis includes constructor functions for the BSDF and material-related types like bsdf(), edf(), material(), material_surface() etc.
Expressions are MDL expressions and appear on the RHS of rules. They consist of function calls, variables, type annotations, node alias expressions and operator expressions.
Expressions can only call functions
nvidia::distilling_support module,All variables used in expressions must be bound in the corresponding pattern or the where clause of the rule.
An expression can be followed by an attribute expression set (see section Attribute expression sets). When the expression is evaluated, the attributes are attached to the resulting node as a side effect. A rule matched later in the Distilling process can than match against the attached attributes.
Attributes are a mechanism for passing values up and down the material graph while performing transformations. Attributes are mapping nodes to sets of key-value pairs, where keys are strings and values are arbitrary expression nodes. The LHS of a rule can match on these attributes to select which rules are applied, and the RHS can attach attributes to newly constructed nodes. Attributes that are being matched on the LHS are defined by attribute pattern sets, and attributes on the RHS are defined by attribute expression sets.
Attributes are retained between applications of different rule sets and can therefore be used to pass values from one rule set to another.
An attribute pattern set consists of a comma-separated sequence of pairs of the form variable ~ pattern between delimiters [[ and ]]. As a shorthand, instead of writing variable ~ _, just writing variable is allowed.
Attribute pattern set matching works as follows:
First, the node pattern to which an attribute pattern set is attached is matched as usual against the current node. For each pair in the attribute pattern set, the attribute name is looked up in the attribute mapping for the current node. If there is no such entry in the map, matching fails. Otherwise, the pattern of the pair is matched against the value in the mapping.
When matching succeeds for the pattern itself and for all elements of the attribute pattern set, all free variables in the patterns are bound to the values they matched against.
The following rule illustrates the use of an attribute pattern. The rule matches on material nodes where the material_surface field contains a diffuse_reflection_bsdf node. Also, the node that is being matched must have two attributes attached: (1) an attribute called is_thin_walled and (2) and attribute called some_color. When the rule is matched, the Distiller ensures that the matched node has the same shape and node types as the pattern. Additionally, it checks that all attributes named in the attribute pattern set are attached to the node and their patterns match the corresponding attribute values.
On the RHS of the rule, the value of the is_thin_walled attribute is used when constructing the result value.
An attribute expression set consists of a comma-separated sequence of pairs of the form variable = expression between delimiters [[ and ]]. The role of an attribute expression set is to attach the given attributes to the node which is created by the expression.
In the following rule, the LHS matches any material node, binding all subexpressions to variables. The RHS constructs an identical material (since the subexpressions are taken directly from the input node) and an attribute set is attached to the result material. This attribute set binds the attribute name is_thin_walled to the value of the thin_walled field of the input material.
Patterns on the left-hand side of a rule can be given names using node aliases. The name given to a pattern can then be used on the right-hand side when constructing a replacement node. This allows the user to re-use parts of patterns (or whole patterns) on the right-hand side without having to writing out the full replacement expression.
A node alias is denoted by prefixing a pattern with a variable name and a ~ (tilde) symbol, like this: name ~ pattern.
In the following example, both patterns are given the name b, and this variable is used to construct the replacement node. This means that the rule does not change the material graph at all, but the use of attribute expression sets means that after the transformation is done, all bsdf() and diffuse_reflection_bsdf() nodes in the graph will have an attribute a attached, with either the values 1 or 2.
The MDLTL compiler is usually able to determine the type of all variables by their use in function calls or arithmetic expressions. In some cases, a pattern or expression can be ambiguous and a type annotation is needed.
A type annotation is denoted by a variable followed by the @ (at) symbol, and the type of a built-in type, like this:
The distiller handles some MDL constructs in a special way, by mapping them to other constructs before applying rules, and then mapping them back afterwards.
Calls to the functions
df::normalized_mixdf::clamped_mixdf::unbounded_mixdf::color_normalized_mixdf::color_clamped_mixdf::color_unbounded_mixare transformed into calls to one of the functions:
bsdf_mix_1, bsdf_mix_2, bsdf_mix_3, bsdf_mix_4bsdf_clamped_mix_1, bsdf_clamped_mix_2, bsdf_clamped_mix_3, bsdf_clamped_mix_4bsdf_unbounded_mix_1, bsdf_unbounded_mix_2, bsdf_unbounded_mix_3, bsdf_unbounded_mix_4bsdf_color_mix_1, bsdf_color_mix_2, bsdf_color_mix_3, bsdf_color_mix_4bsdf_color_clamped_mix_1, bsdf_color_clamped_mix_2, bsdf_color_clamped_mix_3, bsdf_color_clamped_mix_4bsdf_color_unbounded_mix_1, bsdf_color_unbounded_mix_2, bsdf_color_unbounded_mix_3, bsdf_color_unbounded_mix_4edf_mix_1, edf_mix_2, edf_mix_3, edf_mix_4edf_clamped_mix_1, edf_clamped_mix2, edf_clamped_mix_3, edf_clamped_mix_4edf_unbounded_mix_1, edf_unbounded_mix_2, edf_unbounded_mix_3, edf_unbounded_mix_4edf_color_mix_1, edf_color_mix_2, edf_color_mix_3, edf_color_mix_4edf_color_clamped_mix_1, edf_color_clamped_mix_2, edf_color_clamped_mix_3, edf_color_clamped_mix_4edf_color_unbounded_mix_1, edf_color_unbounded_mix_2, edf_color_unbounded_mix_3, edf_color_unbounded_mix_4vdf_mix_1, vdf_mix_2, vdf_mix_3, vdf_mix_4df_clamped_mix_1, vdf_clamped_mix2, vdf_clamped_mix_3, vdf_clamped_mix_4vdf_unbounded_mix_1, vdf_unbounded_mix_2, vdf_unbounded_mix_3, vdf_unbounded_mix_4vdf_color_mix_1, vdf_color_mix_2, vdf_color_mix_3, vdf_color_mix_4vdf_color_clamped_mix_1, vdf_color_clamped_mix_2, vdf_color_clamped_mix_3, vdf_color_clamped_mix_4vdf_color_unbounded_mix_1, vdf_color_unbounded_mix_2, vdf_color_unbounded_mix_3, vdf_color_unbounded_mix_4depending of their number and types of parameters before rule application and back afterwards. That means that rules can only match on, and generate, calls to the transformed functions (..._1, ..._2, ..._3, ..._4). When a mixer call has more than 4 BSDFs, arguments after the fourth are ignored.
Implementation note: The special handling of mixer calls might be subject to change in future versions of the MDL SDK API and may be replaced with a different way of matching on nodes with struct, enum and array parameters.
The mixer call transformation in the rule engine also applies some normalization to the calls. Since the order of parameters for these functions does not matter, the pairs of weights and DFs are sorted by the semantics of the argument DFs. DF semantics are represented by the mi::mdl::IDefinition::Semantics enumeration in the MDL SDK and the calls in mixer calls are ordered according to the the numerical values of the C++ enum variants.
Since the order of arguments for mixer calls must be consistent between the rule engine and the compiled patterns that are applied, the normalization must be implemented both in the compiler and in the rule engine. The MDLTL compiler normalizes patterns before translation into C++, whereas the normalization of the actual DAG calls that is matched against is done in the Distiller at run time.
Mixer normalization in the compiler is switched on using the --normalize-mixers command line option. Mixer call normalization in the Distiller is controlled by a flag in the rule engine. The call Distiller_plugin_api::set_normalize_mixers(bool new_value) can be used by the plugin implementation to switch it on and off before rule application.
Note that mixer normalization can be switched on and off in the compiler (at compile time) and in the Distiller (at run time). Both settings must be consistent, or rules might not match correctly.
There are restrictions on the automatic normalization of mixer calls in the compiler:
--warn=non-normalized-mixers which can warn in that case.Implementation note: Mixer normalization currently only works for mixers with up to 3 BSDFs because of the general restriction on mixers (see above).
Several MDL functions have different names in Distiller rules than their standard names. Additionally, conditional operators on BSDFs are represented as function calls and the Distiller introduces special functions for them.
Some functions from the standard library have special names in MDLTL because overloaded functions and functions with multiple numbers of parameters cannot be distinguished.
These functions from the df module in MDL have special names in the Distiller:
| Standard name | Distiller name | Notes |
|---|---|---|
df::tint | bsdf_tint | 2 parameter version of df::tint |
df::tint | bsdf_tint_ex | 3 parameter version of df::tint |
df::tint | edf_tint | EDF version of df::tint |
df::tint | vdf_tint | VDF version of df::tint |
df::tint | hair_bsdf_tint | hair BSDF version of df::tint |
df::directional_factor | edf_directional_factor | EDF version of df::directional_factor |
The following functions, which do not exist in MDL, are allowed in Distiller rules. They are defined for technical reasons and can be matched against and used to create rule results.
material material_conditional_operator(bool cond, material true_exp, material false_exp)bsdf bsdf_conditional_operator(bool cond, bsdf true_exp, bsdf false_exp)edf edf_conditional_operator(bool cond, edf true_exp, edf false_exp)vdf vdf_conditional_operator(bool cond, vdf true_exp, vdf false_exp)These calls correspond to the ? : conditional operator in MDL for materials and various forms of DFs. They allow transformations on conditional expressions. Other MDL operators can not be matched on.
Implementation note: Handling of overloaded functions and conditional operators may change in future versions of the Distiller.
MDLTL files can import MDL modules in order to make material functions available for creating function calls. For all imported modules in a rule set, all exported functions that return a material or BSDF are added to the set of visible function overloads and can be used for creating replacement expressions. When the imported module is not a standard library module, the imported functions can only be used on the RHS expressions of rules, LHS patterns are restricted to materials from standard modules.
The MDLTL compiler supports the --mdl-path command line option to set search paths for MDL modules.
The C++ code created by the MDLTL compiler is integrated into a Distiller plugin by including the generated header file and linking it with the object file created from the generated .cpp file. The plugin uses the generated classes in its distill() function, which is declared like this:
The function uses the distilling plugin API reference api in order to apply the generated rules as follows:
First, a variable for the result material is created, then the index of the target is checked to see which transformations must be applied. In our example, only one target exists. An instance of the generated class Make_simple_rules is created and the Distiller API function apply_rules() is used to perform the transformation defined by the rules from the original MDLTL file.
A more complex plugin will have multiple rule sets and therefore will declare more instances of matcher classes and apply them one by one.
Error handling is not included in this code for readability, but can bee seen in the full example.
See here for the complete example Distiller plugin from which the above code was extracted. It makes use of the rule set defined at the beginning of this page: Custom Distiller Plugin Example.
The next page MDLTL Grammar documents the formal grammar of mdltl files.