MDL SDK API nvidia_logo_transpbg.gif Up
Example for Compilation of MDL Materials
[Previous] [Up] [Next]

This example introduces compiled materials and highlights differences between different compilation modes.

New Topics

  • Compiled materials
  • Different compilation modes
  • Target code generation

Detailed Description

Compiled materials


A compiled material is a compact, optimized representation of a material instance including all call expressions in its arguments. In this optimization step call expressions for arguments are inlined (similar to function calls in programming languages), constant expressions are folded, common subexpression elimination is taking place, etc.

Different compilation modes


Please refer to Instance-compilation and class-compilation for an introduction into instance and class compilation modes.

In the output of this example, compare the difference between instance and class compilation mode of the initially created material instance. In class compilation mode, the expression for "surface.scattering.tint" is no longer a constant, but a parameter reference.

Now compare the changes in both representations when an argument like "tint" is changed. In instance compilation mode, the constant changes its value accordingly, and consequently, the hash over all temporaries and fields changes, too. In class compilation mode, only the value of argument 0 changes. All temporaries and fields of the compiled material, and consequently the hash value, remain unchanged.

Target code generation


Compiler backends allow to generate code from the abstract representation of compiled materials. There are different backends, e.g., for LLVM IR or CUDA PTX code. These backends can be obtained from mi::neuraylib::IMdl_backend_api::get_backend().

Backends support general and backend-specific options to influence the code generation. See mi::neuraylib::IMdl_backend for details.

Note that target code is not generated for an entire compiled material, but for specific subexpressions of the compiled material, like "surface.scattering.tint".

Example Source

Source Code Location: examples/mdl_sdk/compilation/example_compilation.cpp

/******************************************************************************
* Copyright 2024 NVIDIA Corporation. All rights reserved.
*****************************************************************************/
// examples/mdl_sdk/compilation/example_compilation.cpp
//
// Introduces compiled materials and highlights differences between different compilation modes.
#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>
#include <algorithm>
#include "example_shared.h"
// Command line options structure.
struct Options {
// Materials to use.
std::string material_name
= "::nvidia::sdk_examples::tutorials::example_compilation(color)";
// Expression path to compile.
std::string expr_path = "backface.scattering.tint";
// If true, changes the arguments of the instantiated material.
// Will be set to false if the material name or expression path is changed.
bool change_arguments = true;
};
// Utility function to dump the hash, arguments, temporaries, and fields of a compiled material.
void dump_compiled_material(
std::ostream& s)
{
mdl_factory->create_value_factory( transaction));
mdl_factory->create_expression_factory( transaction));
mi::base::Uuid hash = cm->get_hash();
char buffer[36];
snprintf( buffer, sizeof( buffer),
"%08x %08x %08x %08x", hash.m_id1, hash.m_id2, hash.m_id3, hash.m_id4);
s << " hash overall = " << buffer << std::endl;
snprintf( buffer, sizeof( buffer),
"%08x %08x %08x %08x", hash.m_id1, hash.m_id2, hash.m_id3, hash.m_id4);
s << " hash slot " << std::setw( 2) << i << " = " << buffer << std::endl;
}
mi::Size parameter_count = cm->get_parameter_count();
for( mi::Size i = 0; i < parameter_count; ++i) {
std::stringstream name;
name << i;
value_factory->dump( argument.get(), name.str().c_str(), 1));
s << " argument " << result->get_c_str() << std::endl;
}
mi::Size temporary_count = cm->get_temporary_count();
for( mi::Size i = 0; i < temporary_count; ++i) {
std::stringstream name;
name << i;
expression_factory->dump( temporary.get(), name.str().c_str(), 1));
s << " temporary " << result->get_c_str() << std::endl;
}
mi::base::Handle<const mi::IString> result( expression_factory->dump( body.get(), 0, 1));
s << " body " << result->get_c_str() << std::endl;
s << std::endl;
}
// Creates an instance of the given material definition and stores it in the DB.
void create_material_instance(
const char* material_name,
const char* instance_name)
{
// split module and material name
std::string module_name, material_simple_name;
if (!mi::examples::mdl::parse_cmd_argument_material_name(
material_name, module_name, material_simple_name, true))
exit_failure("Provided material name '%s' is invalid.", material_name);
// Load the module.
mdl_impexp_api->load_module(transaction, module_name.c_str(), context);
if (!print_messages(context))
exit_failure("Loading module '%s' failed.", module_name.c_str());
// Get the database name for the module we loaded
mdl_factory->get_db_module_name(module_name.c_str()));
transaction->access<mi::neuraylib::IModule>(module_db_name->get_c_str()));
if (!module)
exit_failure("Failed to access the loaded module.");
// Construct the material name
std::string material_db_name
= std::string(module_db_name->get_c_str()) + "::" + material_simple_name;
material_db_name = mi::examples::mdl::add_missing_material_signature(
module.get(), material_db_name);
if (material_db_name.empty())
exit_failure("Failed to find the material %s in the module %s.",
material_simple_name.c_str(), module_name.c_str());
// Get the material definition from the database
transaction->access<mi::neuraylib::IFunction_definition>(material_db_name.c_str()));
if (!material_definition)
exit_failure("Accessing definition '%s' failed.", material_db_name.c_str());
// Create a material instance from the material definition with the default arguments.
mi::Sint32 result;
material_definition->create_function_call(0, &result));
if (result != 0)
exit_failure("Instantiating '%s' failed.", material_db_name.c_str());
transaction->store(material_instance.get(), instance_name);
}
// Compiles the given material instance in the given compilation modes, dumps the result, and stores
// it in the DB.
void compile_material_instance(
const char* instance_name,
const char* compiled_material_name,
bool class_compilation)
{
transaction->access<mi::neuraylib::IMaterial_instance>( instance_name));
// convert to target type SID_MATERIAL
tf->get_predefined_struct(mi::neuraylib::IType_struct::SID_MATERIAL));
context->set_option("target_type", standard_material_type.get());
mi::Uint32 flags = class_compilation
material_instance->create_compiled_material( flags, context));
check_success( print_messages( context));
std::cout << "Dumping compiled material (" << ( class_compilation ? "class" : "instance")
<< " compilation) for \"" << instance_name << "\":" << std::endl << std::endl;
dump_compiled_material( transaction, mdl_factory, compiled_material.get(), std::cout);
std::cout << std::endl;
transaction->store( compiled_material.get(), compiled_material_name);
}
// Changes the tint parameter of the given material instance to green.
void change_arguments(
const char* instance_name)
{
mdl_factory->create_value_factory( transaction));
mdl_factory->create_expression_factory( transaction));
// Edit the instance of the material definition "compilation_material".
transaction->edit<mi::neuraylib::IFunction_call>( instance_name));
check_success( material_instance.is_valid_interface());
// Create the new argument for the "tint" parameter from scratch with the new value, and set it.
value_factory->create_color( 0.0f, 1.0f, 0.0f));
expression_factory->create_constant( tint_value.get()));
check_success( material_instance->set_argument( "tint", tint_expr.get()) == 0);
}
// Generates LLVM IR target code for a subexpression of a given compiled material.
void generate_llvm_ir(
const char* compiled_material_name,
const char* path,
const char* fname)
{
transaction->access<mi::neuraylib::ICompiled_material>( compiled_material_name));
check_success(compiled_material.is_valid_interface());
check_success(be_llvm_ir.is_valid_interface());
check_success(be_llvm_ir->set_option( "num_texture_spaces", "16") == 0);
check_success(be_llvm_ir->set_option( "enable_simd", "on") == 0);
be_llvm_ir->translate_material_expression(
transaction, compiled_material.get(), path, fname, context));
check_success(print_messages( context));
check_success(code_llvm_ir);
std::cout << "Dumping LLVM IR code for \"" << path << "\" of \"" << compiled_material_name
<< "\":" << std::endl << std::endl;
std::cout << code_llvm_ir->get_code() << std::endl;
}
// Generates CUDA PTX target code for a subexpression of a given compiled material.
void generate_cuda_ptx(
const char* compiled_material_name,
const char* path,
const char* fname)
{
transaction->access<mi::neuraylib::ICompiled_material>( compiled_material_name));
check_success(compiled_material.is_valid_interface());
check_success(be_cuda_ptx.is_valid_interface());
check_success(be_cuda_ptx->set_option( "num_texture_spaces", "16") == 0);
check_success(be_cuda_ptx->set_option( "sm_version", "50") == 0);
be_cuda_ptx->translate_material_expression(
transaction, compiled_material.get(), path, fname, context));
check_success( print_messages( context));
check_success( code_cuda_ptx);
std::cout << "Dumping CUDA PTX code for \"" << path << "\" of \"" << compiled_material_name
<< "\":" << std::endl << std::endl;
std::cout << code_cuda_ptx->get_code() << std::endl;
}
// Generates HLSL target code for a subexpression of a given compiled material.
void generate_hlsl(
const char* compiled_material_name,
const char* path,
const char* fname)
{
transaction->access<mi::neuraylib::ICompiled_material>( compiled_material_name));
check_success(compiled_material.is_valid_interface());
check_success(be_hlsl.is_valid_interface());
check_success(be_hlsl->set_option( "num_texture_spaces", "1") == 0);
be_hlsl->translate_material_expression(
transaction, compiled_material.get(), path, fname, context));
check_success(print_messages( context));
check_success(code_hlsl);
std::cout << "Dumping HLSL code for \"" << path << "\" of \"" << compiled_material_name
<< "\":" << std::endl << std::endl;
std::cout << code_hlsl->get_code() << std::endl;
}
// Generates GLSL target code for a subexpression of a given compiled material.
void generate_glsl(
const char* compiled_material_name,
const char* path,
const char* fname)
{
transaction->access<mi::neuraylib::ICompiled_material>( compiled_material_name));
check_success(compiled_material.is_valid_interface());
check_success(be_glsl.is_valid_interface());
check_success(be_glsl->set_option( "glsl_version", "450") == 0);
be_glsl->translate_material_expression(
transaction, compiled_material.get(), path, fname, context));
check_success(print_messages( context));
check_success(code_glsl);
std::cout << "Dumping GLSL code for \"" << path << "\" of \"" << compiled_material_name
<< "\":" << std::endl << std::endl;
std::cout << code_glsl->get_code() << std::endl;
}
void usage( char const *prog_name)
{
std::cout
<< "Usage: " << prog_name << " [options] [<material_name>]\n"
<< "Options:\n"
<< " --mdl_path <path> mdl search path, can occur multiple times.\n"
<< " --expr_path expression path to compile, defaults to "
"\"backface.scattering.tint\"."
<< " <material_name> qualified name of materials to use, defaults to\n"
<< " \"::nvidia::sdk_examples::tutorials::example_compilation\"\n"
<< std::endl;
exit_failure();
}
int MAIN_UTF8(int argc, char* argv[])
{
// Parse command line options
Options options;
mi::examples::mdl::Configure_options configure_options;
for (int i = 1; i < argc; ++i) {
char const *opt = argv[i];
if (opt[0] == '-') {
if (strcmp(opt, "--mdl_path") == 0 && i < argc - 1) {
configure_options.additional_mdl_paths.emplace_back(argv[++i]);
}
else if (strcmp(opt, "--expr_path") == 0 && i < argc - 1) {
options.expr_path = argv[++i];
options.change_arguments = false;
}
else {
std::cout << "Unknown option: \"" << opt << "\"" << std::endl;
usage(argv[0]);
}
}
else {
options.material_name = opt;
options.change_arguments = false;
}
}
// Access the MDL SDK
mi::base::Handle<mi::neuraylib::INeuray> neuray(mi::examples::mdl::load_and_get_ineuray());
if (!neuray.is_valid_interface())
exit_failure("Failed to load the SDK.");
// Configure the MDL SDK
if (!mi::examples::mdl::configure(neuray.get(), configure_options))
exit_failure("Failed to initialize the SDK.");
// Start the MDL SDK
mi::Sint32 ret = neuray->start();
if (ret != 0)
exit_failure("Failed to initialize the SDK. Result code: %d", ret);
{
neuray->get_api_component<mi::neuraylib::IMdl_factory>());
neuray->get_api_component<mi::neuraylib::IDatabase>());
mi::base::Handle<mi::neuraylib::IScope> scope(database->get_global_scope());
mi::base::Handle<mi::neuraylib::ITransaction> transaction(scope->create_transaction());
{
// Create an execution context for options and error message handling
mdl_factory->create_execution_context());
// Create MDL import-export API for importing MDL modules
neuray->get_api_component<mi::neuraylib::IMdl_impexp_api>());
// Load the "example" module and create a material instance
std::string instance_name = "instance of compilation_material";
create_material_instance(
mdl_factory.get(),
transaction.get(),
mdl_impexp_api.get(),
context.get(),
options.material_name.c_str(),
instance_name.c_str());
// Compile the material instance in instance compilation mode
std::string instance_compilation_name
= std::string("instance compilation of ") + instance_name;
compile_material_instance(
transaction.get(), mdl_factory.get(), context.get(), instance_name.c_str(),
instance_compilation_name.c_str(), false);
// Compile the material instance in class compilation mode
std::string class_compilation_name
= std::string("class compilation of ") + instance_name;
compile_material_instance(
transaction.get(), mdl_factory.get(), context.get(), instance_name.c_str(),
class_compilation_name.c_str(), true);
// Change some material argument and compile again in both modes. Note the differences
// in instance compilation mode, whereas only the referenced parameter itself changes in
// class compilation mode.
if (options.change_arguments) {
change_arguments(transaction.get(), mdl_factory.get(), instance_name.c_str());
compile_material_instance(
transaction.get(), mdl_factory.get(), context.get(), instance_name.c_str(),
instance_compilation_name.c_str(), false);
compile_material_instance(
transaction.get(), mdl_factory.get(), context.get(), instance_name.c_str(),
class_compilation_name.c_str(), true);
}
// Use the various backends to generate target code for some material expression.
neuray->get_api_component<mi::neuraylib::IMdl_backend_api>());
generate_llvm_ir(
transaction.get(), mdl_backend_api.get(), context.get(),
instance_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_llvm_ir(
transaction.get(), mdl_backend_api.get(), context.get(),
class_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_cuda_ptx(
transaction.get(), mdl_backend_api.get(), context.get(),
instance_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_cuda_ptx(
transaction.get(), mdl_backend_api.get(), context.get(),
class_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_glsl(
transaction.get(), mdl_backend_api.get(), context.get(),
instance_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_glsl(
transaction.get(), mdl_backend_api.get(), context.get(),
class_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_hlsl(
transaction.get(), mdl_backend_api.get(), context.get(),
instance_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
generate_hlsl(
transaction.get(), mdl_backend_api.get(), context.get(),
class_compilation_name.c_str(),
options.expr_path.c_str(), "tint");
}
transaction->commit();
}
// Shut down the MDL SDK
if (neuray->shutdown() != 0)
exit_failure("Failed to shutdown the SDK.");
// Unload the MDL SDK
neuray = nullptr;
if (!mi::examples::mdl::unload())
exit_failure("Failed to unload the SDK.");
exit_success();
}
// Convert command line arguments to UTF8 on Windows
COMMANDLINE_TO_UTF8
This interface represents a compiled material.
Definition: icompiled_material.h:97
virtual Size get_temporary_count() const =0
Returns the number of temporaries.
virtual const IValue * get_argument(Size index) const =0
Returns the value of an argument.
virtual Size get_parameter_count() const =0
Returns the number of parameters used by this compiled material.
virtual const IExpression_direct_call * get_body() const =0
Returns the body (or material root) of the compiled material.
virtual base::Uuid get_hash() const =0
Returns a hash of the body and all temporaries.
virtual base::Uuid get_slot_hash(Material_slot slot) const =0
Returns the hash of a particular material slot.
virtual const IExpression * get_temporary(Size index) const =0
Returns a temporary.
This interface is used to interact with the distributed database.
Definition: idatabase.h:289
This interface represents a function call.
Definition: ifunction_call.h:52
This interface represents a function definition.
Definition: ifunction_definition.h:44
This interface represents a material instance.
Definition: imaterial_instance.h:34
@ CLASS_COMPILATION
Selects class compilation instead of instance compilation.
Definition: imaterial_instance.h:41
@ DEFAULT_OPTIONS
Default compilation options (e.g., instance compilation).
Definition: imaterial_instance.h:40
This interface can be used to obtain the MDL backends.
Definition: imdl_backend_api.h:56
@ MB_CUDA_PTX
Generate CUDA PTX code.
Definition: imdl_backend_api.h:61
@ MB_LLVM_IR
Generate LLVM IR (LLVM 12.0 compatible).
Definition: imdl_backend_api.h:62
@ MB_GLSL
Generate GLSL code.
Definition: imdl_backend_api.h:63
@ MB_HLSL
Generate HLSL code.
Definition: imdl_backend_api.h:65
virtual IMdl_backend * get_backend(Mdl_backend_kind kind)=0
Returns an MDL backend generator.
The execution context can be used to query status information like error and warning messages concern...
Definition: imdl_execution_context.h:131
virtual Sint32 set_option(const char *name, const char *value)=0
Sets a string option.
Factory for various MDL interfaces and functions.
Definition: imdl_factory.h:53
virtual IMdl_execution_context * create_execution_context()=0
Creates an execution context.
virtual IType_factory * create_type_factory(ITransaction *transaction)=0
Returns an MDL type factory for the given transaction.
virtual IExpression_factory * create_expression_factory(ITransaction *transaction)=0
Returns an MDL expression factory for the given transaction.
virtual const IString * get_db_module_name(const char *mdl_name)=0
Returns the DB name for the MDL name of a module (or file path for MDLE modules).
virtual IValue_factory * create_value_factory(ITransaction *transaction)=0
Returns an MDL value factory for the given transaction.
API component for MDL related import and export operations.
Definition: imdl_impexp_api.h:43
virtual Sint32 load_module(ITransaction *transaction, const char *argument, IMdl_execution_context *context=0)=0
Loads an MDL module from disk (or a builtin module) into the database.
This interface represents an MDL module.
Definition: imodule.h:634
A transaction provides a consistent view on the database.
Definition: itransaction.h:82
virtual const base::IInterface * access(const char *name)=0
Retrieves an element from the database.
virtual base::IInterface * edit(const char *name)=0
Retrieves an element from the database and returns it ready for editing.
virtual Sint32 commit()=0
Commits the transaction.
virtual Sint32 store(base::IInterface *db_element, const char *name, Uint8 privacy=LOCAL_SCOPE)=0
Stores the element db_element in the database under the name name and with the privacy level privacy.
@ SID_MATERIAL
The "::material" struct type.
Definition: itype.h:484
Uint32 m_id1
First value.
Definition: uuid.h:27
Uint32 m_id2
Second value.
Definition: uuid.h:28
Uint32 m_id3
Third value.
Definition: uuid.h:29
Uint32 m_id4
Fourth value.
Definition: uuid.h:30
unsigned int Uint32
32-bit unsigned integer.
Definition: types.h:49
Uint64 Size
Unsigned integral type that is large enough to hold the size of all types.
Definition: types.h:112
signed int Sint32
32-bit signed integer.
Definition: types.h:46
Material_slot
Material slots identify parts of a compiled material.
Definition: icompiled_material.h:26
@ SLOT_FIRST
First slot.
Definition: icompiled_material.h:45
@ SLOT_LAST
Last slot.
Definition: icompiled_material.h:46
A 128 bit representation of a universally unique identifier (UUID or GUID).
Definition: uuid.h:26
[Previous] [Up] [Next]