MDL SDK API nvidia_logo_transpbg.gif Up
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Groups Pages
Example for Execution of Compiled MDL Materials (GLSL)
[Previous] [Up] [Next]

This example describes the API of the code generated by the "GLSL" backend for compiled materials and shows how a renderer can call this generated code to evaluate sub-expressions of multiple materials using OpenGL.

New Topics

  • MDL material state (GLSL)
  • Execution of generated code (GLSL)
  • Constant data (GLSL)
  • Loading textures (GLSL)
  • Texture access functions (GLSL)

Detailed Description

MDL material state (GLSL)

The MDL material state is a representation of the renderer state as defined in section 19 "Renderer state" in the MDL specification. It is used to make the state of the renderer (like the position of an intersection point on the surface, the shading normal and the texture coordinates) available to the generated code.

With the GLSL backend, you can define the MDL material state in a pretty flexible way: For each supported state member you can specify whether you want to implement it as a field in the "State" structure ("field" mode), as an input variable of the fragment shader ("arg" mode), as a function ("func" mode), or as constant zero ("zero" mode). Please refer to the documentation of mi::neuraylib::IMdl_backend::set_option() on how to set these modes via the "glsl_state_*" options.

The code snippet below shows how the "State" structure would look like, if all state options were set to "field" mode and "num_textures_spaces" was set to 1.

// The MDL material state structure used by the MDL SDK when all state accessor functions
// are set to "field" mode and "num_texture_spaces" is set to 1.
struct State {
vec3 normal; // state::normal() result
vec3 geometry_normal; // state::geom_normal() result
vec3 position; // state::position() result
float animation_time; // state::animation_time() result
vec3 motion; // state::motion() result
int texture_space_max; // state::texture_space_max() result
vec3[1] texture_coordinate; // state::texture_coordinate() table
vec3[1] texture_tangent_u; // state::texture_tangent_u() table
vec3[1] texture_tangent_v; // state::texture_tangent_v() table
vec3[1] geometry_tangent_u; // state::texture_geometry_tangent_u() table
vec3[1] geometry_tangent_v; // state::texture_geometry_tangent_v() table
int object_id; // state::object_id() result

If all state options were set to "arg" mode and "num_textures_spaces" was set to 1, the "state" input variables would look like this:

in vec3 normal; // state::normal() result
in vec3 geometry_normal; // state::geom_normal() result
in vec3 position; // state::position() result
in float animation_time; // state::animation_time() result
in vec3 motion; // state::motion() result
in int texture_space_max; // state::texture_space_max() result
in vec3[1] texture_coordinate; // state::texture_coordinate() table
in vec3[1] texture_tangent_u; // state::texture_tangent_u() table
in vec3[1] texture_tangent_v; // state::texture_tangent_v() table
in vec3[1] geometry_tangent_u; // state::texture_geometry_tangent_u() table
in vec3[1] geometry_tangent_v; // state::texture_geometry_tangent_v() table
in int object_id; // state::object_id() result

If all state options were set to "func" mode and "num_textures_spaces" was set to 1, the "state" function prototypes would look like this:

vec3 normal(void); // state::normal() implementation
vec3 geometry_normal(void); // state::geom_normal() implementation
vec3 position(void); // state::position() implementation
float animation_time(void); // state::animation_time() implementation
vec3 motion(void); // state::motion() implementation
int texture_space_max(void); // state::texture_space_max() implementation
vec3[1] texture_coordinate(int index); // state::texture_coordinate() implementation
vec3[1] texture_tangent_u(int index); // state::texture_tangent_u() implementation
vec3[1] texture_tangent_v(int index); // state::texture_tangent_v() implementation
vec3[1] geometry_tangent_u(int index); // state::texture_geometry_tangent_u() implementation
vec3[1] geometry_tangent_v(int index); // state::texture_geometry_tangent_v() implementation
int object_id(void); // state::object_id() implementation

Please refer to the documentation of the mi::neuraylib::Shading_state_material structure or the MDL specification for more information about the MDL material state.

The world-to-object and object-to-world matrices can currently not be set for the GLSL backend if a link unit (see below) is used. Thus they will default to the identity matrix.

In this example, we fill the material state structure with some example values and only use one texture space. For state::position() we will use "func" mode with a function which will mirror the position received from the vertex shader across the center. For state::texture_coordinate(int) we will use "arg" mode to use the texture coordinates from the vertex shader directly. For the normals, animation time and the texture tangents we will use "field" mode to make them part of the "State" structure (for the normals, this is the default mode). We will animate the materials by updating the animation_time with every frame.

The scene consists of a 2x2 quad around the center of the world with position x and y coordinates ranging from -1 to 1 and the texture uv-coordinates ranging from 0 to 1, respectively.

Execution of generated code (GLSL)

For the non-native backends, the generated code has to be called directly from the corresponding framework, so we need to know the prototypes of the functions generated by the backend via mi::neuraylib::IMdl_backend::translate_material_expression(). With "NAME" being the function name you provided as fname parameter and "T" being the result type, they simply look like this:

T NAME(State state);

If you need multiple sub-expressions per GLSL program, you should create an mi::neuraylib::ILink_unit via mi::neuraylib::IMdl_backend::create_link_unit() to prevent the backend from generating illegal duplicate functions and globals. With the link unit, you can then add material subexpressions or environment functions by calling mi::neuraylib::ILink_unit::add_material_expression() and mi::neuraylib::ILink_unit::add_environment(), respectively. Finally, you can create the GLSL target code with mi::neuraylib::IMdl_backend::translate_link_unit().

You may not change the backend options between the creation and the destruction of a link unit due to the way the option values are currently stored.

To make the generated functions available by index in the GLSL fragment shader, we generate a switch function "mdl_mat_subexpr()" which calls the function corresponding to the given index (see "generate_glsl_switch_func()").

In this example, we render multiple materials onto the quad with a user-configurable checkerboard pattern by drawing the triangles of the quad with OpenGL and letting the fragment shader call our generated switch function with a material index and an MDL material state.

Constant data (GLSL)

By default, the GLSL backend creates one uniform per constant data object larger than 1024 bytes. Each uniform has to be filled by the renderer via the correct glUniform* functions with the data provided by mi::neuraylib::ITarget_code::get_ro_data_segment_data().

The amount of constant data you can provide within the GLSL code is very limited. If your GLSL code or the code generated by the GLSL backend uses to much constant data, you may get errors about too many registers being used or invalid instructions in the resulting ARB assembly code. While it is possible to force constant data to be placed in uniforms only, the amount of data is still very limited and often already causes problems with 5 kB of data scattered over multiple uniforms.

To avoid these problems, it is highly recommended to enable using Shader Storage Buffer Objects (SSBO) for the generated code by setting the backend option "glsl_place_uniforms_into_ssbo" to "on". There, the main limitation is usually only the available GPU memory.

SSBOs are not available on Mac OS X up to at least version 10.9.

Additionally we recommend to set "glsl_max_const_data" to zero or a very low value to avoid running into above problems when you already use some constant data in your own code.

When SSBOs are enabled, the GLSL backend will place the data of all uniforms into one SSBO. You can access the data for the SSBO via mi::neuraylib::ITarget_code::get_ro_data_segment_data(). In this example, the generated SSBO will look like this:

layout(std430) buffer mdl_buffer_0 {
float[16] mdl_field_1;
int[256] mdl_field_2;
int[256] mdl_field_3;
int[256] mdl_field_4;
int[256] mdl_field_5;
int[256] mdl_field_6;
vec4[128] mdl_field_7;

Please refer to the "Material_opengl_context::set_mdl_readonly_data()" method on how to set the uniform data or the SSBO data in OpenGL.

Loading textures (GLSL)

When the nv_freeimage plugin has been loaded via mi::neuraylib::IPlugin_configuration::load_plugin_library() before starting the MDL SDK, the SDK will automatically load textures on the host side for many common image formats and make them available via mi::neuraylib::ITarget_code::get_texture(). Note, that the first texture is always the invalid texture, so only if there is more than just one texture according to mi::neuraylib::ITarget_code::get_texture_count(), there will be real referenced textures available.

Here's a small code snippet showing how to access the mi::neuraylib::ICanvas of the texture at index i.

The textures still have to be copied to the GPU and possibly they have to be gamma corrected and converted to a format understood by the texture access functions you provide. In this example, we use the mi::neuraylib::IImage_api to apply the gamma correction and to convert the image format to a float32 RGBA format.

The textures are then made available via OpenGL 2D texture samplers.

Currently, the GLSL backend only supports 2D textures.
Texture access functions (GLSL)

For the GLSL backend, the generated code currently expects two texture access functions to be provided by the fragment shader:

vec4 tex_lookup_2d(uint tex, vec2 coord, int wrap_u, int wrap_v, vec2 crop_u, vec2 crop_v);
vec4 tex_texel_2d(uint tex, ivec2 coord);

These correspond directly to the functions described in section 20.3 "Standard library functions - Texture" in the MDL specification.

The "tex" parameter represents the texture index as used by mi::neuraylib::ITarget_code::get_texture().

The tex_lookup_2d() function receives floating-point texture coordinates and should return a sampled value, whereas tex_texel_2d() receives integer texture coordinates and should return a raw texture value.

Example Source

To compile the source code, you need GLEW available at and GLFW available at Please refer to the Getting Started section for details.

Source Code Location: examples/mdl_sdk/execution_glsl/example_execution_glsl.cpp

* Copyright 2021 NVIDIA Corporation. All rights reserved.
// examples/mdl_sdk/execution_glsl/example_execution_glsl.cpp
// Introduces the execution of generated code for compiled materials for
// the GLSL backend and shows how to manually bake a material
// sub-expression to a texture.
#include <iomanip>
#include <iostream>
#include <string>
#include <vector>
#include "example_shared.h"
#include "example_glsl_shared.h"
#include <GL/glew.h>
#include <GLFW/glfw3.h>
// This selects SSBO (Shader Storage Buffer Objects) mode for passing uniforms and MDL const data.
// Should not be disabled unless you only use materials with very small const data.
// In this example, this would only apply to execution_material_2, because the others are using
// lookup tables for noise functions.
#if defined(MI_PLATFORM_MACOSX) || defined(MI_ARCH_ARM_64)
#define MAX_MATERIALS 16
#define MAX_TEXTURES 16
#define USE_SSBO
#define MAX_MATERIALS 64
#define MAX_TEXTURES 32
// If defined, the GLSL backend will remap these functions
// float ::base::perlin_noise(float4 pos)
// float ::base::mi_noise(float3 pos)
// float ::base::mi_noise(int3 pos)
// ::base::worley_return ::base::worley_noise(float3 pos, float jitter, int metric)
// to lut-free alternatives. When enabled, you can avoid to set the USE_SSBO define for this
// example.
// Enable this to dump the generated GLSL code to stdout.
//#define DUMP_GLSL
char const* vertex_shader_filename = "example_execution_glsl.vert";
char const* fragment_shader_filename = "example_execution_glsl.frag";
// Command line options structure.
struct Options {
// If true, no interactive display will be used.
bool no_window;
// An result output file name for non-interactive mode.
std::string outputfile;
// The pattern number representing the combination of materials to display.
unsigned material_pattern;
// The resolution of the display / image.
unsigned res_x, res_y;
// The constructor.
: no_window(false)
, outputfile("output.png")
, material_pattern(7)
, res_x(1024)
, res_y(768)
// Struct representing a vertex of a scene object.
struct Vertex {
// OpenGL code
// Initialize OpenGL and create a window with an associated OpenGL context.
static GLFWwindow *init_opengl(Options const &options)
// Initialize GLFW
#ifdef USE_SSBO
// SSBO requires GLSL 4.30
// else GLSL 3.30 is sufficient
// Hide window in no-window mode
if (options.no_window)
// Create an OpenGL window and a context
GLFWwindow *window = glfwCreateWindow(
options.res_x, options.res_y,
"MDL SDK GLSL Execution Example - Switch pattern with keys 1 - 7", nullptr, nullptr);
if (!window) {
std::cerr << "Error creating OpenGL window!" << std::endl;
// Attach context to window
// Initialize GLEW to get OpenGL extensions
GLenum res = glewInit();
if (res != GLEW_OK) {
std::cerr << "GLEW error: " << glewGetErrorString(res) << std::endl;
// Enable VSync
return window;
// Generate GLSL source code for a function executing an MDL subexpression function
// selected by a given id.
static std::string generate_glsl_switch_func(
// Note: The "State" struct must be in sync with the struct in example_execution_glsl.frag and
// the code generated by the MDL SDK (see dumped code when enabling DUMP_GLSL).
std::string src =
"#version 330 core\n"
"struct State {\n"
" vec3 normal;\n"
" vec3 geometry_normal;\n"
" float animation_time;\n"
" vec3[1] texture_tangent_u;\n"
" vec3[1] texture_tangent_v;\n"
"uint get_mdl_num_mat_subexprs() { return " +
to_string(target_code->get_callable_function_count()) +
"u; }\n"
std::string switch_func =
"vec3 mdl_mat_subexpr(uint id, State state) {\n"
" switch(id) {\n";
// Create one switch case for each callable function in the target code
for (size_t i = 0, num_target_codes = target_code->get_callable_function_count();
i < num_target_codes;
std::string func_name(target_code->get_callable_function(i));
// Add prototype declaration
src += target_code->get_callable_function_prototype(
i, mi::neuraylib::ITarget_code::SL_GLSL);
src += '\n';
switch_func += " case " + to_string(i) + "u: return " + func_name + "(state);\n";
switch_func +=
" default: return vec3(0);\n"
" }\n"
return src + "\n" + switch_func;
// Create the shader program with a fragment shader.
static GLuint create_shader_program(
GLint success;
GLuint program = glCreateProgram();
mi::examples::io::get_executable_folder() + "/" + vertex_shader_filename), program);
std::stringstream sstr;
sstr << "#version 330 core\n";
sstr << "#define MAX_MATERIALS " << to_string(MAX_MATERIALS) << "\n";
sstr << "#define MAX_TEXTURES " << to_string(MAX_TEXTURES) << "\n";
sstr << read_text_file(
mi::examples::io::get_executable_folder() + "/" + fragment_shader_filename);
add_shader(GL_FRAGMENT_SHADER, sstr.str() , program);
std::string code(target_code->get_code());
mi::examples::io::get_executable_folder() + "/" + "noise_no_lut.glsl"));
add_shader(GL_FRAGMENT_SHADER, code, program);
// Generate GLSL switch function for the generated functions
std::string glsl_switch_func = generate_glsl_switch_func(target_code);
#ifdef DUMP_GLSL
std::cout << "Dumping GLSL code for the \"mdl_mat_subexpr\" switch function:\n\n"
<< glsl_switch_func << std::endl;
add_shader(GL_FRAGMENT_SHADER, glsl_switch_func.c_str(), program);
glGetProgramiv(program, GL_LINK_STATUS, &success);
if (!success) {
dump_program_info(program, "Error linking the shader program: ");
return program;
// Create a quad filling the whole screen.
static GLuint create_quad(GLuint program, GLuint* vertex_buffer)
static Vertex const vertices[6] = {
{ { -1.f, -1.f, 0.0f }, { 0.f, 0.f } },
{ { 1.f, -1.f, 0.0f }, { 1.f, 0.f } },
{ { -1.f, 1.f, 0.0f }, { 0.f, 1.f } },
{ { 1.f, -1.f, 0.0f }, { 1.f, 0.f } },
{ { 1.f, 1.f, 0.0f }, { 1.f, 1.f } },
{ { -1.f, 1.f, 0.0f }, { 0.f, 1.f } }
glGenBuffers(1, vertex_buffer);
glBindBuffer(GL_ARRAY_BUFFER, *vertex_buffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW);
GLuint vertex_array;
glGenVertexArrays(1, &vertex_array);
// Get locations of vertex shader inputs
GLint pos_index = glGetAttribLocation(program, "Position");
GLint tex_coord_index = glGetAttribLocation(program, "TexCoord");
check_success(pos_index >= 0 && tex_coord_index >= 0);
pos_index, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), 0);
tex_coord_index, 2, GL_FLOAT, GL_FALSE,
sizeof(Vertex), reinterpret_cast<const GLvoid*>(sizeof(mi::Float32_3_struct)));
return vertex_array;
// Material_opengl_context class
// Helper class responsible for making textures and read-only data available to OpenGL
// by generating and managing a list of Material_data objects.
class Material_opengl_context
Material_opengl_context(GLuint program)
: m_program(program)
, m_next_storage_block_binding(0)
// Free all acquired resources.
// Prepare the needed material data of the given target code.
bool prepare_material_data(
// Sets all collected material data in the OpenGL program.
bool set_material_data();
// Sets the read-only data segments in the current OpenGL program object.
void set_mdl_readonly_data(mi::base::Handle<const mi::neuraylib::ITarget_code> target_code);
// Prepare the texture identified by the texture_index for use by the texture access functions
// in the OpenGL program.
bool prepare_texture(
mi::Size texture_index,
GLuint texture_array);
// The OpenGL program to prepare.
GLuint m_program;
std::vector<GLuint> m_texture_objects;
std::vector<unsigned int> m_material_texture_starts;
std::vector<GLuint> m_buffer_objects;
GLuint m_next_storage_block_binding;
// Free all acquired resources.
if (m_buffer_objects.size() > 0)
glDeleteBuffers(GLsizei(m_buffer_objects.size()), &m_buffer_objects[0]);
if (m_texture_objects.size() > 0)
glDeleteTextures(GLsizei(m_texture_objects.size()), &m_texture_objects[0]);
// Sets the read-only data segments in the current OpenGL program object.
void Material_opengl_context::set_mdl_readonly_data(
mi::Size num_uniforms = target_code->get_ro_data_segment_count();
if (num_uniforms == 0) return;
#ifdef USE_SSBO
size_t cur_buffer_offs = m_buffer_objects.size();
m_buffer_objects.insert(m_buffer_objects.end(), num_uniforms, 0);
glGenBuffers(GLsizei(num_uniforms), &m_buffer_objects[cur_buffer_offs]);
for (mi::Size i = 0; i < num_uniforms; ++i) {
mi::Size segment_size = target_code->get_ro_data_segment_size(i);
char const* segment_data = target_code->get_ro_data_segment_data(i);
#ifdef DUMP_GLSL
std::cout << "Dump ro segment data " << i << " \""
<< target_code->get_ro_data_segment_name(i) << "\" (size = "
<< segment_size << "):\n" << std::hex;
for (int j = 0; j < 16 && j < segment_size; ++j) {
std::cout << "0x" << (unsigned int)(unsigned char)segment_data[j] << ", ";
std::cout << std::dec << std::endl;
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_buffer_objects[cur_buffer_offs + i]);
GL_SHADER_STORAGE_BUFFER, GLsizeiptr(segment_size), segment_data, GL_STATIC_DRAW);
GLuint block_index = glGetProgramResourceIndex(
m_program, GL_SHADER_STORAGE_BLOCK, target_code->get_ro_data_segment_name(i));
glShaderStorageBlockBinding(m_program, block_index, m_next_storage_block_binding);
m_buffer_objects[cur_buffer_offs + i]);
std::vector<char const*> uniform_names;
for (mi::Size i = 0; i < num_uniforms; ++i) {
#ifdef DUMP_GLSL
mi::Size segment_size = target_code->get_ro_data_segment_size(i);
const char* segment_data = target_code->get_ro_data_segment_data(i);
std::cout << "Dump ro segment data " << i << " \""
<< target_code->get_ro_data_segment_name(i) << "\" (size = "
<< segment_size << "):\n" << std::hex;
for (int i = 0; i < 16 && i < segment_size; ++i) {
std::cout << "0x" << (unsigned int)(unsigned char)segment_data[i] << ", ";
std::cout << std::dec << std::endl;
std::vector<GLuint> uniform_indices(num_uniforms, 0);
glGetUniformIndices(m_program, GLsizei(num_uniforms), &uniform_names[0], &uniform_indices[0]);
for (mi::Size i = 0; i < num_uniforms; ++i) {
// uniforms may have been removed, if they were not used
if (uniform_indices[i] == GL_INVALID_INDEX)
GLint uniform_type = 0;
GLuint index = GLuint(uniform_indices[i]);
glGetActiveUniformsiv(m_program, 1, &index, GL_UNIFORM_TYPE, &uniform_type);
#ifdef DUMP_GLSL
std::cout << "Uniform type of " << uniform_names[i]
<< ": 0x" << std::hex << uniform_type << std::dec << std::endl;
mi::Size segment_size = target_code->get_ro_data_segment_size(i);
const char* segment_data = target_code->get_ro_data_segment_data(i);
GLint uniform_location = glGetUniformLocation(m_program, uniform_names[i]);
switch (uniform_type) {
// For bool, the data has to be converted to int, first
#define CASE_TYPE_BOOL(type, func, num) \
case type: { \
GLint *buf = new GLint[segment_size]; \
for (mi::Size j = 0; j < segment_size; ++j) \
buf[j] = GLint(segment_data[j]); \
func(uniform_location, GLsizei(segment_size / num), buf); \
delete[] buf; \
break; \
CASE_TYPE_BOOL(GL_BOOL, glUniform1iv, 1)
CASE_TYPE_BOOL(GL_BOOL_VEC2, glUniform2iv, 2)
CASE_TYPE_BOOL(GL_BOOL_VEC3, glUniform3iv, 3)
CASE_TYPE_BOOL(GL_BOOL_VEC4, glUniform4iv, 4)
#define CASE_TYPE(type, func, num, elemtype) \
case type: \
func(uniform_location, GLsizei(segment_size/(num * sizeof(elemtype))), \
(const elemtype*)segment_data); \
CASE_TYPE(GL_INT, glUniform1iv, 1, GLint);
CASE_TYPE(GL_INT_VEC2, glUniform2iv, 2, GLint);
CASE_TYPE(GL_INT_VEC3, glUniform3iv, 3, GLint);
CASE_TYPE(GL_INT_VEC4, glUniform4iv, 4, GLint);
CASE_TYPE(GL_FLOAT, glUniform1fv, 1, GLfloat);
CASE_TYPE(GL_FLOAT_VEC2, glUniform2fv, 2, GLfloat);
CASE_TYPE(GL_FLOAT_VEC3, glUniform3fv, 3, GLfloat);
CASE_TYPE(GL_FLOAT_VEC4, glUniform4fv, 4, GLfloat);
CASE_TYPE(GL_DOUBLE, glUniform1dv, 1, GLdouble);
CASE_TYPE(GL_DOUBLE_VEC2, glUniform2dv, 2, GLdouble);
CASE_TYPE(GL_DOUBLE_VEC3, glUniform3dv, 3, GLdouble);
CASE_TYPE(GL_DOUBLE_VEC4, glUniform4dv, 4, GLdouble);
#define CASE_TYPE_MAT(type, func, num, elemtype) \
case type: \
func(uniform_location, GLsizei(segment_size/(num * sizeof(elemtype))), \
false, (const elemtype*)segment_data); \
CASE_TYPE_MAT(GL_FLOAT_MAT2_ARB, glUniformMatrix2fv, 4, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT2x3, glUniformMatrix2x3fv, 6, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT3x2, glUniformMatrix3x2fv, 6, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT2x4, glUniformMatrix2x4fv, 8, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT4x2, glUniformMatrix4x2fv, 8, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT3_ARB, glUniformMatrix3fv, 9, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT3x4, glUniformMatrix3x4fv, 12, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT4x3, glUniformMatrix4x3fv, 12, GLfloat);
CASE_TYPE_MAT(GL_FLOAT_MAT4_ARB, glUniformMatrix4fv, 16, GLfloat);
CASE_TYPE_MAT(GL_DOUBLE_MAT2, glUniformMatrix2dv, 4, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT2x3, glUniformMatrix2x3dv, 6, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT3x2, glUniformMatrix3x2dv, 6, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT2x4, glUniformMatrix2x4dv, 8, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT4x2, glUniformMatrix4x2dv, 8, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT3, glUniformMatrix3dv, 9, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT3x4, glUniformMatrix3x4dv, 12, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT4x3, glUniformMatrix4x3dv, 12, GLdouble);
CASE_TYPE_MAT(GL_DOUBLE_MAT4, glUniformMatrix4dv, 16, GLdouble);
std::cerr << "Unsupported uniform type: 0x"
<< std::hex << uniform_type << std::dec << std::endl;
// Prepare the texture identified by the texture_index for use by the texture access functions
// on the GPU.
bool Material_opengl_context::prepare_texture(
mi::Size texture_index,
GLuint texture_obj)
// Get access to the texture data by the texture database name from the target code.
mi::Uint32 tex_width = canvas->get_resolution_x();
mi::Uint32 tex_height = canvas->get_resolution_y();
mi::Uint32 tex_layers = canvas->get_layers_size();
char const *image_type = image->get_type();
if (canvas->get_tiles_size_x() != 1 || canvas->get_tiles_size_y() != 1) {
std::cerr << "The example does not support tiled images!" << std::endl;
return false;
if (tex_layers != 1) {
std::cerr << "The example and the GLSL backend don't support layered images!" << std::endl;
return false;
// For simplicity, the texture access functions are only implemented for float4 and gamma
// is pre-applied here (all images are converted to linear space).
// Convert to linear color space if necessary
if (texture->get_effective_gamma() != 1.0f) {
// Copy/convert to float4 canvas and adjust gamma from "effective gamma" to 1.
image_api->convert(canvas.get(), "Color"));
image_api->adjust_gamma(gamma_canvas.get(), 1.0f);
canvas = gamma_canvas;
} else if (strcmp(image_type, "Color") != 0 && strcmp(image_type, "Float32<4>") != 0) {
// Convert to expected format
canvas = image_api->convert(canvas.get(), "Color");
// This example supports only 2D textures
mi::Float32 const *data = static_cast<mi::Float32 const *>(tile->get_data());
glBindTexture(GL_TEXTURE_2D, texture_obj);
GL_TEXTURE_2D, 0, GL_RGBA, tex_width, tex_height, 0, GL_RGBA, GL_FLOAT, data);
return true;
// Prepare the needed material data of the given target code.
bool Material_opengl_context::prepare_material_data(
// Handle the read-only data segments if necessary
// Handle the textures if there are more than just the invalid texture
size_t cur_tex_offs = m_texture_objects.size();
mi::Size num_textures = target_code->get_texture_count();
if (num_textures > 1) {
m_texture_objects.insert(m_texture_objects.end(), num_textures - 1, 0);
glGenTextures(GLsizei(num_textures - 1), &m_texture_objects[cur_tex_offs]);
// Loop over all textures skipping the first texture,
// which is always the MDL invalid texture
for (mi::Size i = 1; i < num_textures; ++i) {
if (!prepare_texture(
transaction, image_api, target_code,
i, m_texture_objects[cur_tex_offs + i - 1]))
return false;
return true;
// Sets all collected material data in the OpenGL program.
bool Material_opengl_context::set_material_data()
GLsizei total_textures = GLsizei(m_texture_objects.size());
if(total_textures > MAX_TEXTURES)
fprintf( stderr, "Number of required textures (%d) is not supported (max: %d)\n",
total_textures, MAX_TEXTURES);
return false;
#ifdef USE_SSBO
if (glfwExtensionSupported("GL_ARB_bindless_texture"))
if (total_textures > 0) {
std::vector<GLuint64> texture_handles;
for (GLsizei i = 0; i < total_textures; ++i) {
texture_handles[i] = glGetTextureHandleARB(m_texture_objects[i]);
glGetUniformLocation(m_program, "material_texture_samplers_2d"),
glGetUniformLocation(m_program, "material_texture_starts"),
else if (glfwExtensionSupported("GL_NV_bindless_texture"))
if (total_textures > 0) {
std::vector<GLuint64> texture_handles;
for (GLsizei i = 0; i < total_textures; ++i) {
texture_handles[i] = glGetTextureHandleNV(m_texture_objects[i]);
glGetUniformLocation(m_program, "material_texture_samplers_2d"),
glGetUniformLocation(m_program, "material_texture_starts"),
fprintf(stderr, "Sample requires Bindless Textures, "
"that are not supported by the current system.\n");
return false;
#endif // USE_SSBO
// Check for any errors. If you get an error, check whether MAX_TEXTURES and MAX_MATERIALS
// in example_execution_glsl.frag still fit to your needs.
return glGetError() == GL_NO_ERROR;
// MDL material compilation code
class Material_compiler {
// Constructor.
// Generates GLSL target code for a subexpression of a given material.
// path is the path of the sub-expression.
// fname is the function name in the generated code.
bool add_material_subexpr(
const std::string& qualified_module_name,
const std::string& material_db_name,
const char* path,
const char* fname);
// Generates GLSL target code for a subexpression of a given compiled material.
// Creates an instance of the given material.
mi::neuraylib::IMaterial_instance* create_material_instance(
const std::string& qualified_module_name,
const std::string& material_db_name);
// Compiles the given material instance in the given compilation modes.
mi::neuraylib::ICompiled_material* compile_material_instance(
bool class_compilation);
// Creates an instance of the given material.
mi::neuraylib::IMaterial_instance* Material_compiler::create_material_instance(
const std::string& qualified_module_name,
const std::string& material_db_name)
if (!material_definition)
exit_failure("Failed to access material definition '%s'.", material_db_name.c_str());
// Create a material instance from the material definition
// with the default arguments.
mi::Sint32 result;
material_definition->create_material_instance(0, &result));
check_success(result == 0);
if (result != 0)
exit_failure("Failed to instantiate material '%s'.", material_db_name.c_str());
return material_instance.get();
// Compiles the given material instance in the given compilation modes.
mi::neuraylib::ICompiled_material *Material_compiler::compile_material_instance(
bool class_compilation)
mi::Uint32 flags = class_compilation
material_instance->create_compiled_material(flags, m_context.get()));
return compiled_material.get();
// Generates GLSL target code for a subexpression of a given compiled material.
mi::base::Handle<const mi::neuraylib::ITarget_code> Material_compiler::generate_glsl()
m_be_glsl->translate_link_unit(m_link_unit.get(), m_context.get()));
#ifdef DUMP_GLSL
std::cout << "Dumping GLSL code:\n\n" << code_glsl->get_code() << std::endl;
return code_glsl;
// Generates GLSL target code for a subexpression of a given material.
// path is the path of the sub-expression.
// fname is the function name in the generated code.
bool Material_compiler::add_material_subexpr(
const std::string& qualified_module_name,
const std::string& material_db_name,
const char* path,
const char* fname)
// Load the given module and create a material instance
create_material_instance(qualified_module_name, material_db_name));
// Compile the material instance in instance compilation mode
compile_material_instance(material_instance.get(), /*class_compilation=*/false));
m_link_unit->add_material_expression(compiled_material.get(), path, fname,
return print_messages(m_context.get());
// Constructor.
: m_factory(mi::base::make_handle_dup(mdl_factory))
, m_mdl_impexp_api(mi::base::make_handle_dup(mdl_impexp_api))
, m_be_glsl(mdl_backend_api->get_backend(mi::neuraylib::IMdl_backend_api::MB_GLSL))
, m_transaction(mi::base::make_handle_dup(transaction))
, m_context(mdl_factory->create_execution_context())
, m_link_unit()
check_success(m_be_glsl->set_option("num_texture_spaces", "1") == 0);
#ifdef USE_SSBO
// SSBO requires GLSL 4.30
check_success(m_be_glsl->set_option("glsl_version", "430") == 0);
check_success(m_be_glsl->set_option("glsl_version", "330") == 0);
// Specify the implementation modes for some state functions.
// Note that "geometry_normal", "normal" and "position" default to "field" mode.
check_success(m_be_glsl->set_option("glsl_state_animation_time_mode", "field") == 0);
check_success(m_be_glsl->set_option("glsl_state_position_mode", "func") == 0);
check_success(m_be_glsl->set_option("glsl_state_texture_coordinate_mode", "arg") == 0);
check_success(m_be_glsl->set_option("glsl_state_texture_tangent_u_mode", "field") == 0);
check_success(m_be_glsl->set_option("glsl_state_texture_tangent_v_mode", "field") == 0);
#ifdef USE_SSBO
check_success(m_be_glsl->set_option("glsl_max_const_data", "0") == 0);
check_success(m_be_glsl->set_option("glsl_place_uniforms_into_ssbo", "on") == 0);
check_success(m_be_glsl->set_option("glsl_max_const_data", "1024") == 0);
check_success(m_be_glsl->set_option("glsl_place_uniforms_into_ssbo", "off") == 0);
// remap noise functions that access the constant tables
",_ZN4base8mi_noiseEu4int3=noise_mi_int3") == 0);
// After we set the options, we can create the link unit
m_link_unit = mi::base::make_handle(m_be_glsl->create_link_unit(transaction, m_context.get()));
// Application logic
// Context structure for window callback functions.
struct Window_context
// A number from 1 to 7 specifying the material pattern to display.
unsigned material_pattern;
// GLFW callback handler for keyboard inputs.
void handle_key(GLFWwindow *window, int key, int /*scancode*/, int action, int /*mods*/)
// Handle key press events
if (action == GLFW_PRESS) {
// Map keypad numbers to normal numbers
if (GLFW_KEY_KP_0 <= key && key <= GLFW_KEY_KP_9)
key += GLFW_KEY_0 - GLFW_KEY_KP_0;
switch (key) {
// Escape closes the window
glfwSetWindowShouldClose(window, GLFW_TRUE);
// Numbers 1 - 7 select the different material patterns
case GLFW_KEY_1:
case GLFW_KEY_2:
case GLFW_KEY_3:
case GLFW_KEY_4:
case GLFW_KEY_5:
case GLFW_KEY_6:
case GLFW_KEY_7:
Window_context *ctx = static_cast<Window_context*>(
ctx->material_pattern = key - GLFW_KEY_0;
// GLFW callback handler for framebuffer resize events (when window size or resolution changes).
void handle_framebuffer_size(GLFWwindow* /*window*/, int width, int height)
glViewport(0, 0, width, height);
// Initializes OpenGL, creates the shader program and the scene and executes the animation loop.
void show_and_animate_scene(
Options const &options)
Window_context window_context = { options.material_pattern };
// Init OpenGL window
GLFWwindow *window = init_opengl(options);
// Create shader program
GLuint program = create_shader_program(target_code);
// Create scene data
GLuint quad_vertex_buffer;
GLuint quad_vao = create_quad(program, &quad_vertex_buffer);
// Scope for material context resources
// Prepare the needed material data of all target codes for the fragment shader
Material_opengl_context material_opengl_context(program);
transaction, image_api, target_code));
// Get locations of uniform parameters for fragment shader
GLint material_pattern_index = glGetUniformLocation(program, "material_pattern");
GLint animation_time_index = glGetUniformLocation(program, "animation_time");
if (!options.no_window) {
GLfloat animation_time = 0;
double last_frame_time = glfwGetTime();
glfwSetWindowUserPointer(window, &window_context);
glfwSetKeyCallback(window, handle_key);
glfwSetFramebufferSizeCallback(window, handle_framebuffer_size);
// Loop until the user closes the window
while (!glfwWindowShouldClose(window))
// Update animation time
double cur_frame_time = glfwGetTime();
animation_time += GLfloat(cur_frame_time - last_frame_time);
last_frame_time = cur_frame_time;
// Set uniform frame parameters
glUniform1ui(material_pattern_index, window_context.material_pattern);
glUniform1f(animation_time_index, animation_time);
// Render the scene
glDrawArrays(GL_TRIANGLES, 0, 6);
// Swap front and back buffers
// Poll for events and process them
} else { // no_window
// Set up frame buffer
GLuint frame_buffer = 0, color_buffer = 0;
glGenFramebuffers(1, &frame_buffer);
glBindFramebuffer(GL_FRAMEBUFFER, frame_buffer);
glGenRenderbuffers(1, &color_buffer);
glBindRenderbuffer(GL_RENDERBUFFER, color_buffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8, options.res_x, options.res_y);
GL_RENDERBUFFER, color_buffer);
// Set uniform frame parameters
glUniform1ui(material_pattern_index, window_context.material_pattern);
glUniform1f(animation_time_index, 0.f);
// Render the scene
glViewport(0, 0, options.res_x, options.res_y);
glDrawArrays(GL_TRIANGLES, 0, 6);
// Create a canvas (with only one tile) and copy the result image to it
image_api->create_canvas("Rgba", options.res_x, options.res_y));
glReadPixels(0, 0, options.res_x, options.res_y,
GL_RGBA, GL_UNSIGNED_BYTE, tile->get_data());
// Save the image to disk
mdl_impexp_api->export_canvas(options.outputfile.c_str(), canvas.get());
// Cleanup frame buffer
glDeleteRenderbuffers(1, &color_buffer);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &frame_buffer);
// Cleanup OpenGL
glDeleteVertexArrays(1, &quad_vao);
glDeleteBuffers(1, &quad_vertex_buffer);
void usage(char const *prog_name)
<< "Usage: " << prog_name << " [options] [<material_pattern>]\n"
<< "Options:\n"
<< " --nowin don't show interactive display\n"
<< " --res <x> <y> resolution (default: 1024x768)\n"
<< " -o <outputfile> image file to write result in nowin mode (default: output.png)\n"
<< " <material_pattern> a number from 1 to 7 choosing which material combination to use"
<< std::endl;
// Main function
int MAIN_UTF8(int argc, char* argv[])
// Parse command line options
Options options;
for (int i = 1; i < argc; ++i) {
char const *opt = argv[i];
if (opt[0] == '-') {
if (strcmp(opt, "--nowin") == 0)
options.no_window = true;
else if (strcmp(opt, "-o") == 0) {
if (i < argc - 1)
options.outputfile = argv[++i];
else if (strcmp(opt, "--res") == 0) {
if (i < argc - 2) {
options.res_x = std::max(atoi(argv[++i]), 1);
options.res_y = std::max(atoi(argv[++i]), 1);
} else
} else {
options.material_pattern = unsigned(atoi(opt));
if (options.material_pattern < 1 || options.material_pattern > 7) {
std::cerr << "Invalid material_pattern parameter." << std::endl;
// Access the MDL SDK
mi::base::Handle<mi::neuraylib::INeuray> neuray(mi::examples::mdl::load_and_get_ineuray());
if (!neuray.is_valid_interface())
exit_failure("Failed to load the SDK.");
// Configure the MDL SDK
mi::examples::mdl::Configure_options configure_options;
if (!mi::examples::mdl::configure(neuray.get(), configure_options))
exit_failure("Failed to initialize the SDK.");
// Start the MDL SDK
mi::Sint32 ret = neuray->start();
if (ret != 0)
exit_failure("Failed to initialize the SDK. Result code: %d", ret);
// Create a transaction
mi::base::Handle<mi::neuraylib::IScope> scope(database->get_global_scope());
mi::base::Handle<mi::neuraylib::ITransaction> transaction(scope->create_transaction());
// Access needed API components
// Load the module.
std::string module_name = "::nvidia::sdk_examples::tutorials";
mdl_impexp_api->load_module(transaction.get(), module_name.c_str(), context.get());
if (!print_messages(context.get()))
exit_failure("Loading module '%s' failed.", module_name.c_str());
// Get the database name for the module we loaded
if (!module)
exit_failure("Failed to access the loaded module.");
// Set up the materials.
std::vector<std::string> material_simple_names;
std::vector<std::string> fnames;
#if defined(USE_SSBO) || defined(REMAP_NOISE_FUNCTIONS)
#if defined(USE_SSBO) || defined(REMAP_NOISE_FUNCTIONS)
// Construct material DB names.
size_t n = material_simple_names.size();
std::vector<std::string> material_db_names(n);
for (size_t i = 0; i < n; ++i) {
= std::string(module_db_name->get_c_str()) + "::" + material_simple_names[i];
material_db_names[i] = mi::examples::mdl::add_missing_material_signature(
module.get(), material_db_names[i]);
if (material_db_names[i].empty())
exit_failure("Failed to find the material %s in the module %s.",
material_simple_names[i].c_str(), module_name.c_str());
// Add material sub-expressions of different materials to the link unit.
Material_compiler mc(
mdl_impexp_api.get(), mdl_backend_api.get(), mdl_factory.get(), transaction.get());
for (size_t i = 0; i < n; ++i) {
module_name, material_db_names[i], "surface.scattering.tint", fnames[i].c_str());
// Generate the GLSL code for the link unit.
mi::base::Handle<const mi::neuraylib::ITarget_code> target_code(mc.generate_glsl());
// Acquire image API needed to prepare the textures
show_and_animate_scene(transaction, mdl_impexp_api, image_api, target_code, options);
// Shut down the MDL SDK
if (neuray->shutdown() != 0)
exit_failure("Failed to shutdown the SDK.");
// Unload the MDL SDK
neuray = nullptr;
if (!mi::examples::mdl::unload())
exit_failure("Failed to unload the SDK.");
// Convert command line arguments to UTF8 on Windows

Source Code Location: examples/mdl_sdk/execution_glsl/example_execution_glsl.vert

* Copyright 2021 NVIDIA Corporation. All rights reserved.
#version 330 core
layout (location = 0) in vec3 Position;
layout (location = 1) in vec2 TexCoord;
out vec3 vPosition;
out vec3[1] texture_coordinate;
void main()
gl_Position = vec4(Position, 1.0);
vPosition = Position;
texture_coordinate = vec3[1](vec3(TexCoord.x, TexCoord.y, 0.0));

Source Code Location: examples/mdl_sdk/execution_glsl/example_execution_glsl.frag

* Copyright 2021 NVIDIA Corporation. All rights reserved.
// examples/mdl_sdk/execution_glsl/example_execution_glsl.frag
// This file contains the implementations of the texture access functions
// and the fragment shader used to evaluate the material sub-expressions.
// Material pattern as chosen by the user.
uniform uint material_pattern;
// Current time in seconds since the start of the render loop.
uniform float animation_time;
// Mapping from material index to start index in material_texture_samplers_2d.
uniform uint material_texture_starts[MAX_MATERIALS];
// Array containg all 2D texture samplers of all used materials.
uniform sampler2D material_texture_samplers_2d[MAX_TEXTURES];
// Start offset of the current material inside material_texture_samplers_2d, set in main.
uint tex_start;
// The input variables coming from the vertex shader.
in vec3[1] texture_coordinate; // used for state::texture_coordinate(tex_space) in "arg" mode
in vec3 vPosition;
// The color output variable of this fragment shader.
out vec4 FragColor;
// The MDL material state structure as configured via the GLSL backend options.
// Note: Must be in sync with the state struct in generate_glsl_switch_func and the code generated
// by the MDL SDK (see dumped code when enabling DUMP_GLSL in example_execution_glsl.cpp).
struct State {
vec3 normal;
vec3 geometry_normal;
float animation_time;
vec3[1] texture_tangent_u;
vec3[1] texture_tangent_v;
// The prototypes of the functions generated in our generate_glsl_switch_func() function.
// Return the number of available MDL material subexpressions.
uint get_mdl_num_mat_subexprs();
// Return the result of the MDL material subexpression given by the id.
vec3 mdl_mat_subexpr(uint id, State state);
#if __VERSION__ < 400
int bitCount(uint x)
x = x - ((x >> 1u) & 0x55555555u);
x = (x & 0x33333333u) + ((x >> 2u) & 0x33333333u);
x = (x + (x >> 4u)) & 0x0F0F0F0Fu;
x = x + (x >> 8u);
x = x + (x >> 16u);
return int(x & 0x0000003Fu);
// Implementation of tex::lookup_*() for a texture_2d texture.
vec4 tex_lookup_2d(uint tex, vec2 coord, int wrap_u, int wrap_v, vec2 crop_u, vec2 crop_v)
if (tex == 0u) return vec4(0);
return texture(material_texture_samplers_2d[tex_start + tex - 1u], coord);
// Implementation of tex::texel_*() for a texture_2d texture.
vec4 tex_texel_2d(uint tex, ivec2 coord, ivec2 uv_tile)
if (tex == 0u) return vec4(0);
return texelFetch(material_texture_samplers_2d[tex_start + tex - 1u], coord, 0);
// Implementation of state::position() in "func" mode.
vec3 position()
// Mirror the position across the center in this example.
return -vPosition;
// The fragment shader main function evaluating the MDL sub-expression.
void main() {
// Set number of materials to use according to selected pattern
uint num_materials = uint(bitCount(material_pattern));
// Assign materials in a checkerboard pattern
uint material_index =
(uint(texture_coordinate[0].x * 4) ^ uint(texture_coordinate[0].y * 4)) % num_materials;
// Change material index according to selected pattern
switch (material_pattern)
case 2u: material_index = 1u; break;
case 4u: material_index = 2u; break;
case 5u: if (material_index == 1u) material_index = 2u; break;
case 6u: material_index += 1u; break;
if (material_index > get_mdl_num_mat_subexprs())
material_index = get_mdl_num_mat_subexprs();
// Set up texture access for the chosen material
tex_start = material_texture_starts[material_index];
// Set MDL material state for state functions in "field" mode
State state = State(
/*normal=*/ vec3(0.0, 0.0, 1.0),
/*geometry_normal=*/ vec3(0.0, 0.0, 1.0),
/*animation_time=*/ animation_time,
/*texture_tangent_u=*/ vec3[1](vec3(1.0, 0.0, 0.0)),
/*texture_tangent_v=*/ vec3[1](vec3(0.0, 1.0, 0.0))
// Evaluate material sub-expression
vec3 res = mdl_mat_subexpr(material_index, state);
// Apply gamma correction and write to output variable
FragColor = pow(vec4(res, 1.0), vec4(1.0 / 2.2));
[Previous] [Up] [Next]