Example for Importers

[Previous] [Next] [Up]

This example demonstrates the implementation and usage of custom importers to be used in conjunction with the Iray SDK API. In this example the new importer is defined in and used by the main application for simplicity. Note that it is possible to provide custom importers via plugins, as for any other user defined class, see Plugins.

A simple importer called "Vanilla importer" will be used in this example to demonstrate the basic steps. This importer is an illustrative skeleton that implements all interfaces but does not actually parse the file content in a meaningful way.

New Topics

• Implementation of an importer
• Registration of an importer

Detailed Description

Implementation of an importer

The implementation of the Vanilla importer in the example source in structured in three parts:

• Implementation of the mi::neuraylib::IImpexp_state interface
• Implementation of the mi::neuraylib::IImporter interface
• Implementation of the mi::neuraylib::IImporter::import_elements() method

Instances of mi::neuraylib::IImpexp_state are used to pass information about the current importer state, for example to recursive calls of importers. The Vanilla importer does not need to carry around any additional information besides what is required by the interface, therefore this simple implementation is fine. Note that the simple derivation from mi::base::Interface_implement suffices here (in contrast to user-defined classes).

The Vanilla importer is given in the implementation of the mi::neuraylib::IImporter interface. Later, an instance of this class will be registered as importer with the Iray SDK API. Most of the methods implemented here are actually defined in the base interface mi::neuraylib::IImpexp_state, as they are common for importers and exporters. The Vanilla importer claims to handle files with extension ".vnl" and ".van". It does not require specific capabilities of the reader to handle these formats. However, if the readers supports the lookahead capability, it will use a magic header check instead of relying on file name extensions.

The actual work of the Vanilla importer happens in the import_elements() method. It is split into three parts:

• creation of the import result object
• creation of a group object which acts as rootgroup
• reading the file line by line

While performing these tasks the example demonstrates what type of errors to detect, a way to report the errors, and how to implement an include file mechanism and similar things with a recursive call to the mi::neuraylib::IImport_api::import_elements() method.

Registration of an importer

The registration of an importer is similar to the registration of user-defined classes. However, since importers are different from regular classes (e.g., you cannot create instances of them using mi::neuraylib::ITransaction::create()) you need to use a registration method specific to importers. This registration method expects a pointer to an instance of the custom importer.

To run the example, you need to create two files called test1.vnl and test2.van, each 5 lines long and starting with a line saying VANILLA. The example will print the (for demonstration purposes generated) errors messages and the names of the imported elements.

Source Code Location: example/vanilla_importer.h

/******************************************************************************
 * Copyright 2023 NVIDIA Corporation. All rights reserved.
 *****************************************************************************/
 
#include <mi/neuraylib.h>
 
#include <string>
#include <sstream>
 
// Support function to handle mi::base::Message_severity to std::string conversion
static std::string enum_to_str( mi::base::Message_severity severity)
{
    switch( severity) {
        case mi::base::MESSAGE_SEVERITY_FATAL:   return "fatal";
        case mi::base::MESSAGE_SEVERITY_ERROR:   return "error";
        case mi::base::MESSAGE_SEVERITY_WARNING: return "warning";
        case mi::base::MESSAGE_SEVERITY_INFO:    return "information";
        case mi::base::MESSAGE_SEVERITY_VERBOSE: return "verbose";
        case mi::base::MESSAGE_SEVERITY_DEBUG:   return "debug";
        default:                                 return "unknown" ;
    }
}
 
// Define importer state, which is used in the importer function to carry additional data around for
// possible recursive invocations of importers. It contains a URI for the imported resource, a line
// number, and a pointer to a parent state supporting recursive imports.
class Vanilla_import_state
  : public mi::base::Interface_implement< mi::neuraylib::IImpexp_state>
{
public:
    // Constructor.
    Vanilla_import_state( const char* uri, const mi::neuraylib::IImpexp_state* parent_state)
      : m_line_number( 1),
        m_uri( uri ? uri : ""),
        m_parent_state( parent_state, mi::base::DUP_INTERFACE) { }
 
    // Definition of all interface functions.
 
    const char* get_uri() const { return m_uri.empty() ? 0 : m_uri.c_str(); }
 
    mi::Uint32 get_line_number() const { return m_line_number; }
 
    void set_line_number( mi::Uint32 number) { m_line_number = number; }
 
    void incr_line_number() { ++m_line_number; }
 
    const mi::neuraylib::IImpexp_state* get_parent_state() const
    {
        if( m_parent_state)
            m_parent_state->retain();
        return m_parent_state.get();
    }
 
private:
    mi::Uint32 m_line_number;
    std::string m_uri;
    mi::base::Handle<const mi::neuraylib::IImpexp_state> m_parent_state;
};
 
// Define importer. It defines all meta information, for example, author, version numbers, which
// formats it supports etc. The longer format detection functions and the import_elements() function
// are implemented outside of the class body.
class Vanilla_importer
  : public mi::base::Interface_implement< mi::neuraylib::IImporter>
{
public:
    // Constructor.
    Vanilla_importer( mi::neuraylib::IPlugin_api* plugin_api)
        : m_plugin_api( plugin_api, mi::base::DUP_INTERFACE) { }
 
    // Definition of all interface functions.
 
    mi::neuraylib::IImpexp_state* create_impexp_state(
        const char* uri, const mi::neuraylib::IImpexp_state* parent_state) const
    {
        return new Vanilla_import_state( uri, parent_state);
    }
 
    // This importer supports the file name extensions ".vnl" and ".van".
    const char* get_supported_extensions( mi::Uint32 i) const
    {
        switch( i) {
            case 0:  return ".vnl";
            case 1:  return ".van";
            default: return 0;
        }
    }
 
    mi::neuraylib::Impexp_priority get_priority() const
    {
        return mi::neuraylib::IMPEXP_PRIORITY_WELL_DEFINED;
    }
 
    const char* get_name() const { return "NVIDIA example vanilla (v1) importer"; }
 
    const char* get_author() const { return "NVIDIA Corporation, Berlin, Germany"; }
 
    mi::base::Uuid get_uuid() const
    {
        mi::base::Uuid uuid;
        uuid.m_id1 = 0x338eca60;
        uuid.m_id2 = 0x31004802;
        uuid.m_id3 = 0xaab9046b;
        uuid.m_id4 = 0x9e0b1d9b;
        return uuid;
    }
 
    mi::Uint32 get_major_version() const { return 1; }
 
    mi::Uint32 get_minor_version() const { return 0; }
 
    bool test_file_type( const char* extension) const;
 
    bool test_file_type( const char* extension, const mi::neuraylib::IReader* reader) const;
 
    mi::neuraylib::IImport_result* import_elements(
        mi::neuraylib::ITransaction* transaction,
        const char* extension,
        mi::neuraylib::IReader* reader,
        const mi::IMap* options,
        mi::neuraylib::IImpexp_state* import_state) const;
 
private:
    // Definition of support functions. They are not part of the interface.
 
    // Format message with context and append it to the messages in the result.
    static mi::neuraylib::IImport_result_ext* report_message(
        mi::neuraylib::IImport_result_ext* result,
        mi::Uint32 message_number,
        mi::base::Message_severity message_severity,
        std::string message,
        const mi::neuraylib::IImpexp_state* import_state)
    {
        std::ostringstream s;
        s << import_state->get_uri()
          << ":" << import_state->get_line_number() << ": "
          << "Vanilla importer message " << message_number << ", "
          << "severity " << enum_to_str( message_severity) << ": "
          << message;
        // Report context of all parent import states from recursive invocations of
        // import_elements() in their own lines with indentation.
        mi::base::Handle<const mi::neuraylib::IImpexp_state> parent_state(
            import_state->get_parent_state());
        while( parent_state.is_valid_interface()) {
            s << "\n    included from: " << parent_state->get_uri()
              << ":" << parent_state->get_line_number();
            parent_state = parent_state->get_parent_state();
        }
        result->message_push_back( message_number, message_severity, s.str().c_str());
        return result;
    }
 
    mi::base::Handle<mi::neuraylib::IPlugin_api> m_plugin_api;
};
 
bool Vanilla_importer::test_file_type( const char* extension) const
{
    // This importer supports the file name extensions ".vnl" and ".van".
    mi::Size len = std::strlen( extension);
    return (len > 3)
        &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
          || ( 0 == strcmp( extension + len - 4, ".van")));
}
 
bool Vanilla_importer::test_file_type(
    const char* extension, const mi::neuraylib::IReader* reader) const
{
    // Use magic header check if lookahead is available
    if( reader->supports_lookahead()) {
        // File has to start with "VANILLA" and linebreak, which can be \n or \r depending on the
        // line ending convention in the file.
        const char** buffer = 0;
        mi::Sint64 n = reader->lookahead( 8, buffer);
        return ( n >= 8) && (0 == std::strncmp( *buffer, "VANILLA", 7))
                         && (((*buffer)[7] == '\n') || ((*buffer)[7] == '\r'));
    }
    // This importer supports the file name extensions ".vnl" and ".van".
    mi::Size len = std::strlen( extension);
    return (len > 3)
        &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
          || ( 0 == strcmp( extension + len - 4, ".van")));
}
 
mi::neuraylib::IImport_result* Vanilla_importer::import_elements(
    mi::neuraylib::ITransaction* transaction,
    const char* extension,
    mi::neuraylib::IReader* reader,
    const mi::IMap* importer_options,
    mi::neuraylib::IImpexp_state* import_state) const
{
    // Create the importer result instance for the return value. If that fails something is really
    // wrong and we return 0.
    mi::neuraylib::IImport_result_ext* result
        = transaction->create<mi::neuraylib::IImport_result_ext>( "Import_result_ext");
    if( !result)
        return 0;
 
    // Get the 'prefix' option.
    std::string prefix;
    if( importer_options && importer_options->has_key( "prefix")) {
        mi::base::Handle<const mi::IString> option(
            importer_options->get_value<mi::IString>( "prefix"));
        prefix = option->get_c_str();
    }
 
    // Get the 'list_elements' option.
    bool list_elements = false;
    if( importer_options && importer_options->has_key( "list_elements")) {
        mi::base::Handle<const mi::IBoolean> option(
            importer_options->get_value<mi::IBoolean>( "list_elements"));
        list_elements = option->get_value<bool>();
    }
 
    // Before we start parsing the file, we create a group that collects all top-level elements.
    // This will be our rootgroup.
    std::string root_group_name = prefix + "Vanilla::root_group";
    mi::base::Handle<mi::neuraylib::IGroup> rootgroup(
        transaction->create<mi::neuraylib::IGroup>( "Group"));
    mi::Sint32 error_code = transaction->store( rootgroup.get(), root_group_name.c_str());
    if( error_code != 0)
        return report_message( result, 4010, mi::base::MESSAGE_SEVERITY_ERROR,
            "failed to create the root group", import_state);
    rootgroup = transaction->edit<mi::neuraylib::IGroup>( root_group_name.c_str());
 
    // Register the rootgroup with the importer result.
    result->set_rootgroup( root_group_name.c_str());
 
    // If the element list flag is set, record the rootgroup element also in the elements array of
    // the result.
    if( list_elements)
        result->element_push_back( root_group_name.c_str());
 
    // Assume it is a line based text format and read it line by line. Assume lines are no longer
    // than 256 chars, otherwise read it in pieces
    char buffer[257];
    while( reader->readline( buffer, 257) && buffer[0] != '\0') {
 
        // Periodically check whether the transaction is still open. If not, stop importing.
        if( !transaction->is_open())
            break;
 
        // Here you can process the buffer content of size len. It corresponds to the line
        // import_state->get_line_number() of the input file.
        mi::Size len = std::strlen( buffer);
 
        // We illustrate some actions triggered by fixed line numbers: Line 3 of a ".vnl" file
        // triggers the recursive inclusion of the file "test2.van" file with a prefix.
        mi::Size ext_len = std::strlen( extension);
        if( 3 == import_state->get_line_number()
             && (ext_len > 3)
             && 0 == strcmp( extension + ext_len - 4, ".vnl")) {
            // Get a IImport_api handle to call its import_elements() function
            mi::base::Handle<mi::neuraylib::IImport_api> import_api(
                m_plugin_api->get_api_component<mi::neuraylib::IImport_api>());
            if( !import_api.is_valid_interface())
                // Error numbers from 4000 to 5999 are reserved for custom importer messages like
                // this one
                return report_message( result, 4001, mi::base::MESSAGE_SEVERITY_ERROR,
                   "did not get a valid IImport_api object, import failed", import_state);
            // Call the importer recursively to illustrate the handling of include files and similar
            // things. We trigger this import only on a ".vnl" file and include the fixed file
            // "test2.van". We give the included file an extra prefix "Prefix_".
            mi::base::Handle<mi::neuraylib::IFactory> factory(
                m_plugin_api->get_api_component<mi::neuraylib::IFactory>());
            mi::base::Handle<mi::IString> child_prefix( factory->create<mi::IString>( "String"));
            child_prefix->set_c_str( "Prefix_");
            mi::base::Handle<mi::IMap> child_importer_options( factory->clone<mi::IMap>(
                importer_options));
            child_importer_options->erase( "prefix");
            child_importer_options->insert( "prefix", child_prefix.get());
            mi::base::Handle<const mi::neuraylib::IImport_result> include_result(
                import_api->import_elements( transaction, "file:test2.van",
                    child_importer_options.get(), import_state));
            // Safety check, if this fails, the import is not continued.
            if( !include_result.is_valid_interface())
                return report_message( result, 4002, mi::base::MESSAGE_SEVERITY_ERROR,
                   "import was not able to create result object, import failed", import_state);
            // Process the result. Even in the case of an error, we need to process the elements
            // array.
            if( list_elements)
                result->append_elements( include_result.get());
            // Append messages of include to this result
            result->append_messages( include_result.get());
            // React on errors during processing of the include
            if( include_result->get_error_number() > 0) {
                // Report the failure of the include as a separate message too
                report_message( result, 4003, mi::base::MESSAGE_SEVERITY_ERROR,
                    "including file 'test2.van' failed.", import_state);
            } else {
                // Recursive import was successful. The rootgroup of the import is now appended to
                // this rootgroup
                if( 0 == include_result->get_rootgroup())
                    report_message( result, 4004, mi::base::MESSAGE_SEVERITY_ERROR,
                        "include file 'test2.van' did not contain a rootgroup", import_state);
                else
                    rootgroup->attach( include_result->get_rootgroup());
            }
        }
 
        // Line 4 triggers several messages and adds an empty group to the rootgroup.
        if( 4 == import_state->get_line_number()) {
 
            // Several messages, file parsing continues
            report_message( result, 4005, mi::base::MESSAGE_SEVERITY_FATAL,
                "test message in line 4", import_state);
            report_message( result, 4006, mi::base::MESSAGE_SEVERITY_ERROR,
                "test message in line 4", import_state);
            report_message( result, 4007, mi::base::MESSAGE_SEVERITY_WARNING,
                "test message in line 4", import_state);
            report_message( result, 4008, mi::base::MESSAGE_SEVERITY_INFO,
                "test message in line 4", import_state);
            report_message( result, 4009, mi::base::MESSAGE_SEVERITY_VERBOSE,
                 "test message in line 4", import_state);
 
            // Create a group "Vanilla::Group1"
            std::string group_name = prefix + "Vanilla::Group1";
            mi::base::Handle<mi::neuraylib::IGroup> group(
                transaction->create<mi::neuraylib::IGroup>( "Group"));
            mi::Sint32 error_code = transaction->store( group.get(), group_name.c_str());
            if( error_code != 0)
                report_message( result, 4011, mi::base::MESSAGE_SEVERITY_ERROR,
                    "unexpected error in line 4", import_state);
            else {
                // Add this group to the rootgroup
                rootgroup->attach( group_name.c_str());
                // If get_list_elements_flag is set, record the new element in the elements array of
                // the result.
                if( list_elements)
                    result->element_push_back( group_name.c_str());
            }
        }
 
        // Handle line numbers, buffer might end in '\n' or not if line was too long.
        if( (len > 0) && ('\n' == buffer[len-1]))
            import_state->incr_line_number();
    }
 
    if( reader->eof()) {
        // Normal end
        return result;
    }
 
    // Report error condition for a failed reader call
    return report_message(
        result,
        static_cast<mi::Uint32>( reader->get_error_number()),
        mi::base::MESSAGE_SEVERITY_ERROR,
        reader->get_error_message() ? reader->get_error_message() : "",
        import_state);
}

Source Code Location: examples/example_importer.cpp

/******************************************************************************
 * Copyright 2023 NVIDIA Corporation. All rights reserved.
 *****************************************************************************/
 
// examples/example_importer.cpp
//
// Demonstrates the implementation of custom importers
 
#include <iostream>
 
// Include code shared by all examples.
#include "example_shared.h"
 
// Include header file for the Vanilla importer.
#include "vanilla_importer.h"
 
// The importer.
mi::base::Handle<mi::neuraylib::IImporter> importer;
 
void configuration( mi::neuraylib::INeuray* neuray)
{
    // Register the Vanilla importer.
    mi::base::Handle<mi::neuraylib::IExtension_api> extension_api(
        neuray->get_api_component<mi::neuraylib::IExtension_api>());
    check_success( extension_api.is_valid_interface());
    mi::base::Handle<mi::neuraylib::IPlugin_api> plugin_api(
        neuray->get_api_component<mi::neuraylib::IPlugin_api>());
    check_success( plugin_api.is_valid_interface());
    importer = new Vanilla_importer( plugin_api.get());
    check_success( extension_api->register_importer( importer.get()) == 0);
}
 
void test_importer( mi::neuraylib::INeuray* neuray)
{
    // Get the database, the global scope of the database, and create a transaction in the global
    // scope.
    mi::base::Handle<mi::neuraylib::IDatabase> database(
        neuray->get_api_component<mi::neuraylib::IDatabase>());
    check_success( database.is_valid_interface());
    mi::base::Handle<mi::neuraylib::IScope> scope(
        database->get_global_scope());
    mi::base::Handle<mi::neuraylib::ITransaction> transaction(
        scope->create_transaction());
    check_success( transaction.is_valid_interface());
 
    // Prepare the importer options:
    // We do not want to have an additional prefix, but a list of all imported elements.
    mi::base::Handle<mi::IBoolean> list_elements( transaction->create<mi::IBoolean>( "Boolean"));
    list_elements->set_value( true);
    mi::base::Handle<mi::IMap> importer_options( transaction->create<mi::IMap>( "Map<Interface>"));
    importer_options->insert( "list_elements", list_elements.get());
 
    // Import the file test1.vnl (implicitly using the Vanilla importer).
    mi::base::Handle<mi::neuraylib::IImport_api> import_api(
        neuray->get_api_component<mi::neuraylib::IImport_api>());
    check_success( import_api.is_valid_interface());
    mi::base::Handle<const mi::neuraylib::IImport_result> import_result(
        import_api->import_elements( transaction.get(), "file:test1.vnl", importer_options.get()));
    check_success( import_result.is_valid_interface());
 
    // Print all messages
    for( mi::Size i = 0; i < import_result->get_messages_length(); ++i)
        std::cout << import_result->get_message( i) << std::endl;
 
    // Print all imported elements
    for( mi::Size i = 0; i < import_result->get_elements_length(); ++i)
        std::cout << import_result->get_element( i) << std::endl;
 
    transaction->commit();
}
 
int main( int /*argc*/, char* /*argv*/[])
{
    // Access the neuray library
    mi::base::Handle<mi::neuraylib::INeuray> neuray( load_and_get_ineuray());
    check_success( neuray.is_valid_interface());
 
    // Configure the neuray library
    configuration( neuray.get());
 
    // Start the neuray library
    mi::Sint32 result = neuray->start();
    check_start_success( result);
 
    // Test the Vanilla importer
    test_importer( neuray.get());
 
    // Shut down the neuray library
    check_success( neuray->shutdown() == 0);
 
    // Unregister the Vanilla importer.
    mi::base::Handle<mi::neuraylib::IExtension_api> extension_api(
        neuray->get_api_component<mi::neuraylib::IExtension_api>());
    check_success( extension_api->unregister_importer( importer.get()) == 0);
    importer = 0;
    extension_api = 0;
    neuray = 0;
 
    // Unload the neuray library
    check_success( unload());
 
    keep_console_open();
    return EXIT_SUCCESS;
}

[Previous] [Next] [Up]