Category Archives: Lua

Modern C++ Lua Interface

Posted on by Ale
Reply

As we continue to grow the Lua scripting API for Vortex, it’s become clear that we need a better, less verbose way to expose “classes” and their functions to the Lua VM.

In Vortex we decided to build our custom Lua binding. This initial work was done about two years ago, and it was built by directly invoking the Lua C API. This means that exposing something like the vtx::Transform class, along with a few methods, would look something like this:

Now, the problem with this approach is that it’s very verbose and also pretty error prone. Wouldn’t it be nice if we could have a Modern C++ way of exposing classes and their methods to Lua scripts that didn’t require all this typing?

A Better Interface

What we want to do is easily define a class with a few methods and their callbacks into native code.

Leveraging C++11’s initializer lists, it turns out it’s pretty simple to do this. Have a look at the following simplified way of defining functions. It looks more readable and way more in line with Modern C++ in my opinion:

The pushClass method receives the name under which the class will be exposed to Lua scripts and a table of method names that map to the native implementation inside the engine. It also receives the Lua VM handle, of course :)

How can we implement this function?

We want to handle the function list as a vector. A vector of a composite type that maps a string to an implementation. To do this, we start by defining a FunctionDefinition struct. This will “map” function names (provided as strings) to implementations (provided as typed function pointers).

Now that we have a way of defining functions, we provide an implementation of the pushClass() function that can take a vector of this struct:

Notice how the case of receiving an empty vector is gracefully handled. If this happens, an empty table will be defined. This could be useful to have a type with no members to be used as a handle to an engine object.

So is this the best way to interface Lua and modern C++?

This method does work and I have already been using it to vastly simplify the binding code that exposes engine objects to scripts. This has made the client code much more readable and easier to extend.

This is, however, not necessarily the best way of doing this. Other authors have done an amazing job at solving this problem using excellent C++ template metaprogramming concepts. For our case, this simple implementation is enough to manage the complexity of the problems we have today.

Now it’s time to go build something with this! In my next post I’m going to be using this logic to build and expose a camera system to Lua scripts for them to be able to control and animate the point of view.

Stay tuned for more!

Scripting the Runtime

Posted on by Ale

This past weekend I spent some time making sure the Lua scripting runtime worked correctly on the iOS runtime. While archiving already took care of including scripts into generated Vortex Archives, the Lua runtime was not being invoked on device. This had to be addressed :)

The Bouncy Ball Playground, which consists in a ball animated from a Lua script, running on iPhone from a Vortex Archive.

The Bouncy Ball Playground, which consists in a ball animated from a Lua script, running on iPhone from a Vortex Archive.

First things, first

Last time we used the Vortex Editor to build and script a scene where a ball bounces inside a brick box. Let’s see if we can load it up in the iOS Runtime.

We have to start by loading our project into the Vortex Editor on Windows. Once open, we verify that it runs and that the simulation is working as expected. Once we have validated everything is correct, we can “build” the project into a Vortex Archive.

The process of archiving collects all the assets in the project folder and puts them into a binary format. This, of course, includes all Lua script files in the folder.

Moving to the Mac

Awesome, we have our playground archive. How do we put it on device? We will use the Vortex Runtime for iOS for this.

We will take the Vortex Archive and place it under the bundled resources for our App. In the future, we could implement a system that pulls playgrounds from Dropbox, Firebase or iTunes, but for now this is enough.

When running on device, the runtime will extract the archive and deserialize the playground. The unarchiver keeps track of all extracted resources, so all we have to do is scan this list for all Lua files and run them in sequence.

The order of execution is alphabetical, but this could easily be expanded in the future.

Running on Device

The Vortex Runtime is different from the Editor. We will probably never want to edit a playground on device directly – we just want to run it as soon as it’s fully loaded. That’s what we’re going to do!

Xcode profile of the Vortex Runtime running the Bouncy Ball playground.

Xcode profile of the Vortex Runtime running the Bouncy Ball playground.

Now, one concern that I had was the impact that running the Lua scripts might have on the device’s CPU usage, and whether vanilla Lua would be still enough or if I should plan on incorporating LuaJIT soon.

Xcode provides a great way to look into the Hardware as we test our Apps on it. As it can be seen in the image above, even with vanilla Lua, CPU usage stays at a maximum of 19%. This is a mere 1% increase from before (no scripting). Granted, the script being executed is pretty simple, but keep in mind this is all running at 60 FPS. I think I am going to keep it this way (for now).

Conclusion

We finally have a complete Editor-to-Runtime experience now where we can use our PC to visually create a playground, script it, simulate it, package it all together, and load it directly into the runtime.

I was very pleasantly surprised by the results of running the Bouncy Ball Lua script on device. It is a simple script but it shows just how polished the Lua VM is. After this test, we now know that we have enough headroom to build more involved 60 FPS simulations into our playgrounds. I’m excited about what’s to come next.

Stay tuned for more!

Developing the Scripting API

Posted on by Ale

With the vertical slice complete and the 3D renderer in a good spot, this week I decided to shift focus to the scripting API of the engine.

Scripting is a very desirable feature for any engine. It allows adding (and modifying) logic on the fly, without having to recompile or relink any parts of the program. It makes iteration times super fast, enabling creativity.

In Vortex, we chose Lua for the scripting backend. We added initial support about a year ago. At that time, we decided to build a custom binding from scratch and we succeeded, but the work done was mostly proof of concept. This weekend, the objective was to expand this foundation so scripts could perform more useful tasks, such as inspecting and manipulating the world.

In order to achieve this, a number of changes were needed, both at the scripting level and at the editor level. In particular, we needed:

  • A way to wrap and expose entity transforms to Lua scripts.
  • A way to mutate these transforms.
  • A way for scripts to add themselves to the runloop and run logic every frame.
  • A way for the engine and editor to run (and “step”) scripts.
  • A way to hot reload scripts and rebooting VM when things went south.

The video above shows all these concepts coming together to allow creating a simple simulation of a ball bouncing inside a 3D box. The ball has green a point light inside that moves around with it. This is mostly to show that this simulation is still running on the engine’s modern deferred renderer ;)

The Scripting Model

Key to the scripting model is the ability to talk to the engine from a loaded script and find objects in the scene. This allows the user to visually create worlds in the Vortex Editor and then find the important entities from scripts.

Scripts can also create their own entities of course, but for this example, we just wanted to pre-build the world visually.

For the bouncy ball example in the video above, we started off by creating the containing box, the ball object, and the lights in the scene. We used the Editor tools to create all materials and define the look of the entities and lighting.

But once we have our visual scene, how do we script it?

The entry point for scripts running in Vortex is the vtx namespace. Scripts hosted by Vortex automatically get access to a global table with entry points to the engine.

Functions in the vtx namespace are serviced directly from C++. This is a powerful abstraction that allows exposing virtually all engine functionality to a script.

This is exactly what we did. Through the vtx namespace, the bouncy_ball.lua script easily finds the ball, the walls, and the light. Once we have these objects we can get their transforms and register a function that will update them every frame.

Running Scripts

Once our script is ready, we can bring it into the scene directly from within the Editor.

Currently, loading any script will execute it. This runs all code at the file scope inside it. It’s important that scripts that want to respond to engine events register their callbacks at this point.

In order to run every frame, we are interested in the on_frame event inside the vtx.callbacks table. This table behaves essentially like a list. Once every frame, the engine will walk this list and call all functions registered there.

Pausing and Testing

Since the runloop is controlled directly by the engine, this gives the Editor enormous control over script execution. In particular, we can use the Editor to pause and even step scripts!

Coupled with the Editor’s REPL Lua console, this gives the user a lot of control. Through the Editor UI, the user can stop the scripts and inspect and change any Lua objects in realtime. No need to recompile the Editor or reload the scene or scripts.

Show me the Code!

Ok, we covered a lot of ground above. To help the concepts settle in, here’s the complete bouncy_ball.lua script used to build the simulation shown above. The main points of interest are the main and on_frame functions.

The main function is responsible for finding all important entities in the scene and initializing the simulation. As mentioned before, it is run as soon as the script is loaded into the engine. Notice how the main function adds the on_frame function to the runloop.

The on_frame function runs every frame. It receives a time scale that can be used to implement a framerate-independent simulation.

It is worth noting that nothing in the on_frame function allocates memory. In particular, position components are passed into and pulled out of the engine in the Lua stack, with no heap allocations. This is important, as Lua has a Garbage-Collected runtime and we want to avoid collection pauses during the simulation.

Conclusion

It’s been a lot of fun exploring hosting a scripting language inside the engine and manually building the binding between it and C++.

I think the ability of defining the visual appearance of the scene from the Editor and then allowing scripts to find entities at runtime was the right decision at this time. It’s a simple model that solves the problem elegantly and can be performant if you cache things you need access often.

I am going to continue working on the binding further and seeing how far it can go. It’s a good break from just working on the renderer all the time ;)

I’m definitely interested in your thoughts! Please share below and, as usual, stay tuned for more!

Putting it all together

Posted on by Ale

This week has been a big one in terms of wrangling together several big pillars of the engine to provide wider functionality. The image below shows how we can now dynamically run an external Lua script that modifies the 3D world on the fly:

Vortex loading and running an external script that changes the texture of an entity's material.

Vortex loading and running an external script that changes the texture of an entity’s material.

In the image above, I’ve created two boxes. Both of these have different materials and each material references a different texture.

What you can see I’m doing is that I “mistakenly” drag from the Asset Library a character texture and assign it as the second box’s texture. Oh no! How can we fix this? It’s easy: just run an external script that will assign the first box’s texture to the second!

I’ve pasted the code of the script below:

As you can see, the script is pretty straightforward. It finds the boxes, drills all the way down to their materials and then assigns the texture of the first box to the second. The changes are immediately seen in the 3D world.

It’s worth noting that all function calls into the vtx namespace and derived objects are actually jumping into C++. This script is therefore dynamically manipulating engine objects, that’s why we see its effects in the scene view.

The function names are still work in progress, and admittedly, I need to write more scripts to see if these feel comfortable or if they’re too long and therefore hard to remember. My idea is to make the scripting interface as simple to use as possible, so please if you have any suggestions I would love to hear your feedback! Feel free to leave a comment below!

Next week I will continue working on adding more functionality to the scripting API, as well as adding more features to the renderer! Stay tuned for more!

Component Introspection

Posted on by Ale

This week, work on the scripting interface continued. As the image below shows, I can now access an entity’s components and even drill down to its Material through the Lua console.

Introspecting a Render Component to access its material via the Lua interface.

Introspecting a Render Component to access its material via the Lua interface.

The image above shows an example of the scripting interface for entities and components. Here, we are creating a new Entity from the builtin Box primitive and then finding its first component of type “1”. Type 1 is an alias to the Render Component of the Entity. It is responsible for binding a Mesh and a Material together.

Once we have the Render Component, we use it to access its Material property and print its memory address.

As the image shows, although Lua allows for very flexible duck typing, I am performing type checking behind the scenes to make scripting mistakes obvious to the user. Allow me to elaborate:

For the interface, I’ve decided that all components will be hidden behind the vtx.Component “class”. Now, this class will be responsible to exposing the interface to all native component methods, such as get_material(), set_mesh(), get_transform() and so forth.

The problem is, how do we prevent trying to access the material property of a Component that doesn’t have one, such as the Md2AnimationComponent? In my mind, there are two ways. I’m going to call them the “JavaScript” way and the “Python” way.

In the JavaScript way, we don’t really care. We allow calling any method on any component and silently fail when a mismatch is detected. We may return nil or “undefined”, but at no point are we raising an error.

In the Python way, we will perform a sanity check before actually invoking the function on the component, and actually halt the operation when an error is found. You can see that in the example above. Here we’re purposefully attempting to get the material of a Base Component (component type 0), which doesn’t have one. In this case, the Engine detects the inconsistency and raises an error.

I feel the Python way is the way to go to prevent subtle hard-to-debug errors arising from allowing any method to be called on any component and happily carrying on through to -hopefully- reach some sort of result.

A third alternative would have been to actually expose a separate “class” for every component type. This would certainly work, but I’m concerned about a potential “class explosion”, as we continue to add more and more components to the Engine. Furthermore, I feel strongly typed duck typing is a good approach, well in tune with the language philosophy, for a language like Lua.

Now that we can drill all the way down to an Entity’s material, it’s time to expand the interface to allow setting the shader and the material properties, allowing the script developer to control how entities are rendered by the Engine. Stay tuned for more!

Binding C++ to Lua

Posted on by Ale

For the last couple of weeks, a lot of work has been going into developing the scripting API that the engine is exposing to Lua. Embedding a scripting language into a large C++ codebase has been a very interesting experience and I’ve been able to experience first hand why Lua is regarded as such a strong scripting language.

Introspection of an Entity from a Lua script running in the Console.

Introspection of an Entity from a Lua script running in the Console.

Lua offers a myriad of ways we can develop a scripting interface for our native code.

A naïve approach would be to expose every function of the engine in the global namespace and have scripts use these directly. Although this method would certainly work, we want to offer an object-oriented API to the engine and its different components, so a more elaborate solution is required.

I ultimately decided to build the interface from scratch, following the Lua concepts of tables and metatables. The reason being that building everything myself would allow me to clearly see the costs of the binding as objects are passed back and forth. This will help keep an eye on performance.

Initial Test of the Lua-C++ binding. In this example, we query and rename an Entity.

Initial Test of the Lua-C++ binding. In this example, we query and rename an Entity.

In order to keep the global namespace as clean as possible, the idea was to create a Lua Table procedurally from the C++ side where all functions and types would live. Conceptually, this table is our namespace, so I named it vtx, accordingly. It’s really the only global variable that the engine registers.

The next step was to start populating the vtx namespace. Two functions I know I wanted to expose right away were Instantiate and Find:

We now have functions. But how do we do objects? And how do we expose our vtx::Entity objects to Lua?

Let’s recap a bit. We know that entities are “engine objects”, in the sense that they live in C++ and their lifecycles are managed by the Vortex Engine. What we want is to provide a lightweight object that Lua can interact with, but when push comes to shove, the native side will be able to leverage the full C++ interface of the engine.

Lua offers the concept of a metatable that helps achieve this. Metatables are can be associated to any table to provide special semantics to them. One special semantic we are interested in is the __index property, which allows implementing the Prototype design pattern.

I won’t go into details of the the Prototype design pattern works, but suffice it to say that whenever a function is called on a table, and the table does not have an implementation for it, the prototype will be responsible to service it.

This is exactly what we want. What we can do then is wrap our vtx::Entity instances in Lua tables and provide a common metatable to all of them that we implement in the C++ side. Even better, because of this approach Lua will take care of passing the Entity Table we are operating on as the first parameter to every function call. We can use this as the “this” object for the method.

Putting it all together, let’s walk over how entities expose the vtx::Entity::setName() function to Lua:

  1. From the native side, we create a metatable. Call it vtx.Entity.
  2. We register in this metatable a C++ function that receives a table and a string and can set the name of a native Entity. We assign it to the “set_name” property of the metatable.
  3. Whenever a script requests an Entity (instantiate, find), the function servicing the call will:
    1. Create a new table.
    2. Set the table’s metatable to vtx.Entity.
    3. Store a pointer to the C++ Entity in it.
  4. When a script invokes the Entity’s set_name function, it will trigger a lookup into the metatable’s functions.
  5. The function we registered under set_name will be called. We are now back in C++.
  6. The native function will pop from the stack a string (the new name) and the “Entity” on which the method was called.
  7. We reinterpret_cast the Entity Table’s stored pointer as a vtx::Entity pointer and call our normal setName() function, passing down the string.

Et voila. That is everything. The second image above shows in the console log how all this looks to a Lua script. At no point must the script developer know that the logic flow is jumping between Lua and C++ as her program executes.

We can also see in the screenshot how the Editor’s entity list picks up the name change. This shows how we are actually altering the real engine objects and not some mock Lua clone.

As I mentioned in the beginning of this post, developing a Lua binding for a large C++ codebase from scratch is a lot of fun. I will continue adding more functionality over the coming weeks and then we’re going to be ready to go back and revisit scene serialization.

Stay tuned for more!

Building the Engine Scripting API

Posted on by Ale

Last week when we left off, we were able to implement a Lua REPL in the Vortex Editor console. This week, I wanted to take things further by allowing Lua scripts to access the engine’s state, create new entities and modify their properties.

Scripting Interface to the Engine. A C++ cube entity is instantiated from Lua code that is evaluated on the fly in the console.

Scripting Interface to the Engine. A C++ cube entity is instantiated from Lua code that is evaluated on the fly in the console.

In order to get started, I added a simple single function: vtx_instantiate(). This function is available to Lua, but its actual implementation is provided in native code, in C++. The image above shows how we can use this function to add an entity to the scene from the console.

This simple example allows us to test two important concepts: first, that we can effectively call into C++ from Lua. Second, it shows that we are able to pass in parameters between the two languages. In this case, the single argument expected is a string that specifies which primitive or asset to instantiate.

With this in place, we can now move on to building a more intricate API that enables controlling any aspect of the scene, respond to user input and even implementing an elaborate world simulation.

Best of all, because the Lua VM is embedded into the engine, scripts built against the Vortex API will by definition be portable and run on any platform the engine runs on. This includes, of course, mobile devices.

The idea now is to continue to expand the engine API, developing a rich, easy to use set of functions. API design should prove an interesting exercise. Stay tuned for more!

Adding a Scripting Engine to Vortex

Posted on by Ale

This week I added scripting support to the Engine. I chose to go with Lua because of how easy it is to integrate into existing C/C++ codebases.

Initial integration of a Lua VM in the form of an updated Console

Initial integration of a Lua VM in the form of an updated Console

I’ve mentioned Lua several times before in this blog, but if you’re not familiar with it, it’s a great open source programming language developed at the Catholic University of Rio, Brazil (PUC Rio). It’s very easy to pick up.

Here’s a 10,000 feet view of the language, courtesy of Coffeeghost:

Lua cheatsheet by coffeeghost.

Lua cheatsheet by coffeeghost.

I’ve been interested in adding Lua scripting to the engine for a while now. I finally decided to take the step while I was revisiting serialization and a friend suggested going directly with Lua for the manifest file instead of JSON.

Moving from a “declarative” manifest to an “imperative” one might seem strange, however, it will give me the opportunity to start fleshing out the Lua-to-Engine interface that will later serve engine-wide scripting.

I am very happy with the way things turned out. In the image above you can see how I refactored the Vortex Editor console to now support a full Lua REPL.

Powered by the Lua Engine in Vortex, the console is no longer a place where the engine just prints messages, but rather a true editor shell with a direct interface to the engine. This is similar to what some popular 3D modeling software products do with Python.

I am excited about having Lua scripts as first class citizens in the engine. Expect to see much more Lua in this blog in the upcoming months!

Stay tuned for more!