RSS

Author Archives: Alex

Improving Vortex’s Smart Pointers

In out last post, we talked about the shift we are currently making in Vortex, moving towards reference-counted resource management.

In this context, one of the problems we mentioned was the fact that due to the fact that C++ does not provide out-of-the-box type covariance/contravariance like C#, we ended up with some rather uncomfortable constructs for dealing with polymorphic data types.

Now, one of the great features of C++ is that the language is super-extensible. You can literally change the way many things work in C++ and, it turns out that after some research and experimentation (and a little C++ magic too), we found a way to implement C# covariance/contravariance for our custom smart pointers!

The latest improvements enable the following features:

  1. Implicit casts from a smart pointer (to an instance) of a derived class to a smart pointer of a base class.
  2. Static casts from a smart pointer (to an instance) of a base class to a smart pointer of a derived class.
  3. Compile-time errors from attempting invalid casts.

The following code fragments illustrate these points. These are taken directly from out test suite, with minor adaptations to simplify readability.

Fragment #1: Implicit cast from derived instances to base class instances:

ref_ptr<TransformNode> xformNode(new TransformNode());
ref_ptr<Node> node = xformNode; // Valid: implicit cast

Fragment #2: Static cast from a base class instance to a derived class instance. Note we called our static cast operator “ptr_cast”.

ref_ptr<Node> node(new TransformNode()); // Storing as a "more general" pointer
ref_ptr<TransformNode> pxformNode = ptr_cast<TransformNode>(node); // Valid: explicit cast.

Fragment #3: Finally, the following code is invalid, and will trigger a compile-time error:

ref_ptr<SimpleSceneGraph> sg(new SimpleSceneGraph());
ref_ptr<Node> node = ptr_cast<Node>(sg); // Illegal conversion: types are not related

We are very happy with the result and are looking forward to continue migrating the Vortex codebase to our new smart pointers. Programming with Vortex just got a whole lot easier and safer.

 
Leave a comment

Posted by on October 25, 2011 in C++, Vortex-Engine

 

Towards Reference-Counted Resource Management in Vortex

Now that we have advanced support for a dual rendering pipeline in Vortex, we are now shifting our attention into a completely different topic. This time we are looking into overhauling the engine’s inner resource management.

To this purpose, we are conducting experiments shifting resource and scene management from raw pointers to smart pointers. The idea of using smart pointers is to enable each resource to be automatically deleted once it is no longer referenced by any component or object. This is not to be confused with Garbage Collection. The main difference consists in that smart pointers allow us to simplify memory management without sacrificing deterministic object destruction and without incurring into the memory and CPU overhead of having the Garbage Collector kicking in to rearrange the application’s Heap at any time.

So far, this has proven a very interesting experience. We’ve bumped with some unexpected issues along the way, but we’ve also run into a couple of nice surprises. Among the problems encountered, there’s the fact that since C++ does not support out-of-the-box covariance/contravariance (in the sense C# does), exposing the polymorphic behavior of the scene objects became a little more involved that what it used to be when using raw pointers.

On the bright side, using smart pointers dramatically improves some of the roughest edges of the API. Just to pick a simple example, let’s consider image lifecycle management. Without smart pointers, Texture creation from images in Vortex might look like this:

void loadTexture()
{
  Vortex::Image* pimg = Vortex::Image::loadFromFile("./resources/texture.png");

  // Create Texture from image data...

  // loadImageFromFile transfers pointer ownership,
  // we must therefore not forget to delete the image:
  delete pimg;
}

Using Vortex’s smart pointers, however, we can forget about the explicit delete operation:

void loadTexture()
{
  ref_ptr<Vortex::Image> pimg = Vortex::Image::loadFromFile("./resources/texture.png");

  // Create Texture from image data...

  // Image object ownership is now managed by the
  // smart pointer. The pointer releases the data
  // upon exiting this scope (unless the pointer is copied)
}

ref_ptr<T> is the typename we’ve given to our in-house smart pointer implementation. These allow us to easily cast inside type hierarchies as well as dropping into the raw pointer when needed, allowing us to dodge the performance tax in performance-critical code.

So far the results seem promising. We will continue experimenting moving the rest of the codebase to smart pointers and hopefully we will be able to include this along with the dual rendering pipeline in the very next version of Vortex: Vortex 2.0.

 
Leave a comment

Posted by on October 20, 2011 in C++, Vortex-Engine

 

Dual Pipeline support in Vortex Engine

Dual Pipeline support in Vortex Engine

Continuing with the news related to the Vortex Engine, I’m glad to announce that the next version of Vortex will come with two separate rendering pipelines: a fixed rendering pipeline and a programmable rendering pipeline that supports custom shader programs.


Comparison of the Rendering Pipelines available in Vortex Engine. The image on the left represents the Fixed Pipeline. The image on the right represents the Programmable Pipeline (click to enlarge).


Supporting two pipelines enables the developer to use the most appropriate rendering pathway for the target device’s capabilities.

Newer video cards for Desktop computers and the latest mobile devices can be programmed for though the programmable pipeline, whereas older devices (such as the original iPhone and the iPhone 3G) can still be supported through the fixed pipeline.

Anyone in the field of Graphics will be able to tell that the fixed pipeline will not be able to provide the exact same visual capabilities as the programmable pipeline, however, it should be less resource intensive and thus more suitable for some older devices.

The programmable pipeline, on the other hand, will allow Vortex applications to leverage the hottest techniques in Computer Graphics such as Bump Mapping, Specular Mapping, Blur and Depth of Field (among others), dramatically improving the visual detail.

This list provides, at a glance, the criteria for selecting the appropriate Vortex rendering pipeline depending on the target device:

  • OpenGL 1.1 / OpenGL ES 1.1 devices -> Use Vortex’s Fixed Pipeline
  • OpenGL >= 2.0 / OpenGL ES 2.0 devices -> Choose between Vortex’s Fixed Pipeline and Vortex’s Programmable Pipeline (recommended).

Needless to say, we’re very excited about the possibilities that this brings to the table and are very eager to see to what extents can the Vortex Engine be now pushed.

In the picture above you can see a sample comparison between the lightning model of the fixed pipeline (hardware-accelerated Gouraud Shading) and the one that the programmable pipeline offers (hardware-accelerated Phong Shading with specular reflections in this case).

This is just the tip of the iceberg. Stay tuned for more updates on the Vortex Engine!

 
Leave a comment

Posted by on October 14, 2011 in C++, Vortex-Engine

 

Steve Jobs 1955-2011



 
Leave a comment

Posted by on October 6, 2011 in Uncategorized

 

Md2 Models and Vortex Scene Graphs

Md2 Models and Vortex Scene Graphs

Someone once said there is no building the engine without building a game. There is nothing like actually using your own tools in order to pinpoint the weak spots of the Engine.

Perilith Knight with the "knight skin".

Using Vortex to build different 3D applications, I’ve noticed that it may, at times, be a bit more verbose than one would expect from an Object Oriented C++ API.

One of the roughest edges in Vortex was Md2 model animation support. Until last week, integrating animated models into a Scene Graph was unnecessarily complicated. Fortunately, this is no longer the case. Adding an animated Md2 model to a Scene Graph is now as easy as adding any other node type.

Controlling animations and skins of Md2 Models programatically is now a breeze in Vortex.

Here are some captures from the famous Perilith Knight model. I may upload a YouTube video showing the smooth animation at some point.

 

Perilith Knight using the "evil skin".

Perilith Knight using the "ctf_r skin".

Stay tuned for more updates on the Vortex Engine!

 
Leave a comment

Posted by on October 3, 2011 in C++, Vortex-Engine

 

Herb Sutter’s “Why C++?” Talk Sum-up

This is a sum up of the points I believed to be the most interesting in Herb Sutter’s recent “Why C++?” talk. The video was published on Microsoft’s Channel9, but I have also embedded it here for convenience.



The talk starts with a recap on C++’s history. Then Herb Sutter talks about the rise of “Coffee-based” languages (i.e. managed languages) and how for about a decade (from 1999 to 2009) the industry seemed to accept trumping performance for productivity. Mr. Sutter clearly states he believes this is a perfect valid point, however, he notes that the opposite is also true: trumping productivity for performance is valid (and necessary) in many scenarios. The talk focuses on three such scenarios: Mobile, Datacenter and Desktop.

Managed Languages

Before moving into these scenarios, the talk focuses on the subject of managed languages. Mr. Sutter referenced Windows “Longhorn”, and its attempt to push managed languages as a first class citizen in Operating Systems. He evoked a quote of the time stating that “All new Windows APIs would be managed”.

However, as the OS was being developed, they realized that managed languages didn’t have the performance, deployability or serviceability (among others) required to develop an Operating System. That’s why Windows nowadays is all C and C++ and will probably stay that way.

Furthermore, demanding applications, like vision Apps running on mobile devices and Kinect applications, further emphasize the need for high performance in languages. According to Mr. Sutter, these applications are breathing new life into performance languages, and have been triggering a “C++ Renaissance”.

“Was .NET All a Mistake?” Of course it wasn’t, Mr. Sutter states. “But it can’t do everything C++ can, just like C++ can’t do all .NET can”. Managed languages were the hammer that made everything look like a nail during the last decade. Today, the focus is on high demanding applications that need performance. The “king” is Performance per Watt, per Cycle, per Transistor: Performance per Dollar.

Mr. Sutter takes some time to explain the drawbacks he sees on managed language performance. He quotes several reasons including: Garbage Collection, JIT services, lack or no control on caching and memory layout, etc. Mr. Sutter states that some of these drawbacks can be helped, but that programmers basically have to fight against the language in order to do so.

According to Mr. Sutter, except for the last decade, performance has always been the main reason to pick a programming language, and starting two years ago, it is now again.

Mobile

“Let’s talk about Mobile”. Mobile devices are all about battery, size and bigger experiences on smaller hardware.

Smart Phones are small computers. If we look at the main platforms today: iPhone is programable in Objective-C, C and C++. All native code. Android supports Java, but the Native Development Kit is essential when we need performance.

Mr. Sutter recalls it has not always been this way. The original iPhone preached Javascript development, but for its second version it went all native. Android started preaching Java development, and now supports native development as well. Windows Phone 7 started off with .NET, but by looking at the other two platforms, one can guess what will happen in the near future. The trend is to go all native.

What happens with Web Apps? According to Mr. Sutter they are and will still be there, but will be used as a backup when a native App does not exist for the task. We can look at Google+ for an example. Everyone uses the native App, and “default” to the Web App when there is no native App for the platform being used. Why? Mr. Sutter says it is because of their Performance.

Datacenter and Desktop

What happens in the Datacenter? Citing a study, Mr. Sutter claims that the biggest cost is Hardware and Power. These account for 88% of the total cost of running a Datacenter. Performance/Watt has a direct impact on this, and it can be improved by having more efficient programs.

Mr. Sutter states that if we could reduce Time and Space requirements, we could improve efficiency. Efficiency is “king” for managing the costs of the Datacenter.

Quoting Bjarne Stroustrup: “My biggest contribution to the fight against global warming is C++’s efficiency: Just think if Google had to have twice as many server farms! Each uses as much energy as a small town. And it’s not just a factor of two… Efficiency is not just running fast or running bigger programs, it’s also running using less resources.”

Regarding the Desktop Mr. Sutter notes how being efficient on power and heat also matters in the Desktop. I believe this case is particularly true for laptop devices.

C++11 and the next Decade

Mr. Sutter states: “C++11 is upon us”. Not only does C++11 have a lot of new features, but Mr. Sutter emphasizes how these will be available and also required soon. Programmers will be required to use the new Features in C++11 in order to take advantage of new frameworks (like C++ AMP). “C++ matters again as a first class system”.

The talk concludes by presenting view on what will come after the C++ Renaissance decade (2009-2019). Moore’s law is going to end. It will not happen over night, though, according to Mr. Sutter. The industry will be able to see the end coming when the end of we start obtaining diminishing returns.

Efficiency-oriented languages have a bright future, and they are going to stick around.

To end his presentation, Herb Sutter states that “The answer to the question <<Why C++?>> is <<Why, C++, of course!>>.”

 
Leave a comment

Posted by on September 8, 2011 in C++, Video

 

Support std::endl in custom C++ streams

I’ve been working on a custom Logging system for Vortex. After thinking about it for a while, I decided to go with a stream-based API for logging. I figured this would allow me to both: provide output capabilities for custom data types and also to format log messages “on the fly”.

This is where I started to play with the idea of being able to handle std::endl just like the standard streams.

Even though I’ve been a C++ programmer for a while (10 years since my first “hello world” program), I realized I actually didn’t know the type of std::endl. I looked around until I finally found something at Stack Overflow. Much to my surprise, it turns out std::endl is a function template! The correct way to handle it is, in the stream insertion operator (<<), call the passed-in endl function with the stream you are outputting to as the parameter.

// The code in Vortex is more involved,
// but this is the general idea
Logger& Logger::operator<<(std::ostream& (*f)(std::ostream&))
{
    f(std::cout);
    return *this;
}

For Vortex, it was enough just to support std::endl taking and receiving std::ostream instances, it might be different in your code, depending on the degree of customization you want to provide.

Now that everything’s in place, we can handle std::endl just like any other stream:

Logger debug;
debug << "running loggerTest() in file.cpp:12" << std::endl;
 
Leave a comment

Posted by on August 11, 2011 in C++, Vortex-Engine

 

QuakeCon 2011 Keynote

John Carmack, founder and lead developer of Id Software opens the 2011 QuakeCon conference.

In this Keynote, John Carmack discusses some implementation details of the engine powering Id’s next game: Rage. He addresses the issues faced while developing the engine for the PC, the XBox 360 and the PlayStation 3 and the problems they had to address while trying to maintain a steady 60 FPS while managing the game’s huge data sets.

The talk also emphasizes the importance of using Static Analysis tools to find bugs in large code bases. Some tools mentioned include “Microsoft Analyse”, “PVS Studio” and “PC lint”.

John Carmack also shares his views on in-game script interpreters. He mentions these are hard to debug and develop as well as the fact that they significantly impact performance and game quality. He recommends not going with dynamic languages, but instead having a strong type system and a very restrictive language. His views are that it is better to find bugs “as early as possible”, ideally at script compile time. He sees the best direction to go is with a Java-like subset of C++: a very strict, C or C++-ish language.

Finally, he closes the Keynote by looking back into the “early days” of game development and comparing that to the process studios have to go through today. He also announces the upcoming release of the Doom 3 source code later this year and recommends getting a copy of Doom 3 from Steam.



YouTube link.

 
2 Comments

Posted by on August 8, 2011 in Video

 

The L+ Programming Language

Continuing with the L+ saga, after the announcement of the free compiler release and the Fibonacci number example, yesterday we added to the project a new piece of documentation: The L+ Programming Language guide.

This guide aims to introduce the L+ language to developers. It provides the keywords reserved by the compiler, explains the data types available, the language’s constructs and the functions included in the standard library.

The document also contains the full Grammar for the L+ language, something you might find interesting whether you are interested in compiler development or just getting started with Language Theory.

The document was produced using LaTeX and the full source code has been merged into the source tree of the L+ compiler.

-> Download the L+ Programming Language guide.

If you have any comments or questions, don’t hesitate to leave them below.

 
Leave a comment

Posted by on July 16, 2011 in L+

 

The Fibonacci Number: a new L+ example

I’ve just added a new example to illustrate the capabilities of the L+ language: fibo.lp

If you read our last post then, as you might recall, last Saturday we released a Free (as in Freedom) compiler called the “L+ compiler”. This program was a mandatory assignment we had to develop for a Compiler course at the Catholic University of Uruguay back in 2006.

When we introduced the compiler, we featured a simple “Hello World” program that illustrated a basic example of L+, our language. The example was very basic though, so I wanted to raise the bar and show what our little language can do.

This new program reads a number from stdin, calculates its corresponding Fibonacci number and prints the results to stdout.

The good thing about this example is that it illustrates most of L+ in action, including: I/O, function declarations, recursion, branching and string processing.

Here’s the full source code for the example. You can find it at the L+ project homepage and you also get it when you checkout the source code from Subversion.

// Calculate the Fibonacci number for n
int fibo(int n)
{
	if (n == 0)
	{
		return 0;
	}

	if (n == 1)
	{
		return 1;
	}

	return fibo(n-1) + fibo(n-2);
}

// Program fibo.lp
// Read a number, calculate the Fibonacci number for said number
// and print a message with the result.
void main()
{
	print("Enter number to calculate the Fibonacci number for:");
	string nStr = read();

	int f = fibo(toInt(nStr));
	string s = toString(f);

	print("The Fibonacci number for " + nStr + " is: " + s);
}

From a compiler developer’s point of view, I can hardly tell you what it felt like seeing our compiler handling a source code as complex as this one and even catching some grammar errors every time I forgot to add a semicolon to a line.

I really encourage you to test this program, but in case you don’t, here’s the output the L+ compiler produces while compiling this program (yeah, it’s really verbose):

[ale@syaoran fibo]$ LPC_HOME=/opt/lpc /opt/lpc/bin/compiler.sh fibo.lp
Cleaning up directory for new build...
Checking for necessay files...
compiler... assembler... L+ standard library... Java...
Looking for L+ source file

Starting the build process...
Invoking the L+ compiler...
Batch mode. taking as input: fibo.lp
Parsing successful
Pretty Printing result. Saving as "out.txt"
Begin semantic validation...
Optimizing...
Pretty Printing optimized code. Saving as "opt.txt"
Compiling into file "comp.j"
Optimizing object code. Saving as "compopt.j"
Freeing used memory...

Invoking the Jasmin JVM assembler...
Generated: Main.class
Build process completed successfully!
type 'java Main' to run the compiled program.

Finally, this is the output produced from running the program with an input value of “10″:

[ale@syaoran fibo]$ java Main
Enter number to calculate the Fibonacci number for:
10
The Fibonacci number for 10 is: 55.0
 
Leave a comment

Posted by on July 11, 2011 in L+