Multi-Monitor with One screen rotated with Nvidia Drivers - Ubuntu 12.10

I have this monitor and computer sitting on my right for a while now, and as you can see, the screen is rotated so it's in portrait mode. Not only is it great for Tweetdeck, it's also awesome for reading various API docs while I'm working.

I'd tried a few times to set this up with the Nvidia driver, but could never get it to work, and would just give up and use the Nouveau driver, as it was very simple to set up the way I would like.

I decided to come back at this, and did some reading around, and discovered the following solution (I've since lost the link on the Ubuntu forum that pointed this out, if anyone finds it, please add to the comments).

1. Open Nvidia Settings and set up your monitors using Twinview as you would like them positioned, and hit apply.
2. Open the Displays application (the one in the System Settings application), select the window you want to rotate, change it's rotation in the rotate dropdown, and hit apply.
3. Go back to Nvidia Settings and save your configuration to your xorg.conf

That should be about it! Now you have 1 screen rotated!

JRuby, LWJGL & OpenGL - Getting Started with Shaders

In Part 2 we drew a triangle using a Vertex Buffer, and some basic shaders. While on the surface this can seem overtly complicated, it actually becomes the basis of a powerful OpenGL achitecture to enable you to leverage the GPU is a variety of very interesting ways without having to rely on the CPU.

We'll be look at the example show_triangle_vert_frag_offset.rb, which can be run from the command bin/triangle_vert_frag_offset.

This also comes from the "OpenGL's Moving Triangle" section of the Learning Modern 3D Graphics Programming online book.

With this code, we are going to take our original triangle, and we will make it move around a bit, and also change colour at the same time.

The clever thing about this is, we won't be changing the vertex data stored in the buffer, but will instead be manipulating it with Fragment and Vertex shaders. This almost feels like Uber-CSS over the top of HTML.

This is pretty powerful stuff, as we can let the GPU do a lot of the processing by using shader programs, and passing them attributes to control the overall affect that we want.

Let's look at our vertex shader offset_vertex.glsl

#version 330

layout(location = 0) in vec4 position;
uniform vec2 offset;

void main()
{
    vec4 totalOffset = vec4(offset.x, offset.y, 0.0, 0.0);
    gl_Position = position + totalOffset;
}

You can see we now have a uniform vec2 offset;. This defines a value that is going to get passed in from outside the Shader. It has the keyword uniform as the value stays the same on the same rendering frame.

The offset attribute will be expecting a vector with an x and y coordinate to be passed through (vec2) for the uniform value.

We convert the offset vec2 to a vec4, as you can't add a vec2 to a vec4. Then we can add these two together (GLSL will do vector arithmatic for you out of the box) to get our final gl_Position vector.

This means we can change the position of our triangle with relative ease, just by changing the offset of each of the vertices.

To do this from our JRuby code, is quite straight forward. The first thing we need to do is find out the location / position of the offset attribute. This is done through:

@offset_location = GL20.gl_get_uniform_location(@program_id, "offset")

Now we have the capability to change this value as we need to. In our display function, we have:

x_offset, y_offset = compute_position_offsets(time)

compute_position_offsets calculates x and y offsets based on the current time. (Check out the code for more details, it's just some fun trig). To set the uniform value, we then do:

GL20.gl_uniform2f(@offset_location, x_offset, y_offset)

That's it. The shader then does the rest of the work. Our triangle will now go round and round in a circle.

We do similar things to change the colour of the triangle as it goes around in a circle. Here is our fragment shader:

#version 330
out vec4 outputColor;
uniform float fragLoopDuration;
uniform float time;

const vec4 firstColor = vec4(1.0f, 1.0f, 1.0f, 1.0f);
const vec4 secondColor = vec4(0.0f, 1.0f, 0.0f, 1.0f);

void main()
{
    float currTime = mod(time, fragLoopDuration);
    float currLerp = currTime / fragLoopDuration;   
    outputColor = mix(firstColor, secondColor, currLerp);
}

You can see we can define constants with the const keyword. This sets up two colours to mix between, in this case, white and green.

We have two uniform attributes to in this time, fragLoopDuration the loop duration of the colour change, and time, how many seconds have passed since the application began.

mix is a GLSL function that blends two colours together based on the third float that is passed through, and gives us the slow fade between the two as the time value changes.

Setting this up is almost exactly the same as before. We will set the fragLoopDuration in our init_program method, as it never changes across the execution of our code.

frag_loop_location = GL20.gl_get_uniform_location(@program_id, "fragLoopDuration")
@frag_loop = 50.0
GL20.gl_uniform1f(frag_loop_location, @frag_loop)


For the time uniform attribute, we have some code that tracks how much time passes between frames and we just add it all up, and then set the uniform attribute with gl_uniform like usual:

current_time = Sys.get_time
@elapsed_time += (current_time - @last_time)
@last_time = current_time
time = @elapsed_time / 1000.0

GL20.gl_uniform1f(@time_location, time)

That's the basics of using shaders with vertex data. We now have a triangle that goes around in a circle!

JRuby, LWJGL & OpenGL - Drawing a Triangle

In Part 1 we created a window, nothing too fancy. Now we get to actually display a triangle.

Just to follow along as well, I'm moving through the Learning Modern 3D Graphics Programming online book to learn OpenGL (again), so the OpenGL examples I will be displaying will be ports from the code that that is providing. I would suggest for the complete theory behind this code, read the linked section before going through the JRuby code. I expect my explanations to only be commentary on the information already provided in that series and discussion on some of the finer points on JRuby and Java library integration as well.

If you have any questions, please feel free to ask, but be aware, I'm very new to OpenGL, so writing this series is very much part of my learning experience. However, I will attempt to answer the best way I can. On the other hand, if you find anything wrong with what I'm written, please point it out so it can be corrected.

This example is from Hello, Triangle!.

The full code can be seen on Github here.

To run this example use bin/triangle

Making Vertex Data Available to OpenGL

When I first did OpenGL back in University, we used the glBegin() and glEnd() paradigm. This was definitely far easier that the more modern APIs, as it was very clear and easy to draw a simple polygon on the screen (example). However, it did mean more computation was occurring on the CPU and a larger use of the system RAM than the newer APIs. The newer APIs, while (far?) more complicated, shift much of the work to the GPU and also provide a far more flexible implementation. I liken it to working with HTML and tables back in early HTML days. Sure it worked, but CSS and semantic markup gives a clear separation and creates far more flexible implementation options (at least in theory ;) ).

So we have some basic vertex information to display a right angle triangle:

@vertex_positions = [
    0.75, 0.75, 0.0, 1.0,
    0.75, -0.75, 0.0, 1.0,
    -0.75, -0.75, 0.0, 1.0,
]

Each line of this array define the x, y and z coordinates of our triangle. You will notice there is a fourth coordinate (1.0) on each line. This defines the clip space. For now we'll just say this just means that the vertexes you see in the window has to have values between -1 and 1 on the x, y and z axis. More than that will render outside of the window.

As discussed previously, in old school OpenGL you would just look through this list of vertexes and say draw a triangle here, however, this is no longer the case!

I feel like modern OpenGL is almost like a database - you put some data into it, and have an id to reference that data that was placed in. Then you can work on that data that is stored on the GPU through some other techniques (that we will look at in a minute) via that id. This seems to be a concept that is used across the board.

The code that inserts our vertex data into the GPU can be seen in the method init_vertex_buffers

@buffer_id = GL15.gl_gen_buffers
GL15.gl_bind_buffer(GL15::GL_ARRAY_BUFFER, @buffer_id)

So first thing we do, is we generate an id for the vertex buffer, which is where we will store our vertex data. (gl_gen_buffers). Then we tell OpenGL, hey, this is the buffer we want to work with for the moment through gl_bind_buffer, passing in the specific @buffer_id we generated before. We also tell OpenGL that the buffer we are working with an GL_ARRAY_BUFFER, so it knows what data to expect.

In case you aren't aware, JRuby will convert Java static constants to Ruby constants, so we can access these static fields very easily.

To pass the vertex data into the new vertex array buffer, LWJGL has us use it's BufferUtils class to create a NIO buffer, and push the data into it, like so:

float_buffer = BufferUtils.create_float_buffer(@vertex_positions.size)
float_buffer.put(@vertex_positions.to_java(:float))
#MUST FLIP THE BUFFER! THIS PUTS IT BACK TO THE BEGINNING!
float_buffer.flip

A couple of interesting notes:

1. You will noticed the .to_java(:float). That is the JRuby code for converting a Ruby array to a Java array. Passing in :float tells it to make it an array of primitive floats.
2. The .flip at the end. This is very important (and took me a day to work out, as I'm not familiar with NIO buffers). Here is a great article that explains it more detail, but essentially the buffer tracks where it is at, and flip sends it back to the beginning. Without this, no data goes to our Vertex Array, and nothing happens!

GL15.gl_buffer_data(GL15::GL_ARRAY_BUFFER, float_buffer, GL15::GL_STATIC_DRAW)

This then pushes our vertex data to the bound vertex buffer.

#cleanupGL15.gl_bind_buffer(GL15::GL_ARRAY_BUFFER, 0)

After we are done, we tell OpenGL not to be bound to any array buffers (0 works like NULL in OpenGL land). This could be considered optional, but ensures that weird things don't occur.

Now all we have to do is actually write the code that makes the data that displays the triangle!

Telling OpenGL how to Render the Vertexes

So now we have the vertex data stored on the GPU, we have to tell it how to render it, and to do that, we have to build a program of a couple of different types of shaders. Think of Shaders kind of like the CSS of HTML. They simply work on the existing data in the GPU and tell it how it to render (although that's a bit of an over simplification).

First thing, we'll write a simple vertex shader in GLSL, the language for writing shaders. This specifies to the GPU where the vertexes actually are from the data you entered earlier. We'll just say that it's correct and basically pass it through.

#version 330
layout(location = 0) in vec4 position;
void main()
{
    gl_Position = position;
}

Then we write a fragment shader to it what colours to make things. We'll just make everything white for convenience.

#version 330
out vec4 outputColor;
void main()
{
    outputColor = vec4(1.0f, 1.0f, 1.0f, 1.0f);
}

To make life easier for myself, I wrote a quick little create_shader function, that loads the shader from a file loads it into memory and compiles it. I just want to make note of one line:

puts file_name, GL20.gl_get_shader_info_log(shader_id, 200)

Without this, you have no idea why your shader fails (if it does). Mine was failing, and I didn't realise it until I looked deeper. (Version 330 of GLSL wasn't supported on my Ultrabook on Linux with Mesa. I had to switch to my main laptop with the Nvidia graphics card)

Make sure to look at the output logs!

Creating the program to define how our data is output on the screen, is quite similar to what we did before. We generate an id and then link the shaders to the program like so:

vertex_shader = create_shader(GL20::GL_VERTEX_SHADER, 'basic_vertex.glsl')
frag_shader = create_shader(GL20::GL_FRAGMENT_SHADER, 'basic_fragment.glsl')

@program_id = GL20.gl_create_program

GL20.gl_attach_shader(@program_id, vertex_shader)
GL20.gl_attach_shader(@program_id, frag_shader)
GL20.gl_link_program(@program_id)
GL20.gl_validate_program(@program_id)

puts "Validate Program", GL20.gl_get_program_info_log(@program_id, 200)


A few things to note:

1. create_shader returns the id of the shader. Much like a database, you reference everything you are creating using OpenGL by the generated ID.
2. We attach the shaders to the program by the id we have for that program.
3. gl_link_program essentially turns the program we're creating and makes it able to executedon the GPU.
4. We also get the program info log with gl_get_program_info_log much like we did with the shader. Make sure to check those logs!

Finally, we can get to our display loop, which you can see in the display method.

#set the colour to clear.
GL11.gl_clear_color(0.0, 0.0, 0.0, 0.0)

We clear the page with a nice transparent colour.

GL11.gl_clear(GL11::GL_COLOR_BUFFER_BIT)

This clears the screen, so we can redraw with impunity.

GL20.gl_use_program(@program_id)

Tell it to use our shaders

GL15.gl_bind_buffer(GL15::GL_ARRAY_BUFFER, @buffer_id)

This tells it to use our vertex buffer

GL20.gl_enable_vertex_attrib_array(0)

Tells it to pass each value of the vertex buffer to the first argument of our vertex shader.

GL20.gl_vertex_attrib_pointer(0, 4, GL11::GL_FLOAT, false, 0, 0)

This tells OpenGL how the structure of the vertex buffer data matches is structured. Here we are saying that we have an array of floats, and every 3 elements is a x, y and then z member, with the fourth defining the clip space, as we saw earlier. (0 is the start, 4 is the size).

GL11.gl_draw_arrays(GL11::GL_TRIANGLES, 0, 3)

DRAW THE TRIANGLE! 0 is the start of the array of vertices you want to draw, and 3 is the number of indexes you want to process - in this case 3 to make a triangle.

GL20.gl_disable_vertex_attrib_array(0)GL20.gl_use_program(0)

Finally we use the magic OpenGL 0 to clean things up again, and we are done!

We now have this amazing triangle.

Creating a window with LWJGL and JRuby

This is the most basic of steps before getting anything done with LWJGL and JRuby, so I figured I would document it to help people with the initial hurdles.

Before doing anything with LWJGL and JRuby, you need to set things up correctly so that LWJGL can find its native extensions. The easiest way I found to do this was the following (tested on Linux):

1. Install JRuby with RVM
2. Download LWJGL
3. Put lwdgj.jar in ./lib/java
4. Unzip the native extensions (flatten out the directory, ignore solaris) and put them into ./lib/java/native
5. Create a .rvmrc in the root of the dir with the following code in it:

export JRUBY_OPTS="-J-Djava.library.path=lib/java/native"

This sets the JRUBY_OPTS environment variable, which tells JRuby to append these arguments to all JRuby operations. This obviously doesn't work for deployment, but during development it makes thing very handy.

Now to write some Ruby code to get a window up and running!

First we need to require java, and the lwjgl jar file:

require 'java'
require 'java/lwjgl.jar'



Drawing the window is now very straightforward:

java_import org.lwjgl.opengl.Display
java_import org.lwjgl.opengl.DisplayMode
#
# Just a basic display using lwjgl
class OpenGL::BasicDisplay
   # initialise
   def initialize
      Display.display_mode = DisplayMode.new(800, 600)
      Display.create

      while(!Display.is_close_requested)
         Display.update()
      end

      Display.destroy

   end
  
   def self.start
      OpenGL::BasicDisplay.new
   end

end

We can then write a little bin file to get this to run:

OpenGL::BasicDisplay.start



And there we go, we have a window!

Next, we'll start to write some OpenGL!

The full source can be dowloaded from GitHub

Writing a Game: Feels like I'm Going Around in Circles...

Warning: This is going to be far less technical, and more about the journey of learning that I'm enjoying.

As per my previous blog post, and from several tweets I've posted, I've been slowly plodding away on doing some game development in my spare time and in my holiday break. I originally started doing this because I've always loved games, and was deeply inspired by the indie game development scene and some of the incredible ideas that were coming out of that.

That also being said, I was really keen to try a new programming challenge. Something totally outside of my usual purview, and doing game programming seemed like a perfect fit for that, as it sat so far outside my usual environment of building web based business applications. What I didn't realise was how much it was going to push me in both directions that we new and also topics that I've learnt (far?) in the past and unfortunately forgotten much about.

One aspect that has totally blown me away was the architectural idea of "Entity Systems" (ES) for game development. (I'm not going to explain it here, there are way too many good articles on this, but if you want to read more click the link above). For what is such a relatively new idea in software development (< 10 years old if what I believe is correct), it fits so incredibly perfectly into how to write some insanely flexible game architectures, I have been rolling it around inside my head for other ideas of where it can be applied.

Since the last blog post, I refactored my entire code base into an ES architecture, and that was a fair bit of a mental shift away from a traditional OO model, but the more I use it, the more it just makes an incredible amount of sense. From the screen shot below you can also see I started adding some graphics to the page too, to make a little nicer to look at.

Some of the basics (and beginnings) for this came from the awesome SpriteLib project, but I also decided to start doing some drawing and try my hand at doing some of my own pixel art. Most people probably don't know this, but I used to draw a lot. I even spent a year in design school... but to be honest, I was nothing special at it and ended finding a real passion for programming. But doing this game programming stuff has got me sketching out little characters and whatnot again, so it's been a real pleasure to rediscover this passion for drawing that I haven't touched in probably over a decade.

My next step in the game is to do some basic physics, nothing too special, just a player and some jumping to move around the world. Suddenly I'm back in high school doing kinematics and dynamics! I've still got all my old maths and physics notes from high school, so I'm pouring over that and also reading some great websites as well, and the concepts and equations are slowly coming back to me.

While doing that though, I started to look at the development cycle of Slick2d and the fact that it seemed like many people (including the original developers themselves) where moving over to libGDX. Comparing Slick2D and libGDX, you can clearly see that Slick2d has a far lower barrier to entry that libGDX. From my review of libGDX, having some knowledge of OpenGL and how 3D graphics work (even on a 2D project) is going to be a huge boon. So now I'm wondering - do I switch to libGDX? It looks more active, performant and gives far more options for distribution (desktop, web, android).

Again, I'm back to looking at old code from University - specifically my 3D programming class, and trying to make heads and tails of the OpenGL code I had previously written (in C++ no less). It vaguely makes sense, but there are concepts there I have either partially or totally forgotten (what is a Quad again? And different projection modes? no idea). Therefore, I'm going to keep playing with OpenGL some more, just to get that basic foundation under my feet, starting with the very low level lwjgl project with JRuby, and just keep going forward one step at a time, and then probably refactor my code again into libGDX once I'm done. While I don't think I have to really know OpenGL to use libGDX, I always feel more confident having the base understanding under my feet. That and I think it's a direct path to re-learning concept as vectors and 3d math with matrices that really is core to any sort of advanced game programming (yeah, forgot most of that too).

So overall, it feels like I am going around in circles a bit - but not in a bad way. I'm finding old things that I used to love doing (drawing and math) and rediscovering the joy I had in doing them, and although it's frustrating knowing that you have forgotten the knowledge you wish you still knew, it's a delight to reopen these memories and play with them once again.